2024-04-13T04:43:42.635 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-13T04:43:42.636 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --name vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} --archive /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333 --verbose -- /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/orig.config.yaml 2024-04-13T04:43:42.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T04:43:42.776 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333 branch: wip-vshankar-testing-20240412.042930 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7654333' kernel: flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240412.042930 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240412.042930 sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7517 sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 sleep_before_teardown: 0 subset: 55/1024 suite: fs suite_branch: wip-vshankar-testing-20240412.042930 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDL696pxUtIvSGsIC5IxNAufcSOOrHmaypFhy12QJcliW3277EYfme8HPSz4bhfjHSWFpZvvsGViIy4z7lNnLJs= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHIz3tzAlZi3o6XF3NbEaoHuU43YGUfsQ23hf5io0UwzEQQzcIUp+cgc0KV9Qy2JpI/rXST96idmIhDJXVrDXsY= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ8rd9bLFYlXuasK4uvpAMOtj6Mb1e9zAaiS8Ec5MQ8iMZrrPVl4kEHlM0/Zm4qyDwYipskKnBus6+r+G07mIE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-12_17:34:28 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-13T04:43:42.776 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa; will attempt to use it 2024-04-13T04:43:42.776 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_218a691417ef57c5d22a8d5cf3c5f41f7b542838/qa/tasks 2024-04-13T04:43:42.777 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-13T04:43:42.778 INFO:teuthology.task.internal:Checking packages... 2024-04-13T04:43:42.802 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '218a691417ef57c5d22a8d5cf3c5f41f7b542838' 2024-04-13T04:43:42.802 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-13T04:43:42.803 INFO:teuthology.packaging:ref: None 2024-04-13T04:43:42.803 INFO:teuthology.packaging:tag: None 2024-04-13T04:43:42.803 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240412.042930 2024-04-13T04:43:42.803 INFO:teuthology.packaging:sha1: 218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T04:43:42.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240412.042930 2024-04-13T04:43:42.939 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2884.g218a6914 2024-04-13T04:43:42.941 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-13T04:43:42.947 INFO:teuthology.task.internal:no buildpackages task found 2024-04-13T04:43:42.947 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-13T04:43:42.985 INFO:teuthology.task.internal:Saving configuration 2024-04-13T04:43:43.004 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-13T04:43:43.009 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T04:43:43.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 04:37:44.318644', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDL696pxUtIvSGsIC5IxNAufcSOOrHmaypFhy12QJcliW3277EYfme8HPSz4bhfjHSWFpZvvsGViIy4z7lNnLJs='} 2024-04-13T04:43:43.055 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 04:37:44.316080', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHIz3tzAlZi3o6XF3NbEaoHuU43YGUfsQ23hf5io0UwzEQQzcIUp+cgc0KV9Qy2JpI/rXST96idmIhDJXVrDXsY='} 2024-04-13T04:43:43.077 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 04:37:44.317472', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ8rd9bLFYlXuasK4uvpAMOtj6Mb1e9zAaiS8Ec5MQ8iMZrrPVl4kEHlM0/Zm4qyDwYipskKnBus6+r+G07mIE='} 2024-04-13T04:43:43.077 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-13T04:43:43.083 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-13T04:43:43.083 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-13T04:43:43.083 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-13T04:43:43.083 INFO:teuthology.run_tasks:Running task console_log... 2024-04-13T04:43:43.182 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f44dd607160>, signals=[15]) 2024-04-13T04:43:43.183 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-13T04:43:43.189 INFO:teuthology.task.internal:Opening connections... 2024-04-13T04:43:43.189 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T04:43:43.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:43:43.297 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-13T04:43:43.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:43:43.387 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-04-13T04:43:43.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:43:43.473 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-13T04:43:43.485 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2024-04-13T04:43:43.503 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2024-04-13T04:43:43.503 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2024-04-13T04:43:43.560 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2024-04-13T04:43:43.560 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="9" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="9" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el9" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:LOGO="fedora-logo-icon" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-13T04:43:43.561 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-13T04:43:43.562 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2024-04-13T04:43:43.595 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-13T04:43:43.613 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-13T04:43:43.613 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-13T04:43:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-13T04:43:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-13T04:43:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-13T04:43:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-13T04:43:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T04:43:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-13T04:43:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-13T04:43:43.671 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-13T04:43:43.686 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-04-13T04:43:43.703 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-04-13T04:43:43.704 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-04-13T04:43:43.760 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-04-13T04:43:43.760 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-04-13T04:43:43.760 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-13T04:43:43.761 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-13T04:43:43.762 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-04-13T04:43:43.787 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-13T04:43:43.796 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-13T04:43:43.802 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-13T04:43:43.802 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2024-04-13T04:43:43.805 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-13T04:43:43.807 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-04-13T04:43:43.823 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-13T04:43:43.829 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-13T04:43:43.829 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2024-04-13T04:43:43.862 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-13T04:43:43.864 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-04-13T04:43:43.895 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-13T04:43:43.971 INFO:teuthology.run_tasks:Running task kernel... 2024-04-13T04:43:43.990 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-13T04:43:43.990 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, timeout 300 2024-04-13T04:43:43.990 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-13T04:43:43.991 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-13T04:43:43.991 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-13T04:43:44.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T04:43:44.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T04:43:44.008 DEBUG:teuthology.task.kernel:sha1 for mon.b is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.009 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-13T04:43:44.009 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-13T04:43:44.009 DEBUG:teuthology.task.kernel:sha1 for mon.c is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.009 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-13T04:43:44.009 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-04-13T04:43:44.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T04:43:44.010 DEBUG:teuthology.task.kernel:sha1 for mon.a is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.010 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-13T04:43:44.010 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-04-13T04:43:44.065 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-361.el9.x86_64 2024-04-13T04:43:44.065 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.065 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-13T04:43:44.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.070 INFO:teuthology.orchestra.run.smithi057.stdout:5.14.0-361.el9.x86_64 2024-04-13T04:43:44.070 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-361.el9.x86_64 2024-04-13T04:43:44.070 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.070 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-13T04:43:44.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.072 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.072 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-13T04:43:44.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:43:44.209 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum upgrade -y linux-firmware 2024-04-13T04:43:44.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum upgrade -y linux-firmware 2024-04-13T04:43:44.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum upgrade -y linux-firmware 2024-04-13T04:43:46.518 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-13T04:43:46.604 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-04-13T04:43:46.606 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-13T04:43:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.5 MB 00:00 2024-04-13T04:43:48.346 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.5 MB 00:00 2024-04-13T04:43:48.449 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.5 MB 00:00 2024-04-13T04:43:49.739 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-04-13T04:43:50.022 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.0 MB 00:01 2024-04-13T04:43:51.823 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-13T04:43:55.371 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 19 MB 00:04 2024-04-13T04:43:56.735 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-13T04:43:58.183 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-13T04:43:58.924 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 812 kB/s | 8.0 MB 00:10 2024-04-13T04:43:59.434 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-13T04:43:59.546 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.2 MB 00:00 2024-04-13T04:44:00.980 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-13T04:44:01.892 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-13T04:44:02.732 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-13T04:44:04.534 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-13T04:44:06.276 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-13T04:44:06.804 INFO:teuthology.orchestra.run.smithi189.stderr:Modular dependency problems: 2024-04-13T04:44:06.804 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 1: conflicting requests 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 2: conflicting requests 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 3: conflicting requests 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 4: conflicting requests 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 5: conflicting requests 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.805 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 6: conflicting requests 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 7: conflicting requests 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 8: conflicting requests 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 9: conflicting requests 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 10: conflicting requests 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 11: conflicting requests 2024-04-13T04:44:06.806 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 12: conflicting requests 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 13: conflicting requests 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 14: conflicting requests 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 15: conflicting requests 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 16: conflicting requests 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 17: conflicting requests 2024-04-13T04:44:06.807 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 18: conflicting requests 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 19: conflicting requests 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 20: conflicting requests 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 21: conflicting requests 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 22: conflicting requests 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 23: conflicting requests 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.808 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 24: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 25: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 26: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 27: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 28: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 29: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 30: conflicting requests 2024-04-13T04:44:06.809 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 31: conflicting requests 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 32: conflicting requests 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 33: conflicting requests 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 34: conflicting requests 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 35: conflicting requests 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.810 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 36: conflicting requests 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 37: conflicting requests 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 39: conflicting requests 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.811 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.812 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T04:44:06.813 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T04:44:06.814 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T04:44:07.370 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 1 Package 2024-04-13T04:44:07.379 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T04:44:07.380 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 376 M 2024-04-13T04:44:07.380 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-04-13T04:44:07.807 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-13T04:44:07.951 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-13T04:44:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:Modular dependency problems: 2024-04-13T04:44:10.061 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-13T04:44:10.061 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 1: conflicting requests 2024-04-13T04:44:10.061 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.061 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 2: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 3: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 4: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 5: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 6: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 7: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 8: conflicting requests 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.062 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 9: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 10: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 11: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 12: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 13: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 14: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 15: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 16: conflicting requests 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.063 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 17: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 18: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 19: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 20: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 21: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 22: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 23: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 24: conflicting requests 2024-04-13T04:44:10.064 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 25: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 26: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 27: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 28: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 29: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 30: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 31: conflicting requests 2024-04-13T04:44:10.065 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 32: conflicting requests 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 33: conflicting requests 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 34: conflicting requests 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 35: conflicting requests 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 36: conflicting requests 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 37: conflicting requests 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.066 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 39: conflicting requests 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.067 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.068 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.069 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T04:44:10.338 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 21 MB 00:02 2024-04-13T04:44:10.627 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-13T04:44:10.638 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 1 Package 2024-04-13T04:44:10.639 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T04:44:10.639 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 376 M 2024-04-13T04:44:10.639 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-13T04:44:15.577 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-04-13T04:44:17.926 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-04-13T04:44:17.926 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-04-13T04:44:17.926 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-04-13T04:44:17.926 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.926 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-04-13T04:44:17.927 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.928 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-04-13T04:44:17.929 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.930 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: conflicting requests 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.931 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: conflicting requests 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.932 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.933 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.934 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T04:44:17.935 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T04:44:18.599 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 1 Package 2024-04-13T04:44:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T04:44:18.613 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 376 M 2024-04-13T04:44:18.613 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-04-13T04:44:21.001 INFO:teuthology.orchestra.run.smithi189.stdout:linux-firmware-20240219-143.el9.noarch.rpm 28 MB/s | 376 MB 00:13 2024-04-13T04:44:21.001 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-04-13T04:44:21.001 INFO:teuthology.orchestra.run.smithi189.stdout:Total 28 MB/s | 376 MB 00:13 2024-04-13T04:44:22.358 INFO:teuthology.orchestra.run.smithi083.stdout:linux-firmware-20240219-143.el9.noarch.rpm 32 MB/s | 376 MB 00:11 2024-04-13T04:44:22.358 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-13T04:44:22.358 INFO:teuthology.orchestra.run.smithi083.stdout:Total 32 MB/s | 376 MB 00:11 2024-04-13T04:44:24.042 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-04-13T04:44:24.204 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-04-13T04:44:24.204 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-04-13T04:44:25.394 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-13T04:44:25.489 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-13T04:44:25.489 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-13T04:44:27.399 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-04-13T04:44:27.400 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-04-13T04:44:28.680 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-13T04:44:28.680 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-13T04:44:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-04-13T04:44:34.956 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-13T04:44:35.045 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:44:35.938 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-13T04:44:36.226 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:44:36.226 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-13T04:44:36.260 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-13T04:44:36.345 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:44:36.669 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:44:36.669 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T04:44:36.669 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-04-13T04:44:36.669 INFO:teuthology.orchestra.run.smithi189.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-13T04:44:36.669 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T04:44:36.669 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-04-13T04:44:36.974 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.c... 2024-04-13T04:44:36.975 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:44:37.335 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-13T04:44:37.336 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-13T04:44:37.339 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi189.front.sepia.ceph.com to complete... 2024-04-13T04:44:37.557 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:44:37.557 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-13T04:44:37.980 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:44:37.980 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T04:44:37.980 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-13T04:44:37.980 INFO:teuthology.orchestra.run.smithi083.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-13T04:44:37.981 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T04:44:37.981 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-13T04:44:38.284 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.b... 2024-04-13T04:44:38.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:44:38.436 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-13T04:44:38.436 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-13T04:44:38.438 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi083.front.sepia.ceph.com to complete... 2024-04-13T04:44:50.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13 04:44:50 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-13T04:44:50.539 INFO:teuthology.orchestra.run.smithi189.stderr:FINISHED --2024-04-13 04:44:50-- 2024-04-13T04:44:50.539 INFO:teuthology.orchestra.run.smithi189.stderr:Total wall clock time: 13s 2024-04-13T04:44:50.539 INFO:teuthology.orchestra.run.smithi189.stderr:Downloaded: 1 files, 341M in 13s (26.4 MB/s) 2024-04-13T04:44:50.542 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.c... 2024-04-13T04:44:50.542 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-13T04:44:52.600 INFO:teuthology.orchestra.run.smithi189.stdout:Verifying... ######################################## 2024-04-13T04:44:52.711 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing... ######################################## 2024-04-13T04:44:52.791 INFO:teuthology.orchestra.run.smithi189.stdout:Updating / installing... 2024-04-13T04:45:00.745 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-13T04:45:17.379 INFO:teuthology.orchestra.run.smithi189.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-13T04:45:40.276 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-13T04:45:40.277 DEBUG:teuthology.orchestra.run.smithi189:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-13T04:45:40.344 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-13T04:45:40.345 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-13T04:45:40.346 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-13T04:45:40.347 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-13T04:45:40.348 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-13T04:45:40.349 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-13T04:45:40.350 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-13T04:45:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-13T04:45:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-13T04:45:40.353 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-13T04:45:40.354 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-13T04:45:40.355 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-13T04:45:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-13T04:45:40.357 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-13T04:45:40.358 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-13T04:45:40.359 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-13T04:45:40.360 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-13T04:45:40.361 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-13T04:45:40.362 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-13T04:45:40.363 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-13T04:45:40.364 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-13T04:45:40.365 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-13T04:45:40.366 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-13T04:45:40.367 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-13T04:45:40.368 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-13T04:45:40.369 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-13T04:45:40.370 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-13T04:45:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-13T04:45:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-13T04:45:40.373 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-13T04:45:40.374 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-13T04:45:40.375 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-13T04:45:40.376 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-13T04:45:40.377 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-13T04:45:40.378 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-13T04:45:40.379 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-13T04:45:40.380 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-13T04:45:40.381 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-13T04:45:40.382 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-13T04:45:40.383 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-13T04:45:40.384 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-13T04:45:40.385 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-13T04:45:40.386 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-13T04:45:40.387 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-13T04:45:40.388 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-13T04:45:40.389 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-13T04:45:40.390 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-13T04:45:40.391 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-13T04:45:40.392 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-13T04:45:40.393 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-13T04:45:40.394 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-13T04:45:40.395 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-13T04:45:40.396 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-13T04:45:40.397 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-13T04:45:40.398 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-13T04:45:40.399 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-13T04:45:40.400 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-13T04:45:40.401 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-13T04:45:40.402 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-13T04:45:40.403 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-13T04:45:40.404 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-13T04:45:40.405 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-13T04:45:40.406 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-13T04:45:40.407 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-13T04:45:40.408 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-13T04:45:40.409 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-13T04:45:40.410 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-13T04:45:40.411 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-13T04:45:40.412 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-13T04:45:40.413 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-13T04:45:40.414 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-13T04:45:40.415 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-13T04:45:40.416 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-13T04:45:40.417 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-13T04:45:40.418 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-13T04:45:40.419 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-13T04:45:40.420 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-13T04:45:40.421 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-13T04:45:40.422 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-13T04:45:40.423 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-13T04:45:40.424 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-13T04:45:40.425 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-13T04:45:40.426 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-13T04:45:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-13T04:45:40.428 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-13T04:45:40.429 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-13T04:45:40.430 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-13T04:45:40.431 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-13T04:45:40.432 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-13T04:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-13T04:45:40.434 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-13T04:45:40.435 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-13T04:45:40.436 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-13T04:45:40.437 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-13T04:45:40.438 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-13T04:45:40.439 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-13T04:45:40.440 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-13T04:45:40.441 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-13T04:45:40.442 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-13T04:45:40.443 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-13T04:45:40.444 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-13T04:45:40.445 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-13T04:45:40.446 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-13T04:45:40.447 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-13T04:45:40.448 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-13T04:45:40.449 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-13T04:45:40.450 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-13T04:45:40.451 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-13T04:45:40.452 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-13T04:45:40.453 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-13T04:45:40.454 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-13T04:45:40.455 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-13T04:45:40.456 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-13T04:45:40.457 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-13T04:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-13T04:45:40.459 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-13T04:45:40.460 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-13T04:45:40.461 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-13T04:45:40.462 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-13T04:45:40.463 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-13T04:45:40.464 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-13T04:45:40.465 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-13T04:45:40.466 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-13T04:45:40.467 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-13T04:45:40.468 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-13T04:45:40.469 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-13T04:45:40.470 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-13T04:45:40.471 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-13T04:45:40.472 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-13T04:45:40.473 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-13T04:45:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-13T04:45:40.475 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-13T04:45:40.475 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-13T04:45:40.479 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.479 DEBUG:teuthology.orchestra.run.smithi189:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-13T04:45:40.500 INFO:teuthology.orchestra.run.smithi189.stdout:postinstall scriptlet (using /bin/sh): 2024-04-13T04:45:40.500 INFO:teuthology.orchestra.run.smithi189.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-13T04:45:40.500 INFO:teuthology.orchestra.run.smithi189.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout:for file in vmlinuz System.map config; do 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: fi 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout:done 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-13T04:45:40.501 INFO:teuthology.orchestra.run.smithi189.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.502 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-13T04:45:40.502 INFO:teuthology.orchestra.run.smithi189.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-13T04:45:40.502 INFO:teuthology.orchestra.run.smithi189.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-13T04:45:40.502 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.502 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-13T04:45:40.503 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 68.el9 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8264254 2024-04-13T04:45:40.578 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T04:45:40.579 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-04-13T04:45:40.580 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-13T04:45:40.580 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:40.581 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-13T04:45:41.209 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-04-13T04:45:42.601 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-13T04:45:42.621 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-04-13T04:45:42.623 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-04-13T04:45:42.638 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.uHgU6is90u 2024-04-13T04:45:42.638 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uHgU6is90u 2024-04-13T04:45:42.706 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.uHgU6is90u 2024-04-13T04:45:42.848 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.uHgU6is90u is 6KB 2024-04-13T04:45:42.896 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.uHgU6is90u 2024-04-13T04:45:42.910 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-04-13T04:45:42.974 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-13T04:45:42.974 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-13T04:45:42.974 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-13T04:45:42.974 INFO:teuthology.orchestra.run.smithi189.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-13T04:45:42.975 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:45:43.141 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-04-13T04:46:06.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13 04:46:06 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-13T04:46:06.941 INFO:teuthology.orchestra.run.smithi083.stderr:FINISHED --2024-04-13 04:46:06-- 2024-04-13T04:46:06.941 INFO:teuthology.orchestra.run.smithi083.stderr:Total wall clock time: 1m 28s 2024-04-13T04:46:06.942 INFO:teuthology.orchestra.run.smithi083.stderr:Downloaded: 1 files, 341M in 1m 28s (3.88 MB/s) 2024-04-13T04:46:06.945 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.b... 2024-04-13T04:46:06.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-13T04:46:08.994 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying... ######################################## 2024-04-13T04:46:09.057 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing... ######################################## 2024-04-13T04:46:09.137 INFO:teuthology.orchestra.run.smithi083.stdout:Updating / installing... 2024-04-13T04:46:13.144 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-13T04:46:13.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-13T04:46:13.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:46:17.083 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-13T04:46:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-13T04:46:47.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-04-13T04:46:55.157 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-13T04:46:55.157 DEBUG:teuthology.orchestra.run.smithi083:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-13T04:46:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-13T04:46:55.233 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-13T04:46:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-13T04:46:55.235 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-13T04:46:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-13T04:46:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-13T04:46:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-13T04:46:55.239 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-13T04:46:55.240 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-13T04:46:55.241 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-13T04:46:55.242 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-13T04:46:55.243 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-13T04:46:55.244 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-13T04:46:55.245 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-13T04:46:55.246 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-13T04:46:55.247 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-13T04:46:55.248 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-13T04:46:55.249 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-13T04:46:55.250 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-13T04:46:55.251 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-13T04:46:55.252 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-13T04:46:55.253 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-13T04:46:55.254 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-13T04:46:55.255 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-13T04:46:55.256 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-13T04:46:55.257 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-13T04:46:55.258 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-13T04:46:55.259 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-13T04:46:55.260 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-13T04:46:55.261 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-13T04:46:55.262 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-13T04:46:55.263 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-13T04:46:55.264 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-13T04:46:55.265 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-13T04:46:55.266 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-13T04:46:55.267 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-13T04:46:55.268 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-13T04:46:55.269 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-13T04:46:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-13T04:46:55.271 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-13T04:46:55.272 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-13T04:46:55.273 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-13T04:46:55.274 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-13T04:46:55.275 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-13T04:46:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-13T04:46:55.277 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-13T04:46:55.278 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-13T04:46:55.279 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-13T04:46:55.280 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-13T04:46:55.281 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-13T04:46:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-13T04:46:55.283 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-13T04:46:55.284 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-13T04:46:55.285 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-13T04:46:55.286 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-13T04:46:55.287 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-13T04:46:55.288 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-13T04:46:55.289 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-13T04:46:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-13T04:46:55.291 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-13T04:46:55.292 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-13T04:46:55.293 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-13T04:46:55.294 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-13T04:46:55.295 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-13T04:46:55.296 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-13T04:46:55.297 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-13T04:46:55.298 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-13T04:46:55.299 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-13T04:46:55.300 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-13T04:46:55.301 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-13T04:46:55.302 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-13T04:46:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-13T04:46:55.304 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-13T04:46:55.305 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-13T04:46:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-13T04:46:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-13T04:46:55.308 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-13T04:46:55.309 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-13T04:46:55.310 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-13T04:46:55.311 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-13T04:46:55.312 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-13T04:46:55.313 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-13T04:46:55.314 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-13T04:46:55.315 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-13T04:46:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-13T04:46:55.317 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-13T04:46:55.318 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-13T04:46:55.319 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-13T04:46:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-13T04:46:55.321 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-13T04:46:55.322 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-13T04:46:55.323 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-13T04:46:55.324 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-13T04:46:55.325 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-13T04:46:55.326 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-13T04:46:55.327 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-13T04:46:55.328 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-13T04:46:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-13T04:46:55.330 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-13T04:46:55.331 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-13T04:46:55.332 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-13T04:46:55.333 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-13T04:46:55.334 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-13T04:46:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-13T04:46:55.336 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-13T04:46:55.337 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-13T04:46:55.338 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-13T04:46:55.339 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-13T04:46:55.340 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-13T04:46:55.341 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-13T04:46:55.342 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-13T04:46:55.343 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-13T04:46:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-13T04:46:55.345 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-13T04:46:55.346 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-13T04:46:55.347 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-13T04:46:55.348 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-13T04:46:55.349 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-13T04:46:55.350 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-13T04:46:55.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-13T04:46:55.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-13T04:46:55.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-13T04:46:55.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-13T04:46:55.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-13T04:46:55.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-13T04:46:55.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-13T04:46:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-13T04:46:55.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-13T04:46:55.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-13T04:46:55.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-13T04:46:55.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-13T04:46:55.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-13T04:46:55.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-13T04:46:55.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-13T04:46:55.369 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.369 DEBUG:teuthology.orchestra.run.smithi083:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:postinstall scriptlet (using /bin/sh): 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:for file in vmlinuz System.map config; do 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: fi 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:46:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.394 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-13T04:46:55.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 68.el9 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-04-13T04:46:55.482 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8264254 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-13T04:46:55.483 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-04-13T04:46:55.484 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T04:46:55.484 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-04-13T04:46:55.484 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-13T04:46:55.485 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:55.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-13T04:46:56.067 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-13T04:46:56.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-13T04:46:56.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:46:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-13T04:46:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-13T04:46:57.479 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-13T04:46:57.495 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.fJjkTQ94qQ 2024-04-13T04:46:57.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fJjkTQ94qQ 2024-04-13T04:46:57.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.fJjkTQ94qQ 2024-04-13T04:46:57.707 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.fJjkTQ94qQ is 6KB 2024-04-13T04:46:57.726 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.fJjkTQ94qQ 2024-04-13T04:46:57.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-04-13T04:46:57.805 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-13T04:46:57.806 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-13T04:46:57.806 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-13T04:46:57.806 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-13T04:46:57.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:46:57.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-04-13T04:47:27.964 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-13T04:47:27.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-13T04:47:27.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:47:56.845 DEBUG:teuthology.orchestra.remote:timed out 2024-04-13T04:48:08.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-13T04:48:08.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:48:09.370 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-04-13T04:48:10.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-13T04:48:10.047 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-13T04:48:10.047 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-04-13T04:48:10.100 INFO:teuthology.orchestra.run.smithi189.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:48:10.101 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:48:10.101 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-13T04:48:10.101 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-13T04:48:11.102 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-13T04:48:11.103 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-13T04:48:11.243 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-04-13T04:48:11.306 DEBUG:teuthology.parallel:result is None 2024-04-13T04:48:27.968 DEBUG:teuthology.orchestra.remote:timed out 2024-04-13T04:48:36.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-13T04:48:36.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:48:40.472 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-13T04:48:41.187 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-13T04:48:41.187 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-13T04:48:41.187 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-13T04:48:41.226 INFO:teuthology.orchestra.run.smithi083.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:48:41.226 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:48:41.226 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-13T04:48:41.226 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-13T04:48:42.228 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-13T04:48:42.228 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-13T04:48:42.354 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-04-13T04:48:42.396 DEBUG:teuthology.parallel:result is None 2024-04-13T04:49:34.918 INFO:teuthology.orchestra.run.smithi057.stdout:linux-firmware-20240219-143.el9.noarch.rpm 1.2 MB/s | 376 MB 05:16 2024-04-13T04:49:34.918 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-04-13T04:49:34.918 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.2 MB/s | 376 MB 05:16 2024-04-13T04:49:38.102 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-04-13T04:49:38.214 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-04-13T04:49:38.214 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-04-13T04:49:41.564 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-04-13T04:49:41.565 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-04-13T04:49:48.858 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-04-13T04:49:49.252 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-13T04:49:49.349 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:49:50.334 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:49:50.334 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-13T04:49:50.796 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-13T04:49:50.796 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T04:49:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2024-04-13T04:49:50.796 INFO:teuthology.orchestra.run.smithi057.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-13T04:49:50.796 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T04:49:50.796 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-04-13T04:49:51.094 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.a... 2024-04-13T04:49:51.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-13T04:49:51.246 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-13T04:49:51.246 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-13T04:49:51.249 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi057.front.sepia.ceph.com to complete... 2024-04-13T04:50:04.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13 04:50:04 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-13T04:50:04.521 INFO:teuthology.orchestra.run.smithi057.stderr:FINISHED --2024-04-13 04:50:04-- 2024-04-13T04:50:04.521 INFO:teuthology.orchestra.run.smithi057.stderr:Total wall clock time: 13s 2024-04-13T04:50:04.521 INFO:teuthology.orchestra.run.smithi057.stderr:Downloaded: 1 files, 341M in 13s (26.5 MB/s) 2024-04-13T04:50:04.526 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.a... 2024-04-13T04:50:04.527 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-13T04:50:06.678 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying... ######################################## 2024-04-13T04:50:06.738 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing... ######################################## 2024-04-13T04:50:06.822 INFO:teuthology.orchestra.run.smithi057.stdout:Updating / installing... 2024-04-13T04:50:15.162 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-13T04:50:31.954 INFO:teuthology.orchestra.run.smithi057.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-13T04:50:55.678 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-13T04:50:55.679 DEBUG:teuthology.orchestra.run.smithi057:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-13T04:50:55.766 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-13T04:50:55.767 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-13T04:50:55.768 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-13T04:50:55.769 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-13T04:50:55.770 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-13T04:50:55.771 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-13T04:50:55.772 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-13T04:50:55.773 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-13T04:50:55.774 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-13T04:50:55.775 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-13T04:50:55.776 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-13T04:50:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-13T04:50:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-13T04:50:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-13T04:50:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-13T04:50:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-13T04:50:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-13T04:50:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-13T04:50:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-13T04:50:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-13T04:50:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-13T04:50:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-13T04:50:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-13T04:50:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-13T04:50:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-13T04:50:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-13T04:50:55.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-13T04:50:55.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-13T04:50:55.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-13T04:50:55.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-13T04:50:55.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-13T04:50:55.783 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-13T04:50:55.784 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-13T04:50:55.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-13T04:50:55.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-13T04:50:55.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-13T04:50:55.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-13T04:50:55.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-13T04:50:55.785 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-13T04:50:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-13T04:50:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-13T04:50:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-13T04:50:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-13T04:50:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-13T04:50:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-13T04:50:55.787 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-13T04:50:55.788 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-13T04:50:55.789 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-13T04:50:55.790 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-13T04:50:55.791 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-13T04:50:55.792 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-13T04:50:55.793 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-13T04:50:55.794 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-13T04:50:55.795 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-13T04:50:55.796 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-13T04:50:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-13T04:50:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-13T04:50:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-13T04:50:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-13T04:50:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-13T04:50:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-13T04:50:55.803 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-13T04:50:55.804 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-13T04:50:55.805 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-13T04:50:55.806 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-13T04:50:55.807 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-13T04:50:55.808 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-13T04:50:55.809 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-13T04:50:55.810 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-13T04:50:55.811 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-13T04:50:55.812 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-13T04:50:55.813 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-13T04:50:55.814 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-13T04:50:55.815 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-13T04:50:55.816 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-13T04:50:55.817 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-13T04:50:55.818 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-13T04:50:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-13T04:50:55.820 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-13T04:50:55.821 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-13T04:50:55.822 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-13T04:50:55.823 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-13T04:50:55.824 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-13T04:50:55.825 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-13T04:50:55.826 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-13T04:50:55.827 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-13T04:50:55.828 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-13T04:50:55.829 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-13T04:50:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-13T04:50:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-13T04:50:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-13T04:50:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-13T04:50:55.830 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-13T04:50:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-13T04:50:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-13T04:50:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-13T04:50:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-13T04:50:55.831 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-13T04:50:55.832 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-13T04:50:55.833 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-13T04:50:55.834 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-13T04:50:55.835 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-13T04:50:55.836 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-13T04:50:55.837 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-13T04:50:55.838 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-13T04:50:55.839 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-13T04:50:55.840 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-13T04:50:55.841 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-13T04:50:55.842 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-13T04:50:55.843 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-13T04:50:55.844 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-13T04:50:55.845 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-13T04:50:55.846 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-13T04:50:55.847 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-13T04:50:55.848 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-13T04:50:55.849 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-13T04:50:55.850 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-13T04:50:55.851 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-13T04:50:55.852 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-13T04:50:55.853 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-13T04:50:55.854 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-13T04:50:55.855 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-13T04:50:55.856 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-13T04:50:55.857 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-13T04:50:55.858 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-13T04:50:55.859 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-13T04:50:55.860 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-13T04:50:55.861 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-13T04:50:55.862 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-13T04:50:55.863 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-13T04:50:55.864 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-13T04:50:55.865 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-13T04:50:55.866 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-13T04:50:55.867 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-13T04:50:55.868 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-13T04:50:55.869 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-13T04:50:55.870 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-13T04:50:55.871 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-13T04:50:55.872 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-13T04:50:55.873 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-13T04:50:55.874 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-13T04:50:55.875 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-13T04:50:55.876 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-13T04:50:55.877 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-13T04:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-13T04:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-13T04:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-13T04:50:55.881 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-13T04:50:55.882 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-13T04:50:55.883 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-13T04:50:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-13T04:50:55.885 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-13T04:50:55.886 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-13T04:50:55.887 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-13T04:50:55.888 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-13T04:50:55.889 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-13T04:50:55.890 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-13T04:50:55.891 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-13T04:50:55.892 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-13T04:50:55.893 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-13T04:50:55.894 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-13T04:50:55.895 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-13T04:50:55.896 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-13T04:50:55.897 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-13T04:50:55.898 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-13T04:50:55.899 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-13T04:50:55.900 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-13T04:50:55.901 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-13T04:50:55.902 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-13T04:50:55.903 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-13T04:50:55.904 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-13T04:50:55.905 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-13T04:50:55.906 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-13T04:50:55.907 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-13T04:50:55.908 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-13T04:50:55.909 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-13T04:50:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-13T04:50:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-13T04:50:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-13T04:50:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-13T04:50:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-13T04:50:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-13T04:50:55.914 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:55.914 DEBUG:teuthology.orchestra.run.smithi057:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-13T04:50:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:postinstall scriptlet (using /bin/sh): 2024-04-13T04:50:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-13T04:50:55.938 INFO:teuthology.orchestra.run.smithi057.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-13T04:50:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-04-13T04:50:55.938 INFO:teuthology.orchestra.run.smithi057.stdout:for file in vmlinuz System.map config; do 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: fi 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:done 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-13T04:50:55.939 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:55.940 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-04-13T04:50:55.940 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2024-04-13T04:50:56.017 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.06 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 68.el9 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Group : Unspecified 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 8264254 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : builder@centos.org 2024-04-13T04:50:56.018 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : CentOS 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T04:50:56.019 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2024-04-13T04:50:56.020 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-13T04:50:56.020 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:56.020 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-13T04:50:56.722 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2024-04-13T04:50:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-13T04:50:58.172 INFO:teuthology.orchestra.run.smithi057.stderr:done 2024-04-13T04:50:58.174 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-04-13T04:50:58.230 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.RbVaoBwfog 2024-04-13T04:50:58.230 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RbVaoBwfog 2024-04-13T04:50:58.304 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.RbVaoBwfog 2024-04-13T04:50:58.445 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.RbVaoBwfog is 6KB 2024-04-13T04:50:58.494 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.RbVaoBwfog 2024-04-13T04:50:58.509 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2024-04-13T04:50:58.582 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-13T04:50:58.583 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-13T04:50:58.583 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-13T04:50:58.583 INFO:teuthology.orchestra.run.smithi057.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-13T04:50:58.584 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:50:58.743 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2024-04-13T04:51:28.745 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-13T04:51:28.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:51:28.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:51:47.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-04-13T04:51:56.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:51:56.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:51:59.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-04-13T04:52:11.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:52:11.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:52:14.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-04-13T04:52:29.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:52:29.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:53:01.285 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-04-13T04:53:01.985 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:53:01.985 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-13T04:53:01.986 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-04-13T04:53:02.029 INFO:teuthology.orchestra.run.smithi057.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:53:02.030 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-13T04:53:02.030 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-13T04:53:02.030 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-13T04:53:03.031 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-13T04:53:03.031 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-13T04:53:03.172 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2024-04-13T04:53:03.218 DEBUG:teuthology.parallel:result is None 2024-04-13T04:53:03.218 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-13T04:53:03.226 INFO:teuthology.task.internal:Creating test directory... 2024-04-13T04:53:03.226 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-13T04:53:03.232 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-13T04:53:03.241 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-13T04:53:03.300 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-13T04:53:03.314 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-13T04:53:03.320 INFO:teuthology.task.internal:Creating archive directory... 2024-04-13T04:53:03.321 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-13T04:53:03.327 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-13T04:53:03.333 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-13T04:53:03.410 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-13T04:53:03.417 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-13T04:53:03.417 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-13T04:53:03.422 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-13T04:53:03.429 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-13T04:53:03.507 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T04:53:03.528 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T04:53:03.532 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T04:53:03.545 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T04:53:03.569 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T04:53:03.575 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-13T04:53:03.580 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-13T04:53:03.588 INFO:teuthology.task.internal:Configuring sudo... 2024-04-13T04:53:03.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-13T04:53:03.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-13T04:53:03.600 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-13T04:53:03.691 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-13T04:53:03.700 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-13T04:53:03.701 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-13T04:53:03.706 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-13T04:53:03.713 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-13T04:53:03.765 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T04:53:03.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T04:53:03.967 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T04:53:04.036 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T04:53:04.121 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T04:53:04.121 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-13T04:53:04.192 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T04:53:04.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T04:53:04.401 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T04:53:04.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T04:53:04.587 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T04:53:04.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-13T04:53:04.682 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T04:53:04.769 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-13T04:53:04.884 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T04:53:04.964 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-13T04:53:05.067 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T04:53:05.067 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-13T04:53:05.156 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2024-04-13T04:53:05.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-13T04:53:05.168 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-04-13T04:53:05.262 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T04:53:05.291 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T04:53:05.345 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T04:53:05.826 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-13T04:53:05.834 INFO:teuthology.task.internal:Starting timer... 2024-04-13T04:53:05.834 INFO:teuthology.run_tasks:Running task pcp... 2024-04-13T04:53:05.845 INFO:teuthology.run_tasks:Running task selinux... 2024-04-13T04:53:05.854 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-13T04:53:05.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2024-04-13T04:53:05.965 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2024-04-13T04:53:05.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-13T04:53:06.087 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-13T04:53:06.092 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-04-13T04:53:06.220 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-04-13T04:53:06.225 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-13T04:53:06.225 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2024-04-13T04:53:06.303 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2024-04-13T04:53:06.303 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-13T04:53:06.359 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-13T04:53:06.360 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-04-13T04:53:06.422 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-04-13T04:53:06.422 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-04-13T04:53:06.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T04:53:06.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T04:53:06.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T04:53:06.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T04:53:06.582 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T04:53:06.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T04:53:06.676 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-13T04:53:06.676 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2024-04-13T04:53:06.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-13T04:53:06.855 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-04-13T04:53:06.943 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-13T04:53:06.953 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-13T04:53:06.975 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-13T04:53:06.976 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi083.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-13T04:59:51.213 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-04-13T04:59:51.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:59:51.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:59:51.369 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-04-13T04:59:51.530 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-13T04:59:51.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-13T04:59:51.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:59:51.676 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-13T04:59:51.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-13T04:59:51.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-13T04:59:51.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T04:59:51.968 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-04-13T04:59:52.131 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-13T04:59:52.131 INFO:teuthology.run_tasks:Running task clock... 2024-04-13T04:59:52.142 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-13T04:59:52.142 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-13T04:59:52.143 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T04:59:52.150 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-13T04:59:52.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T04:59:52.157 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-13T04:59:52.157 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T04:59:52.250 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-13T04:59:52.256 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-13T04:59:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-13T04:59:52.307 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-13T04:59:52.308 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-13T04:59:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-13T04:59:52.403 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-04-13T04:59:52.407 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2024-04-13T04:59:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-13T04:59:52.444 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-04-13T04:59:52.449 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2024-04-13T04:59:52.451 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-13T04:59:52.496 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-13T04:59:52.502 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-13T04:59:52.507 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-13T04:59:52.547 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-13T04:59:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-13T04:59:52.565 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-13T04:59:52.683 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-04-13T04:59:52.691 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T04:59:52.691 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-04-13T04:59:52.691 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.692 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.692 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.692 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.692 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-13T04:59:52.701 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T04:59:52.701 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-13T04:59:52.701 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.701 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.701 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.701 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.713 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-04-13T04:59:52.723 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T04:59:52.723 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-04-13T04:59:52.724 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.724 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.724 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.724 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T04:59:52.725 INFO:teuthology.run_tasks:Running task install... 2024-04-13T04:59:52.734 DEBUG:teuthology.task.install:project ceph 2024-04-13T04:59:52.734 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'}} 2024-04-13T04:59:52.734 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838'} 2024-04-13T04:59:52.734 INFO:teuthology.task.install:Using flavor: default 2024-04-13T04:59:52.740 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-13T04:59:52.740 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-13T04:59:52.741 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'tag': None, 'wait_for_package': False} 2024-04-13T04:59:52.741 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T04:59:52.743 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'tag': None, 'wait_for_package': False} 2024-04-13T04:59:52.743 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T04:59:52.744 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'tag': None, 'wait_for_package': False} 2024-04-13T04:59:52.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T04:59:52.965 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/ 2024-04-13T04:59:52.965 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2884.g218a6914 2024-04-13T04:59:52.972 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/ 2024-04-13T04:59:52.972 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2884.g218a6914 2024-04-13T04:59:52.980 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/ 2024-04-13T04:59:52.980 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2884.g218a6914 2024-04-13T04:59:53.071 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-13T04:59:53.071 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T04:59:53.071 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-13T04:59:53.083 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-13T04:59:53.083 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T04:59:53.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-13T04:59:53.134 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-13T04:59:53.134 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T04:59:53.134 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-13T04:59:53.174 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-13T04:59:53.174 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/218a691417ef57c5d22a8d5cf3c5f41f7b542838/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-13T04:59:53.179 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-13T04:59:53.179 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/218a691417ef57c5d22a8d5cf3c5f41f7b542838/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-13T04:59:53.225 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-13T04:59:53.225 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/218a691417ef57c5d22a8d5cf3c5f41f7b542838/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-13T04:59:53.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-13T04:59:53.321 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-13T04:59:53.363 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-13T04:59:53.454 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-13T04:59:53.459 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-13T04:59:53.496 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-13T04:59:53.544 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-13T04:59:53.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-13T04:59:53.551 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2024-04-13T04:59:53.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2024-04-13T04:59:53.581 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-04-13T04:59:53.585 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-04-13T04:59:53.906 INFO:teuthology.orchestra.run.smithi083.stdout:74 files removed 2024-04-13T04:59:53.930 INFO:teuthology.orchestra.run.smithi057.stdout:74 files removed 2024-04-13T04:59:53.934 INFO:teuthology.orchestra.run.smithi189.stdout:74 files removed 2024-04-13T04:59:53.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-13T04:59:53.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-13T04:59:53.973 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 216 kB/s | 83 kB 00:00 2024-04-13T04:59:54.921 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-04-13T04:59:54.978 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 174 kB/s | 83 kB 00:00 2024-04-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-13T04:59:55.925 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-13T04:59:55.947 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-13T04:59:56.562 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-13T04:59:56.601 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-13T04:59:56.741 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-13T04:59:58.472 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.5 MB 00:00 2024-04-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.5 MB 00:00 2024-04-13T04:59:58.714 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.5 MB 00:00 2024-04-13T05:00:00.046 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-13T05:00:00.219 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-04-13T05:00:02.159 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-13T05:00:02.443 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 19 MB 00:02 2024-04-13T05:00:06.724 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-13T05:00:08.708 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-04-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-13T05:00:10.505 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-13T05:00:11.167 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-13T05:00:11.364 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-13T05:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-13T05:00:16.802 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-13T05:00:18.494 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-13T05:00:19.582 INFO:teuthology.orchestra.run.smithi057.stderr:Modular dependency problems: 2024-04-13T05:00:19.582 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 1: conflicting requests 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 2: conflicting requests 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 3: conflicting requests 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 4: conflicting requests 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 5: conflicting requests 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.583 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 6: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 7: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 8: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 9: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 10: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 11: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 12: conflicting requests 2024-04-13T05:00:19.584 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 13: conflicting requests 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 14: conflicting requests 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 15: conflicting requests 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 16: conflicting requests 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 17: conflicting requests 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.585 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 18: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 19: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 20: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 21: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 22: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 23: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 24: conflicting requests 2024-04-13T05:00:19.586 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 25: conflicting requests 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 26: conflicting requests 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 27: conflicting requests 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 28: conflicting requests 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 29: conflicting requests 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.587 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 30: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 31: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 32: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 33: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 34: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 35: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 36: conflicting requests 2024-04-13T05:00:19.588 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 37: conflicting requests 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 39: conflicting requests 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.589 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.590 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T05:00:19.591 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T05:00:19.592 INFO:teuthology.orchestra.run.smithi057.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T05:00:20.218 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T05:00:20.220 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T05:00:20.244 INFO:teuthology.orchestra.run.smithi057.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-13T05:00:20.246 INFO:teuthology.orchestra.run.smithi057.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-13T05:00:20.247 INFO:teuthology.orchestra.run.smithi057.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-13T05:00:20.249 INFO:teuthology.orchestra.run.smithi057.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-13T05:00:20.252 INFO:teuthology.orchestra.run.smithi057.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-13T05:00:20.255 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-13T05:00:20.257 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-13T05:00:20.364 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-04-13T05:00:20.379 INFO:teuthology.orchestra.run.smithi057.stdout:========================================================================================= 2024-04-13T05:00:20.379 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-04-13T05:00:20.379 INFO:teuthology.orchestra.run.smithi057.stdout:========================================================================================= 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:19.0.0-2884.g218a6914.el9 ceph 6.6 k 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.4 M 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 891 k 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2884.g218a6914.el9 ceph 145 k 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:19.0.0-2884.g218a6914.el9 ceph 1.5 M 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 142 k 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 3.5 M 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 7.4 M 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 49 k 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 10 M 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:19.0.0-2884.g218a6914.el9 ceph 49 M 2024-04-13T05:00:20.380 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 264 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 763 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 225 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-shell x86_64 2:19.0.0-2884.g218a6914.el9 ceph 27 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 18 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 32 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 746 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 126 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-13T05:00:20.381 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:19.0.0-2884.g218a6914.el9 ceph 162 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:19.0.0-2884.g218a6914.el9 ceph 321 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 298 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 100 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 85 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 170 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-13T05:00:20.382 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.4 M 2024-04-13T05:00:20.383 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-13T05:00:20.384 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 21 M 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 34 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:19.0.0-2884.g218a6914.el9 ceph 2.4 M 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 249 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:19.0.0-2884.g218a6914.el9 ceph 4.7 M 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 17 M 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 16 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:19.0.0-2884.g218a6914.el9 ceph 25 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-13T05:00:20.385 INFO:teuthology.orchestra.run.smithi057.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:19.0.0-2884.g218a6914.el9 ceph 163 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 500 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.2 M 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-13T05:00:20.386 INFO:teuthology.orchestra.run.smithi057.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-13T05:00:20.387 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 45 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 135 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-13T05:00:20.388 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-13T05:00:20.389 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-13T05:00:20.390 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-13T05:00:20.391 INFO:teuthology.orchestra.run.smithi057.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout:Installing weak dependencies: 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-13T05:00:20.392 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout:========================================================================================= 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout:Install 150 Packages 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 24 Packages 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 229 M 2024-04-13T05:00:20.393 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-04-13T05:00:21.171 INFO:teuthology.orchestra.run.smithi083.stderr:Modular dependency problems: 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 1: conflicting requests 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 2: conflicting requests 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 3: conflicting requests 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 4: conflicting requests 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 5: conflicting requests 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 6: conflicting requests 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.172 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 7: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 8: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 9: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 10: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 11: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 12: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 13: conflicting requests 2024-04-13T05:00:21.173 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 14: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 15: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 16: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 17: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 18: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 19: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 20: conflicting requests 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.174 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 21: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 22: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 23: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 24: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 25: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 26: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 27: conflicting requests 2024-04-13T05:00:21.175 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 28: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 29: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 30: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 31: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 32: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 33: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 34: conflicting requests 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.176 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 35: conflicting requests 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 36: conflicting requests 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 37: conflicting requests 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 39: conflicting requests 2024-04-13T05:00:21.177 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.178 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T05:00:21.179 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.180 INFO:teuthology.orchestra.run.smithi083.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr:Modular dependency problems: 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 1: conflicting requests 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 2: conflicting requests 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 3: conflicting requests 2024-04-13T05:00:21.392 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 4: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 5: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 6: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 7: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 8: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 9: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 10: conflicting requests 2024-04-13T05:00:21.393 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 11: conflicting requests 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 12: conflicting requests 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 13: conflicting requests 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 14: conflicting requests 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 15: conflicting requests 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.394 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 16: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 17: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 18: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 19: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 20: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 21: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 22: conflicting requests 2024-04-13T05:00:21.395 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 23: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 24: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 25: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 26: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 27: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 28: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 29: conflicting requests 2024-04-13T05:00:21.396 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 30: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 31: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 32: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 33: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 34: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 35: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 36: conflicting requests 2024-04-13T05:00:21.397 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 37: conflicting requests 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 38: cannot install the best candidate for the job 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 39: conflicting requests 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.398 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 40: cannot install the best candidate for the job 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 41: cannot install the best candidate for the job 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.399 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 42: cannot install the best candidate for the job 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 43: cannot install the best candidate for the job 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 44: cannot install the best candidate for the job 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.400 INFO:teuthology.orchestra.run.smithi189.stderr: Problem 45: cannot install the best candidate for the job 2024-04-13T05:00:21.401 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.401 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.401 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.401 INFO:teuthology.orchestra.run.smithi189.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-13T05:00:21.436 INFO:teuthology.orchestra.run.smithi057.stdout:(1/174): ceph-19.0.0-2884.g218a6914.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-13T05:00:21.774 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T05:00:21.775 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T05:00:21.795 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-13T05:00:21.796 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-13T05:00:21.797 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-13T05:00:21.799 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-13T05:00:21.801 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-13T05:00:21.805 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-13T05:00:21.806 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-13T05:00:21.907 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout:========================================================================================= 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout:========================================================================================= 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-2884.g218a6914.el9 ceph 6.6 k 2024-04-13T05:00:21.928 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.4 M 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 891 k 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2884.g218a6914.el9 ceph 145 k 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-2884.g218a6914.el9 ceph 1.5 M 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 142 k 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 3.5 M 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 7.4 M 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 49 k 2024-04-13T05:00:21.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 10 M 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-2884.g218a6914.el9 ceph 49 M 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 264 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 763 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 225 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-2884.g218a6914.el9 ceph 27 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 18 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 32 k 2024-04-13T05:00:21.930 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 746 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 126 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-2884.g218a6914.el9 ceph 162 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-2884.g218a6914.el9 ceph 321 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 298 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 100 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 85 k 2024-04-13T05:00:21.931 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 170 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-13T05:00:21.932 INFO:teuthology.orchestra.run.smithi083.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.4 M 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-13T05:00:21.933 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 21 M 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 34 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-2884.g218a6914.el9 ceph 2.4 M 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 249 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-2884.g218a6914.el9 ceph 4.7 M 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 17 M 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 16 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-2884.g218a6914.el9 ceph 25 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-13T05:00:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-2884.g218a6914.el9 ceph 163 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 500 k 2024-04-13T05:00:21.935 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.2 M 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-13T05:00:21.936 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 45 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 135 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-13T05:00:21.937 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-13T05:00:21.938 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-13T05:00:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-13T05:00:21.940 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-13T05:00:21.941 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-13T05:00:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:00:21.942 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-13T05:00:21.942 INFO:teuthology.orchestra.run.smithi083.stdout:========================================================================================= 2024-04-13T05:00:21.942 INFO:teuthology.orchestra.run.smithi083.stdout:Install 150 Packages 2024-04-13T05:00:21.942 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 24 Packages 2024-04-13T05:00:21.942 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:00:21.943 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 229 M 2024-04-13T05:00:21.943 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi057.stdout:(2/174): ceph-base-19.0.0-2884.g218a6914.el9.x8 5.8 MB/s | 5.4 MB 00:00 2024-04-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T05:00:22.063 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi057.stdout:(3/174): ceph-immutable-object-cache-19.0.0-288 2.2 MB/s | 145 kB 00:00 2024-04-13T05:00:22.187 INFO:teuthology.orchestra.run.smithi057.stdout:(4/174): ceph-fuse-19.0.0-2884.g218a6914.el9.x8 1.2 MB/s | 891 kB 00:00 2024-04-13T05:00:22.200 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-04-13T05:00:22.223 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================= 2024-04-13T05:00:22.223 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-04-13T05:00:22.223 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================= 2024-04-13T05:00:22.223 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-04-13T05:00:22.223 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.0.0-2884.g218a6914.el9 ceph 6.6 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.4 M 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 891 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2884.g218a6914.el9 ceph 145 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.0.0-2884.g218a6914.el9 ceph 1.5 M 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 142 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 3.5 M 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 7.4 M 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 49 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 10 M 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.0.0-2884.g218a6914.el9 ceph 49 M 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 264 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 763 k 2024-04-13T05:00:22.224 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 225 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.0.0-2884.g218a6914.el9 ceph 27 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 18 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 32 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 746 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.0.0-2884.g218a6914.el9 ceph 126 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-13T05:00:22.225 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.0.0-2884.g218a6914.el9 ceph 162 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.0.0-2884.g218a6914.el9 ceph 321 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 298 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.0.0-2884.g218a6914.el9 ceph 100 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 85 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 170 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-13T05:00:22.226 INFO:teuthology.orchestra.run.smithi189.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.4 M 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 3.1 M 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-13T05:00:22.227 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 21 M 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 34 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.0.0-2884.g218a6914.el9 ceph 2.4 M 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 249 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.0.0-2884.g218a6914.el9 ceph 4.7 M 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.0.0-2884.g218a6914.el9 ceph 17 M 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2884.g218a6914.el9 ceph-noarch 16 k 2024-04-13T05:00:22.228 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.0.0-2884.g218a6914.el9 ceph 25 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.0.0-2884.g218a6914.el9 ceph 163 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-13T05:00:22.229 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 500 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.0.0-2884.g218a6914.el9 ceph 5.2 M 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-13T05:00:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.0.0-2884.g218a6914.el9 ceph 45 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.0.0-2884.g218a6914.el9 ceph 135 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-13T05:00:22.231 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-13T05:00:22.232 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-13T05:00:22.233 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-13T05:00:22.234 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-13T05:00:22.235 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================= 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout:Install 150 Packages 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 24 Packages 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 229 M 2024-04-13T05:00:22.236 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-04-13T05:00:22.412 INFO:teuthology.orchestra.run.smithi057.stdout:(5/174): ceph-mds-19.0.0-2884.g218a6914.el9.x86 7.7 MB/s | 2.4 MB 00:00 2024-04-13T05:00:22.913 INFO:teuthology.orchestra.run.smithi057.stdout:(6/174): ceph-mon-19.0.0-2884.g218a6914.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-04-13T05:00:22.946 INFO:teuthology.orchestra.run.smithi083.stdout:(1/174): ceph-19.0.0-2884.g218a6914.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-13T05:00:23.021 INFO:teuthology.orchestra.run.smithi057.stdout:(7/174): ceph-mgr-19.0.0-2884.g218a6914.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-13T05:00:23.324 INFO:teuthology.orchestra.run.smithi189.stdout:(1/174): ceph-19.0.0-2884.g218a6914.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-13T05:00:24.199 INFO:teuthology.orchestra.run.smithi189.stdout:(2/174): ceph-fuse-19.0.0-2884.g218a6914.el9.x8 1.0 MB/s | 891 kB 00:00 2024-04-13T05:00:24.254 INFO:teuthology.orchestra.run.smithi083.stdout:(2/174): ceph-fuse-19.0.0-2884.g218a6914.el9.x8 681 kB/s | 891 kB 00:01 2024-04-13T05:00:24.324 INFO:teuthology.orchestra.run.smithi189.stdout:(3/174): ceph-immutable-object-cache-19.0.0-288 1.1 MB/s | 145 kB 00:00 2024-04-13T05:00:24.580 INFO:teuthology.orchestra.run.smithi083.stdout:(3/174): ceph-immutable-object-cache-19.0.0-288 446 kB/s | 145 kB 00:00 2024-04-13T05:00:25.022 INFO:teuthology.orchestra.run.smithi083.stdout:(4/174): ceph-base-19.0.0-2884.g218a6914.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-13T05:00:25.238 INFO:teuthology.orchestra.run.smithi057.stdout:(8/174): ceph-osd-19.0.0-2884.g218a6914.el9.x86 7.2 MB/s | 17 MB 00:02 2024-04-13T05:00:25.322 INFO:teuthology.orchestra.run.smithi057.stdout:(9/174): ceph-selinux-19.0.0-2884.g218a6914.el9 305 kB/s | 25 kB 00:00 2024-04-13T05:00:25.425 INFO:teuthology.orchestra.run.smithi189.stdout:(4/174): ceph-mds-19.0.0-2884.g218a6914.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-04-13T05:00:25.589 INFO:teuthology.orchestra.run.smithi083.stdout:(5/174): ceph-mgr-19.0.0-2884.g218a6914.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-13T05:00:25.975 INFO:teuthology.orchestra.run.smithi189.stdout:(5/174): ceph-mgr-19.0.0-2884.g218a6914.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-04-13T05:00:27.025 INFO:teuthology.orchestra.run.smithi189.stdout:(6/174): ceph-mon-19.0.0-2884.g218a6914.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-04-13T05:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout:(7/174): ceph-base-19.0.0-2884.g218a6914.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-13T05:00:27.147 INFO:teuthology.orchestra.run.smithi083.stdout:(6/174): ceph-mds-19.0.0-2884.g218a6914.el9.x86 943 kB/s | 2.4 MB 00:02 2024-04-13T05:00:27.439 INFO:teuthology.orchestra.run.smithi083.stdout:(7/174): ceph-mon-19.0.0-2884.g218a6914.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-13T05:00:29.717 INFO:teuthology.orchestra.run.smithi189.stdout:(8/174): ceph-osd-19.0.0-2884.g218a6914.el9.x86 6.3 MB/s | 17 MB 00:02 2024-04-13T05:00:29.818 INFO:teuthology.orchestra.run.smithi189.stdout:(9/174): ceph-selinux-19.0.0-2884.g218a6914.el9 253 kB/s | 25 kB 00:00 2024-04-13T05:00:30.589 INFO:teuthology.orchestra.run.smithi083.stdout:(8/174): ceph-common-19.0.0-2884.g218a6914.el9. 2.6 MB/s | 21 MB 00:08 2024-04-13T05:00:30.681 INFO:teuthology.orchestra.run.smithi083.stdout:(9/174): ceph-selinux-19.0.0-2884.g218a6914.el9 278 kB/s | 25 kB 00:00 2024-04-13T05:00:31.547 INFO:teuthology.orchestra.run.smithi057.stdout:(10/174): ceph-radosgw-19.0.0-2884.g218a6914.el 1.2 MB/s | 10 MB 00:08 2024-04-13T05:00:31.556 INFO:teuthology.orchestra.run.smithi083.stdout:(10/174): ceph-radosgw-19.0.0-2884.g218a6914.el 2.4 MB/s | 10 MB 00:04 2024-04-13T05:00:31.755 INFO:teuthology.orchestra.run.smithi057.stdout:(11/174): cephfs-mirror-19.0.0-2884.g218a6914.e 1.1 MB/s | 225 kB 00:00 2024-04-13T05:00:31.815 INFO:teuthology.orchestra.run.smithi083.stdout:(11/174): cephfs-mirror-19.0.0-2884.g218a6914.e 873 kB/s | 225 kB 00:00 2024-04-13T05:00:31.864 INFO:teuthology.orchestra.run.smithi057.stdout:(12/174): cephfs-shell-19.0.0-2884.g218a6914.el 252 kB/s | 27 kB 00:00 2024-04-13T05:00:31.915 INFO:teuthology.orchestra.run.smithi083.stdout:(12/174): cephfs-shell-19.0.0-2884.g218a6914.el 272 kB/s | 27 kB 00:00 2024-04-13T05:00:31.972 INFO:teuthology.orchestra.run.smithi057.stdout:(13/174): libcephfs-devel-19.0.0-2884.g218a6914 294 kB/s | 32 kB 00:00 2024-04-13T05:00:32.039 INFO:teuthology.orchestra.run.smithi083.stdout:(13/174): libcephfs-devel-19.0.0-2884.g218a6914 258 kB/s | 32 kB 00:00 2024-04-13T05:00:32.516 INFO:teuthology.orchestra.run.smithi083.stdout:(14/174): libcephfs2-19.0.0-2884.g218a6914.el9. 1.5 MB/s | 746 kB 00:00 2024-04-13T05:00:32.531 INFO:teuthology.orchestra.run.smithi057.stdout:(14/174): libcephfs2-19.0.0-2884.g218a6914.el9. 1.3 MB/s | 746 kB 00:00 2024-04-13T05:00:32.731 INFO:teuthology.orchestra.run.smithi057.stdout:(15/174): libcephsqlite-19.0.0-2884.g218a6914.e 818 kB/s | 163 kB 00:00 2024-04-13T05:00:32.849 INFO:teuthology.orchestra.run.smithi083.stdout:(15/174): libcephsqlite-19.0.0-2884.g218a6914.e 491 kB/s | 163 kB 00:00 2024-04-13T05:00:32.882 INFO:teuthology.orchestra.run.smithi057.stdout:(16/174): librados-devel-19.0.0-2884.g218a6914. 848 kB/s | 126 kB 00:00 2024-04-13T05:00:33.050 INFO:teuthology.orchestra.run.smithi083.stdout:(16/174): librados-devel-19.0.0-2884.g218a6914. 631 kB/s | 126 kB 00:00 2024-04-13T05:00:33.349 INFO:teuthology.orchestra.run.smithi057.stdout:(17/174): libradosstriper1-19.0.0-2884.g218a691 1.0 MB/s | 500 kB 00:00 2024-04-13T05:00:33.408 INFO:teuthology.orchestra.run.smithi083.stdout:(17/174): libradosstriper1-19.0.0-2884.g218a691 1.4 MB/s | 500 kB 00:00 2024-04-13T05:00:33.501 INFO:teuthology.orchestra.run.smithi189.stdout:(10/174): ceph-common-19.0.0-2884.g218a6914.el9 2.0 MB/s | 21 MB 00:10 2024-04-13T05:00:33.667 INFO:teuthology.orchestra.run.smithi189.stdout:(11/174): ceph-radosgw-19.0.0-2884.g218a6914.el 1.5 MB/s | 10 MB 00:06 2024-04-13T05:00:33.785 INFO:teuthology.orchestra.run.smithi189.stdout:(12/174): cephfs-shell-19.0.0-2884.g218a6914.el 233 kB/s | 27 kB 00:00 2024-04-13T05:00:33.843 INFO:teuthology.orchestra.run.smithi189.stdout:(13/174): cephfs-mirror-19.0.0-2884.g218a6914.e 659 kB/s | 225 kB 00:00 2024-04-13T05:00:33.882 INFO:teuthology.orchestra.run.smithi057.stdout:(18/174): ceph-common-19.0.0-2884.g218a6914.el9 1.6 MB/s | 21 MB 00:12 2024-04-13T05:00:33.902 INFO:teuthology.orchestra.run.smithi189.stdout:(14/174): libcephfs-devel-19.0.0-2884.g218a6914 274 kB/s | 32 kB 00:00 2024-04-13T05:00:34.082 INFO:teuthology.orchestra.run.smithi057.stdout:(19/174): python3-ceph-argparse-19.0.0-2884.g21 227 kB/s | 45 kB 00:00 2024-04-13T05:00:34.185 INFO:teuthology.orchestra.run.smithi189.stdout:(15/174): libcephsqlite-19.0.0-2884.g218a6914.e 578 kB/s | 163 kB 00:00 2024-04-13T05:00:34.291 INFO:teuthology.orchestra.run.smithi057.stdout:(20/174): python3-ceph-common-19.0.0-2884.g218a 648 kB/s | 135 kB 00:00 2024-04-13T05:00:34.438 INFO:teuthology.orchestra.run.smithi189.stdout:(16/174): librados-devel-19.0.0-2884.g218a6914. 503 kB/s | 126 kB 00:00 2024-04-13T05:00:34.542 INFO:teuthology.orchestra.run.smithi057.stdout:(21/174): python3-cephfs-19.0.0-2884.g218a6914. 649 kB/s | 162 kB 00:00 2024-04-13T05:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout:(17/174): libcephfs2-19.0.0-2884.g218a6914.el9. 807 kB/s | 746 kB 00:00 2024-04-13T05:00:34.942 INFO:teuthology.orchestra.run.smithi057.stdout:(22/174): python3-rados-19.0.0-2884.g218a6914.e 804 kB/s | 321 kB 00:00 2024-04-13T05:00:35.019 INFO:teuthology.orchestra.run.smithi189.stdout:(18/174): libradosstriper1-19.0.0-2884.g218a691 862 kB/s | 500 kB 00:00 2024-04-13T05:00:35.137 INFO:teuthology.orchestra.run.smithi189.stdout:(19/174): python3-ceph-argparse-19.0.0-2884.g21 387 kB/s | 45 kB 00:00 2024-04-13T05:00:35.250 INFO:teuthology.orchestra.run.smithi057.stdout:(23/174): python3-rbd-19.0.0-2884.g218a6914.el9 969 kB/s | 298 kB 00:00 2024-04-13T05:00:35.354 INFO:teuthology.orchestra.run.smithi189.stdout:(20/174): python3-ceph-common-19.0.0-2884.g218a 624 kB/s | 135 kB 00:00 2024-04-13T05:00:35.367 INFO:teuthology.orchestra.run.smithi057.stdout:(24/174): python3-rgw-19.0.0-2884.g218a6914.el9 859 kB/s | 100 kB 00:00 2024-04-13T05:00:35.485 INFO:teuthology.orchestra.run.smithi057.stdout:(25/174): rbd-fuse-19.0.0-2884.g218a6914.el9.x8 735 kB/s | 85 kB 00:00 2024-04-13T05:00:35.554 INFO:teuthology.orchestra.run.smithi189.stdout:(21/174): python3-cephfs-19.0.0-2884.g218a6914. 813 kB/s | 162 kB 00:00 2024-04-13T05:00:35.921 INFO:teuthology.orchestra.run.smithi189.stdout:(22/174): python3-rados-19.0.0-2884.g218a6914.e 876 kB/s | 321 kB 00:00 2024-04-13T05:00:36.221 INFO:teuthology.orchestra.run.smithi189.stdout:(23/174): python3-rbd-19.0.0-2884.g218a6914.el9 996 kB/s | 298 kB 00:00 2024-04-13T05:00:36.346 INFO:teuthology.orchestra.run.smithi189.stdout:(24/174): python3-rgw-19.0.0-2884.g218a6914.el9 800 kB/s | 100 kB 00:00 2024-04-13T05:00:36.463 INFO:teuthology.orchestra.run.smithi189.stdout:(25/174): rbd-fuse-19.0.0-2884.g218a6914.el9.x8 736 kB/s | 85 kB 00:00 2024-04-13T05:00:37.092 INFO:teuthology.orchestra.run.smithi083.stdout:(18/174): librgw2-19.0.0-2884.g218a6914.el9.x86 1.4 MB/s | 5.2 MB 00:03 2024-04-13T05:00:37.192 INFO:teuthology.orchestra.run.smithi083.stdout:(19/174): python3-ceph-argparse-19.0.0-2884.g21 456 kB/s | 45 kB 00:00 2024-04-13T05:00:37.359 INFO:teuthology.orchestra.run.smithi083.stdout:(20/174): python3-ceph-common-19.0.0-2884.g218a 814 kB/s | 135 kB 00:00 2024-04-13T05:00:37.609 INFO:teuthology.orchestra.run.smithi083.stdout:(21/174): python3-cephfs-19.0.0-2884.g218a6914. 649 kB/s | 162 kB 00:00 2024-04-13T05:00:37.976 INFO:teuthology.orchestra.run.smithi083.stdout:(22/174): python3-rados-19.0.0-2884.g218a6914.e 877 kB/s | 321 kB 00:00 2024-04-13T05:00:38.026 INFO:teuthology.orchestra.run.smithi057.stdout:(26/174): rbd-mirror-19.0.0-2884.g218a6914.el9. 1.2 MB/s | 3.1 MB 00:02 2024-04-13T05:00:38.193 INFO:teuthology.orchestra.run.smithi057.stdout:(27/174): rbd-nbd-19.0.0-2884.g218a6914.el9.x86 1.0 MB/s | 170 kB 00:00 2024-04-13T05:00:38.226 INFO:teuthology.orchestra.run.smithi083.stdout:(23/174): python3-rbd-19.0.0-2884.g218a6914.el9 1.2 MB/s | 298 kB 00:00 2024-04-13T05:00:38.310 INFO:teuthology.orchestra.run.smithi057.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2884.g 295 kB/s | 34 kB 00:00 2024-04-13T05:00:38.402 INFO:teuthology.orchestra.run.smithi083.stdout:(24/174): python3-rgw-19.0.0-2884.g218a6914.el9 572 kB/s | 100 kB 00:00 2024-04-13T05:00:38.527 INFO:teuthology.orchestra.run.smithi057.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2884.g218a691 658 kB/s | 142 kB 00:00 2024-04-13T05:00:38.569 INFO:teuthology.orchestra.run.smithi083.stdout:(25/174): rbd-fuse-19.0.0-2884.g218a6914.el9.x8 513 kB/s | 85 kB 00:00 2024-04-13T05:00:39.013 INFO:teuthology.orchestra.run.smithi189.stdout:(26/174): rbd-mirror-19.0.0-2884.g218a6914.el9. 1.2 MB/s | 3.1 MB 00:02 2024-04-13T05:00:39.271 INFO:teuthology.orchestra.run.smithi057.stdout:(30/174): librgw2-19.0.0-2884.g218a6914.el9.x86 903 kB/s | 5.2 MB 00:05 2024-04-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi189.stdout:(27/174): librgw2-19.0.0-2884.g218a6914.el9.x86 1.1 MB/s | 5.2 MB 00:04 2024-04-13T05:00:39.347 INFO:teuthology.orchestra.run.smithi189.stdout:(28/174): rbd-nbd-19.0.0-2884.g218a6914.el9.x86 513 kB/s | 170 kB 00:00 2024-04-13T05:00:39.406 INFO:teuthology.orchestra.run.smithi189.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2884.g 375 kB/s | 34 kB 00:00 2024-04-13T05:00:39.665 INFO:teuthology.orchestra.run.smithi189.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2884.g218a691 449 kB/s | 142 kB 00:00 2024-04-13T05:00:41.102 INFO:teuthology.orchestra.run.smithi083.stdout:(26/174): rbd-mirror-19.0.0-2884.g218a6914.el9. 1.2 MB/s | 3.1 MB 00:02 2024-04-13T05:00:41.277 INFO:teuthology.orchestra.run.smithi083.stdout:(27/174): rbd-nbd-19.0.0-2884.g218a6914.el9.x86 980 kB/s | 170 kB 00:00 2024-04-13T05:00:41.311 INFO:teuthology.orchestra.run.smithi057.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2884.g218a6 1.3 MB/s | 3.5 MB 00:02 2024-04-13T05:00:41.361 INFO:teuthology.orchestra.run.smithi083.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2884.g 413 kB/s | 34 kB 00:00 2024-04-13T05:00:41.536 INFO:teuthology.orchestra.run.smithi057.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2884.g21 1.1 MB/s | 249 kB 00:00 2024-04-13T05:00:41.628 INFO:teuthology.orchestra.run.smithi057.stdout:(33/174): ceph-mgr-rook-19.0.0-2884.g218a6914.e 540 kB/s | 49 kB 00:00 2024-04-13T05:00:41.745 INFO:teuthology.orchestra.run.smithi057.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2884.g2 142 kB/s | 16 kB 00:00 2024-04-13T05:00:41.753 INFO:teuthology.orchestra.run.smithi083.stdout:(29/174): ceph-osd-19.0.0-2884.g218a6914.el9.x8 1.2 MB/s | 17 MB 00:14 2024-04-13T05:00:41.778 INFO:teuthology.orchestra.run.smithi083.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2884.g218a691 341 kB/s | 142 kB 00:00 2024-04-13T05:00:41.946 INFO:teuthology.orchestra.run.smithi057.stdout:(35/174): ceph-volume-19.0.0-2884.g218a6914.el9 1.3 MB/s | 264 kB 00:00 2024-04-13T05:00:42.198 INFO:teuthology.orchestra.run.smithi189.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2884.g218a6 1.3 MB/s | 3.5 MB 00:02 2024-04-13T05:00:42.398 INFO:teuthology.orchestra.run.smithi189.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2884.g21 1.2 MB/s | 249 kB 00:00 2024-04-13T05:00:42.473 INFO:teuthology.orchestra.run.smithi189.stdout:(33/174): ceph-mgr-rook-19.0.0-2884.g218a6914.e 664 kB/s | 49 kB 00:00 2024-04-13T05:00:42.549 INFO:teuthology.orchestra.run.smithi189.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2884.g2 221 kB/s | 16 kB 00:00 2024-04-13T05:00:42.749 INFO:teuthology.orchestra.run.smithi189.stdout:(35/174): ceph-volume-19.0.0-2884.g218a6914.el9 1.3 MB/s | 264 kB 00:00 2024-04-13T05:00:42.805 INFO:teuthology.orchestra.run.smithi057.stdout:(36/174): ceph-test-19.0.0-2884.g218a6914.el9.x 2.8 MB/s | 49 MB 00:17 2024-04-13T05:00:42.846 INFO:teuthology.orchestra.run.smithi057.stdout:(37/174): cephadm-19.0.0-2884.g218a6914.el9.noa 848 kB/s | 763 kB 00:00 2024-04-13T05:00:42.955 INFO:teuthology.orchestra.run.smithi057.stdout:(38/174): cephfs-top-19.0.0-2884.g218a6914.el9. 122 kB/s | 18 kB 00:00 2024-04-13T05:00:42.988 INFO:teuthology.orchestra.run.smithi057.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 125 kB/s | 18 kB 00:00 2024-04-13T05:00:43.266 INFO:teuthology.orchestra.run.smithi189.stdout:(36/174): cephadm-19.0.0-2884.g218a6914.el9.noa 1.4 MB/s | 763 kB 00:00 2024-04-13T05:00:43.350 INFO:teuthology.orchestra.run.smithi189.stdout:(37/174): cephfs-top-19.0.0-2884.g218a6914.el9. 221 kB/s | 18 kB 00:00 2024-04-13T05:00:43.484 INFO:teuthology.orchestra.run.smithi189.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 133 kB/s | 18 kB 00:00 2024-04-13T05:00:43.625 INFO:teuthology.orchestra.run.smithi189.stdout:(39/174): fuse-2.9.9-15.el9.x86_64.rpm 570 kB/s | 80 kB 00:00 2024-04-13T05:00:43.701 INFO:teuthology.orchestra.run.smithi189.stdout:(40/174): jq-1.6-16.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-04-13T05:00:43.756 INFO:teuthology.orchestra.run.smithi057.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 101 kB/s | 80 kB 00:00 2024-04-13T05:00:43.793 INFO:teuthology.orchestra.run.smithi189.stdout:(41/174): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-13T05:00:43.843 INFO:teuthology.orchestra.run.smithi189.stdout:(42/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-13T05:00:43.910 INFO:teuthology.orchestra.run.smithi189.stdout:(43/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 218 kB 00:00 2024-04-13T05:00:43.956 INFO:teuthology.orchestra.run.smithi057.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 194 kB/s | 187 kB 00:00 2024-04-13T05:00:43.977 INFO:teuthology.orchestra.run.smithi189.stdout:(44/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.8 MB/s | 253 kB 00:00 2024-04-13T05:00:44.094 INFO:teuthology.orchestra.run.smithi189.stdout:(45/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-13T05:00:44.219 INFO:teuthology.orchestra.run.smithi189.stdout:(46/174): ceph-mgr-diskprediction-local-19.0.0- 1.6 MB/s | 7.4 MB 00:04 2024-04-13T05:00:44.253 INFO:teuthology.orchestra.run.smithi189.stdout:(47/174): python3-ply-3.11-14.el9.noarch.rpm 674 kB/s | 106 kB 00:00 2024-04-13T05:00:44.328 INFO:teuthology.orchestra.run.smithi189.stdout:(48/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-13T05:00:44.387 INFO:teuthology.orchestra.run.smithi189.stdout:(49/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-13T05:00:44.437 INFO:teuthology.orchestra.run.smithi189.stdout:(50/174): python3-pycparser-2.20-6.el9.noarch.r 624 kB/s | 135 kB 00:00 2024-04-13T05:00:45.123 INFO:teuthology.orchestra.run.smithi057.stdout:(42/174): libquadmath-11.4.1-3.el9.x86_64.rpm 168 kB/s | 195 kB 00:01 2024-04-13T05:00:45.170 INFO:teuthology.orchestra.run.smithi083.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-04-13T05:00:45.295 INFO:teuthology.orchestra.run.smithi083.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2884.g21 2.0 MB/s | 249 kB 00:00 2024-04-13T05:00:45.329 INFO:teuthology.orchestra.run.smithi189.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 732 kB/s | 689 kB 00:00 2024-04-13T05:00:45.371 INFO:teuthology.orchestra.run.smithi083.stdout:(33/174): ceph-mgr-rook-19.0.0-2884.g218a6914.e 664 kB/s | 49 kB 00:00 2024-04-13T05:00:45.463 INFO:teuthology.orchestra.run.smithi189.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 807 kB/s | 107 kB 00:00 2024-04-13T05:00:45.463 INFO:teuthology.orchestra.run.smithi083.stdout:(34/174): ceph-mgr-dashboard-19.0.0-2884.g218a6 977 kB/s | 3.5 MB 00:03 2024-04-13T05:00:45.488 INFO:teuthology.orchestra.run.smithi083.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2884.g2 141 kB/s | 16 kB 00:00 2024-04-13T05:00:45.498 INFO:teuthology.orchestra.run.smithi057.stdout:(43/174): ceph-mgr-diskprediction-local-19.0.0- 1.2 MB/s | 7.4 MB 00:06 2024-04-13T05:00:45.505 INFO:teuthology.orchestra.run.smithi189.stdout:(53/174): bison-3.7.4-5.el9.x86_64.rpm 884 kB/s | 943 kB 00:01 2024-04-13T05:00:45.572 INFO:teuthology.orchestra.run.smithi189.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-13T05:00:45.630 INFO:teuthology.orchestra.run.smithi189.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-04-13T05:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-04-13T05:00:45.738 INFO:teuthology.orchestra.run.smithi083.stdout:(36/174): ceph-volume-19.0.0-2884.g218a6914.el9 961 kB/s | 264 kB 00:00 2024-04-13T05:00:45.806 INFO:teuthology.orchestra.run.smithi189.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 157 kB/s | 18 kB 00:00 2024-04-13T05:00:45.822 INFO:teuthology.orchestra.run.smithi083.stdout:(37/174): cephadm-19.0.0-2884.g218a6914.el9.noa 2.2 MB/s | 763 kB 00:00 2024-04-13T05:00:45.847 INFO:teuthology.orchestra.run.smithi083.stdout:(38/174): cephfs-top-19.0.0-2884.g218a6914.el9. 169 kB/s | 18 kB 00:00 2024-04-13T05:00:45.931 INFO:teuthology.orchestra.run.smithi189.stdout:(58/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-13T05:00:45.981 INFO:teuthology.orchestra.run.smithi057.stdout:(44/174): libgfortran-11.4.1-3.el9.x86_64.rpm 361 kB/s | 803 kB 00:02 2024-04-13T05:00:46.048 INFO:teuthology.orchestra.run.smithi083.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 78 kB/s | 18 kB 00:00 2024-04-13T05:00:46.073 INFO:teuthology.orchestra.run.smithi189.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 321 kB/s | 45 kB 00:00 2024-04-13T05:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout:(60/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.7 MB/s | 3.0 MB 00:00 2024-04-13T05:00:46.398 INFO:teuthology.orchestra.run.smithi057.stdout:(45/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 171 kB/s | 218 kB 00:01 2024-04-13T05:00:46.816 INFO:teuthology.orchestra.run.smithi189.stdout:(61/174): ceph-test-19.0.0-2884.g218a6914.el9.x 2.9 MB/s | 49 MB 00:16 2024-04-13T05:00:46.840 INFO:teuthology.orchestra.run.smithi083.stdout:(40/174): ceph-test-19.0.0-2884.g218a6914.el9.x 3.0 MB/s | 49 MB 00:16 2024-04-13T05:00:46.858 INFO:teuthology.orchestra.run.smithi189.stdout:(62/174): librdkafka-1.6.1-102.el9.x86_64.rpm 846 kB/s | 662 kB 00:00 2024-04-13T05:00:46.865 INFO:teuthology.orchestra.run.smithi083.stdout:(41/174): fuse-2.9.9-15.el9.x86_64.rpm 79 kB/s | 80 kB 00:01 2024-04-13T05:00:46.891 INFO:teuthology.orchestra.run.smithi189.stdout:(63/174): libtool-2.4.6-45.el9.x86_64.rpm 932 kB/s | 582 kB 00:00 2024-04-13T05:00:46.957 INFO:teuthology.orchestra.run.smithi057.stdout:(46/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 183 kB/s | 253 kB 00:01 2024-04-13T05:00:47.017 INFO:teuthology.orchestra.run.smithi189.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 300 kB/s | 47 kB 00:00 2024-04-13T05:00:47.050 INFO:teuthology.orchestra.run.smithi189.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-04-13T05:00:47.151 INFO:teuthology.orchestra.run.smithi189.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-13T05:00:47.176 INFO:teuthology.orchestra.run.smithi189.stdout:(67/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 71 kB/s | 25 kB 00:00 2024-04-13T05:00:47.201 INFO:teuthology.orchestra.run.smithi189.stdout:(68/174): openblas-0.3.21-2.el9.x86_64.rpm 235 kB/s | 35 kB 00:00 2024-04-13T05:00:47.207 INFO:teuthology.orchestra.run.smithi083.stdout:(42/174): jq-1.6-16.el9.x86_64.rpm 162 kB/s | 187 kB 00:01 2024-04-13T05:00:47.457 INFO:teuthology.orchestra.run.smithi057.stdout:(47/174): python3-ply-3.11-14.el9.noarch.rpm 101 kB/s | 106 kB 00:01 2024-04-13T05:00:47.699 INFO:teuthology.orchestra.run.smithi083.stdout:(43/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 444 kB/s | 218 kB 00:00 2024-04-13T05:00:47.874 INFO:teuthology.orchestra.run.smithi083.stdout:(44/174): libquadmath-11.4.1-3.el9.x86_64.rpm 194 kB/s | 195 kB 00:01 2024-04-13T05:00:48.149 INFO:teuthology.orchestra.run.smithi083.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 563 kB/s | 253 kB 00:00 2024-04-13T05:00:48.433 INFO:teuthology.orchestra.run.smithi083.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 377 kB/s | 106 kB 00:00 2024-04-13T05:00:48.557 INFO:teuthology.orchestra.run.smithi057.stdout:(48/174): python3-requests-2.25.1-8.el9.noarch. 114 kB/s | 125 kB 00:01 2024-04-13T05:00:48.617 INFO:teuthology.orchestra.run.smithi083.stdout:(47/174): libgfortran-11.4.1-3.el9.x86_64.rpm 452 kB/s | 803 kB 00:01 2024-04-13T05:00:48.667 INFO:teuthology.orchestra.run.smithi083.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 580 kB/s | 135 kB 00:00 2024-04-13T05:00:48.725 INFO:teuthology.orchestra.run.smithi083.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-13T05:00:48.951 INFO:teuthology.orchestra.run.smithi083.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 759 kB/s | 215 kB 00:00 2024-04-13T05:00:48.983 INFO:teuthology.orchestra.run.smithi057.stdout:(49/174): python3-pycparser-2.20-6.el9.noarch.r 67 kB/s | 135 kB 00:02 2024-04-13T05:00:49.100 INFO:teuthology.orchestra.run.smithi057.stdout:(50/174): python3-cryptography-36.0.1-4.el9.x86 403 kB/s | 1.2 MB 00:03 2024-04-13T05:00:49.600 INFO:teuthology.orchestra.run.smithi057.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-04-13T05:00:49.675 INFO:teuthology.orchestra.run.smithi057.stdout:(52/174): python3-urllib3-1.26.5-5.el9.noarch.r 192 kB/s | 215 kB 00:01 2024-04-13T05:00:49.726 INFO:teuthology.orchestra.run.smithi057.stdout:(53/174): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 943 kB 00:00 2024-04-13T05:00:49.759 INFO:teuthology.orchestra.run.smithi057.stdout:(54/174): boost-program-options-1.75.0-8.el9.x8 676 kB/s | 107 kB 00:00 2024-04-13T05:00:49.776 INFO:teuthology.orchestra.run.smithi057.stdout:(55/174): elfutils-libelf-devel-0.190-2.el9.x86 217 kB/s | 21 kB 00:00 2024-04-13T05:00:49.977 INFO:teuthology.orchestra.run.smithi057.stdout:(56/174): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-04-13T05:00:50.043 INFO:teuthology.orchestra.run.smithi057.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 287 kB/s | 18 kB 00:00 2024-04-13T05:00:50.061 INFO:teuthology.orchestra.run.smithi057.stdout:(58/174): flexiblas-3.0.4-8.el9.x86_64.rpm 109 kB/s | 33 kB 00:00 2024-04-13T05:00:50.161 INFO:teuthology.orchestra.run.smithi057.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-13T05:00:50.178 INFO:teuthology.orchestra.run.smithi057.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 390 kB/s | 45 kB 00:00 2024-04-13T05:00:50.262 INFO:teuthology.orchestra.run.smithi057.stdout:(61/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.2 MB/s | 3.0 MB 00:00 2024-04-13T05:00:50.326 INFO:teuthology.orchestra.run.smithi083.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 431 kB/s | 689 kB 00:01 2024-04-13T05:00:50.401 INFO:teuthology.orchestra.run.smithi083.stdout:(52/174): python3-cryptography-36.0.1-4.el9.x86 497 kB/s | 1.2 MB 00:02 2024-04-13T05:00:50.439 INFO:teuthology.orchestra.run.smithi057.stdout:(62/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 144 kB/s | 25 kB 00:00 2024-04-13T05:00:50.479 INFO:teuthology.orchestra.run.smithi057.stdout:(63/174): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-04-13T05:00:50.512 INFO:teuthology.orchestra.run.smithi057.stdout:(64/174): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-04-13T05:00:50.530 INFO:teuthology.orchestra.run.smithi057.stdout:(65/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 531 kB/s | 47 kB 00:00 2024-04-13T05:00:50.597 INFO:teuthology.orchestra.run.smithi057.stdout:(66/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-13T05:00:50.613 INFO:teuthology.orchestra.run.smithi057.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 424 kB/s | 35 kB 00:00 2024-04-13T05:00:50.647 INFO:teuthology.orchestra.run.smithi057.stdout:(68/174): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-13T05:00:50.685 INFO:teuthology.orchestra.run.smithi083.stdout:(53/174): elfutils-libelf-devel-0.190-2.el9.x86 76 kB/s | 21 kB 00:00 2024-04-13T05:00:50.860 INFO:teuthology.orchestra.run.smithi083.stdout:(54/174): boost-program-options-1.75.0-8.el9.x8 200 kB/s | 107 kB 00:00 2024-04-13T05:00:50.907 INFO:teuthology.orchestra.run.smithi057.stdout:(69/174): postgresql-13.14-1.el9.x86_64.rpm 6.2 MB/s | 1.6 MB 00:00 2024-04-13T05:00:50.910 INFO:teuthology.orchestra.run.smithi083.stdout:(55/174): bison-3.7.4-5.el9.x86_64.rpm 481 kB/s | 943 kB 00:01 2024-04-13T05:00:51.056 INFO:teuthology.orchestra.run.smithi057.stdout:(70/174): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-13T05:00:51.123 INFO:teuthology.orchestra.run.smithi057.stdout:(71/174): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-04-13T05:00:51.127 INFO:teuthology.orchestra.run.smithi083.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 123 kB/s | 33 kB 00:00 2024-04-13T05:00:51.174 INFO:teuthology.orchestra.run.smithi057.stdout:(72/174): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-13T05:00:51.258 INFO:teuthology.orchestra.run.smithi057.stdout:(73/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 6.4 MB/s | 4.1 MB 00:00 2024-04-13T05:00:51.333 INFO:teuthology.orchestra.run.smithi057.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-13T05:00:51.370 INFO:teuthology.orchestra.run.smithi083.stdout:(57/174): flex-2.6.4-9.el9.x86_64.rpm 459 kB/s | 313 kB 00:00 2024-04-13T05:00:51.395 INFO:teuthology.orchestra.run.smithi083.stdout:(58/174): flexiblas-openblas-openmp-3.0.4-8.el9 68 kB/s | 18 kB 00:00 2024-04-13T05:00:51.418 INFO:teuthology.orchestra.run.smithi057.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-04-13T05:00:51.508 INFO:teuthology.orchestra.run.smithi057.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.7 MB/s | 245 kB 00:00 2024-04-13T05:00:51.600 INFO:teuthology.orchestra.run.smithi057.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-04-13T05:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout:(69/174): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-04-13T05:00:51.717 INFO:teuthology.orchestra.run.smithi057.stdout:(78/174): postgresql-server-13.14-1.el9.x86_64. 9.7 MB/s | 5.8 MB 00:00 2024-04-13T05:00:51.743 INFO:teuthology.orchestra.run.smithi057.stdout:(79/174): python3-jsonpatch-1.21-16.el9.noarch. 181 kB/s | 26 kB 00:00 2024-04-13T05:00:51.745 INFO:teuthology.orchestra.run.smithi083.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 129 kB/s | 45 kB 00:00 2024-04-13T05:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout:(70/174): postgresql-contrib-13.14-1.el9.x86_64 6.8 MB/s | 911 kB 00:00 2024-04-13T05:00:51.793 INFO:teuthology.orchestra.run.smithi057.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-04-13T05:00:51.826 INFO:teuthology.orchestra.run.smithi057.stdout:(81/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-13T05:00:51.877 INFO:teuthology.orchestra.run.smithi057.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 35 kB 00:00 2024-04-13T05:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout:(71/174): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-13T05:00:51.978 INFO:teuthology.orchestra.run.smithi083.stdout:(60/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 263 kB/s | 160 kB 00:00 2024-04-13T05:00:51.985 INFO:teuthology.orchestra.run.smithi057.stdout:(83/174): python3-babel-2.9.1-2.el9.noarch.rpm 9.1 MB/s | 6.0 MB 00:00 2024-04-13T05:00:52.070 INFO:teuthology.orchestra.run.smithi057.stdout:(84/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-13T05:00:52.111 INFO:teuthology.orchestra.run.smithi057.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-13T05:00:52.162 INFO:teuthology.orchestra.run.smithi057.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 848 kB/s | 77 kB 00:00 2024-04-13T05:00:52.195 INFO:teuthology.orchestra.run.smithi057.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 512 kB/s | 42 kB 00:00 2024-04-13T05:00:52.271 INFO:teuthology.orchestra.run.smithi057.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-13T05:00:52.304 INFO:teuthology.orchestra.run.smithi057.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-13T05:00:52.421 INFO:teuthology.orchestra.run.smithi057.stdout:(90/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.6 MB/s | 5.1 MB 00:00 2024-04-13T05:00:52.463 INFO:teuthology.orchestra.run.smithi057.stdout:(91/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-04-13T05:00:52.480 INFO:teuthology.orchestra.run.smithi057.stdout:(92/174): python3-pytz-2021.1-5.el9.noarch.rpm 292 kB/s | 51 kB 00:00 2024-04-13T05:00:52.505 INFO:teuthology.orchestra.run.smithi057.stdout:(93/174): python3-requests-oauthlib-1.3.0-12.el 648 kB/s | 54 kB 00:00 2024-04-13T05:00:52.564 INFO:teuthology.orchestra.run.smithi057.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 501 kB/s | 42 kB 00:00 2024-04-13T05:00:52.606 INFO:teuthology.orchestra.run.smithi057.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 445 kB/s | 44 kB 00:00 2024-04-13T05:00:52.698 INFO:teuthology.orchestra.run.smithi057.stdout:(96/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-13T05:00:52.712 INFO:teuthology.orchestra.run.smithi083.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 686 kB/s | 662 kB 00:00 2024-04-13T05:00:52.721 INFO:teuthology.orchestra.run.smithi057.stdout:(97/174): qatlib-service-24.02.0-1.el9.x86_64.r 317 kB/s | 36 kB 00:00 2024-04-13T05:00:52.804 INFO:teuthology.orchestra.run.smithi057.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 450 kB/s | 47 kB 00:00 2024-04-13T05:00:52.852 INFO:teuthology.orchestra.run.smithi189.stdout:(72/174): postgresql-server-13.14-1.el9.x86_64. 6.1 MB/s | 5.8 MB 00:00 2024-04-13T05:00:52.855 INFO:teuthology.orchestra.run.smithi057.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-13T05:00:52.904 INFO:teuthology.orchestra.run.smithi083.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 630 kB/s | 582 kB 00:00 2024-04-13T05:00:52.922 INFO:teuthology.orchestra.run.smithi057.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 496 kB/s | 58 kB 00:00 2024-04-13T05:00:52.963 INFO:teuthology.orchestra.run.smithi083.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 102 kB/s | 25 kB 00:00 2024-04-13T05:00:52.965 INFO:teuthology.orchestra.run.smithi057.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 866 kB/s | 94 kB 00:00 2024-04-13T05:00:53.019 INFO:teuthology.orchestra.run.smithi189.stdout:(73/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-13T05:00:53.048 INFO:teuthology.orchestra.run.smithi057.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 509 kB/s | 64 kB 00:00 2024-04-13T05:00:53.073 INFO:teuthology.orchestra.run.smithi057.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-04-13T05:00:53.140 INFO:teuthology.orchestra.run.smithi057.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-04-13T05:00:53.152 INFO:teuthology.orchestra.run.smithi189.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 660 kB/s | 87 kB 00:00 2024-04-13T05:00:53.246 INFO:teuthology.orchestra.run.smithi083.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-04-13T05:00:53.466 INFO:teuthology.orchestra.run.smithi057.stdout:(105/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-13T05:00:53.480 INFO:teuthology.orchestra.run.smithi083.stdout:(65/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.2 MB/s | 3.0 MB 00:02 2024-04-13T05:00:53.516 INFO:teuthology.orchestra.run.smithi057.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-13T05:00:53.545 INFO:teuthology.orchestra.run.smithi189.stdout:(75/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 664 kB/s | 4.1 MB 00:06 2024-04-13T05:00:53.611 INFO:teuthology.orchestra.run.smithi189.stdout:(76/174): postgresql-13.14-1.el9.x86_64.rpm 254 kB/s | 1.6 MB 00:06 2024-04-13T05:00:53.692 INFO:teuthology.orchestra.run.smithi057.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 112 kB/s | 19 kB 00:00 2024-04-13T05:00:53.742 INFO:teuthology.orchestra.run.smithi057.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-04-13T05:00:53.753 INFO:teuthology.orchestra.run.smithi189.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-13T05:00:53.755 INFO:teuthology.orchestra.run.smithi083.stdout:(66/174): openblas-0.3.21-2.el9.x86_64.rpm 128 kB/s | 35 kB 00:00 2024-04-13T05:00:53.787 INFO:teuthology.orchestra.run.smithi189.stdout:(78/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-04-13T05:00:53.789 INFO:teuthology.orchestra.run.smithi083.stdout:(67/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 354 kB/s | 292 kB 00:00 2024-04-13T05:00:53.800 INFO:teuthology.orchestra.run.smithi057.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 926 kB/s | 309 kB 00:00 2024-04-13T05:00:53.823 INFO:teuthology.orchestra.run.smithi083.stdout:(68/174): lua-5.4.4-4.el9.x86_64.rpm 327 kB/s | 188 kB 00:00 2024-04-13T05:00:53.826 INFO:teuthology.orchestra.run.smithi057.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 314 kB/s | 26 kB 00:00 2024-04-13T05:00:53.871 INFO:teuthology.orchestra.run.smithi189.stdout:(79/174): python3-jsonpatch-1.21-16.el9.noarch. 221 kB/s | 26 kB 00:00 2024-04-13T05:00:53.884 INFO:teuthology.orchestra.run.smithi057.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-04-13T05:00:53.904 INFO:teuthology.orchestra.run.smithi189.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 165 kB/s | 19 kB 00:00 2024-04-13T05:00:53.910 INFO:teuthology.orchestra.run.smithi057.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 813 kB/s | 67 kB 00:00 2024-04-13T05:00:53.988 INFO:teuthology.orchestra.run.smithi189.stdout:(81/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-13T05:00:54.010 INFO:teuthology.orchestra.run.smithi057.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-13T05:00:54.021 INFO:teuthology.orchestra.run.smithi189.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-04-13T05:00:54.149 INFO:teuthology.orchestra.run.smithi189.stdout:(83/174): python3-babel-2.9.1-2.el9.noarch.rpm 6.0 MB/s | 6.0 MB 00:00 2024-04-13T05:00:54.218 INFO:teuthology.orchestra.run.smithi057.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-13T05:00:54.281 INFO:teuthology.orchestra.run.smithi189.stdout:(84/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 971 kB/s | 250 kB 00:00 2024-04-13T05:00:54.302 INFO:teuthology.orchestra.run.smithi057.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 525 kB/s | 43 kB 00:00 2024-04-13T05:00:54.306 INFO:teuthology.orchestra.run.smithi189.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-13T05:00:54.377 INFO:teuthology.orchestra.run.smithi057.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-04-13T05:00:54.394 INFO:teuthology.orchestra.run.smithi057.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 354 kB/s | 32 kB 00:00 2024-04-13T05:00:54.423 INFO:teuthology.orchestra.run.smithi189.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 546 kB/s | 77 kB 00:00 2024-04-13T05:00:54.448 INFO:teuthology.orchestra.run.smithi189.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 299 kB/s | 42 kB 00:00 2024-04-13T05:00:54.536 INFO:teuthology.orchestra.run.smithi057.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 95 kB/s | 15 kB 00:00 2024-04-13T05:00:54.565 INFO:teuthology.orchestra.run.smithi189.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-04-13T05:00:54.570 INFO:teuthology.orchestra.run.smithi057.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 985 kB/s | 172 kB 00:00 2024-04-13T05:00:54.598 INFO:teuthology.orchestra.run.smithi189.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-13T05:00:54.699 INFO:teuthology.orchestra.run.smithi189.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 279 kB 00:00 2024-04-13T05:00:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-13T05:00:54.724 INFO:teuthology.orchestra.run.smithi189.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 409 kB/s | 51 kB 00:00 2024-04-13T05:00:54.754 INFO:teuthology.orchestra.run.smithi057.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-04-13T05:00:54.787 INFO:teuthology.orchestra.run.smithi057.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-13T05:00:54.808 INFO:teuthology.orchestra.run.smithi189.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 497 kB/s | 54 kB 00:00 2024-04-13T05:00:54.813 INFO:teuthology.orchestra.run.smithi057.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 387 kB/s | 35 kB 00:00 2024-04-13T05:00:54.888 INFO:teuthology.orchestra.run.smithi057.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-13T05:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-04-13T05:00:54.963 INFO:teuthology.orchestra.run.smithi057.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 144 kB/s | 11 kB 00:00 2024-04-13T05:00:55.005 INFO:teuthology.orchestra.run.smithi057.stdout:(126/174): python3-google-auth-2.28.2-1.el9.noa 810 kB/s | 203 kB 00:00 2024-04-13T05:00:55.033 INFO:teuthology.orchestra.run.smithi189.stdout:(94/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 381 kB/s | 44 kB 00:00 2024-04-13T05:00:55.039 INFO:teuthology.orchestra.run.smithi057.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-04-13T05:00:55.089 INFO:teuthology.orchestra.run.smithi057.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-13T05:00:55.115 INFO:teuthology.orchestra.run.smithi057.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-04-13T05:00:55.167 INFO:teuthology.orchestra.run.smithi189.stdout:(95/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-13T05:00:55.173 INFO:teuthology.orchestra.run.smithi057.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-13T05:00:55.274 INFO:teuthology.orchestra.run.smithi057.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 467 kB/s | 46 kB 00:00 2024-04-13T05:00:55.292 INFO:teuthology.orchestra.run.smithi189.stdout:(96/174): qatlib-service-24.02.0-1.el9.x86_64.r 288 kB/s | 36 kB 00:00 2024-04-13T05:00:55.332 INFO:teuthology.orchestra.run.smithi057.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-13T05:00:55.358 INFO:teuthology.orchestra.run.smithi057.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 952 kB/s | 79 kB 00:00 2024-04-13T05:00:55.408 INFO:teuthology.orchestra.run.smithi057.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 780 kB/s | 58 kB 00:00 2024-04-13T05:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout:(97/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 408 kB/s | 47 kB 00:00 2024-04-13T05:00:55.483 INFO:teuthology.orchestra.run.smithi057.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 221 kB/s | 16 kB 00:00 2024-04-13T05:00:55.517 INFO:teuthology.orchestra.run.smithi057.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-13T05:00:55.523 INFO:teuthology.orchestra.run.smithi083.stdout:(69/174): postgresql-13.14-1.el9.x86_64.rpm 957 kB/s | 1.6 MB 00:01 2024-04-13T05:00:55.534 INFO:teuthology.orchestra.run.smithi189.stdout:(98/174): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-13T05:00:55.559 INFO:teuthology.orchestra.run.smithi057.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-13T05:00:55.593 INFO:teuthology.orchestra.run.smithi057.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 379 kB/s | 28 kB 00:00 2024-04-13T05:00:55.626 INFO:teuthology.orchestra.run.smithi057.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 463 kB/s | 31 kB 00:00 2024-04-13T05:00:55.693 INFO:teuthology.orchestra.run.smithi189.stdout:(99/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.0 MB/s | 5.1 MB 00:01 2024-04-13T05:00:55.718 INFO:teuthology.orchestra.run.smithi057.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 650 kB/s | 59 kB 00:00 2024-04-13T05:00:55.719 INFO:teuthology.orchestra.run.smithi189.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 315 kB/s | 58 kB 00:00 2024-04-13T05:00:55.743 INFO:teuthology.orchestra.run.smithi057.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-13T05:00:55.786 INFO:teuthology.orchestra.run.smithi057.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 542 kB/s | 36 kB 00:00 2024-04-13T05:00:55.811 INFO:teuthology.orchestra.run.smithi189.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 807 kB/s | 94 kB 00:00 2024-04-13T05:00:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 682 kB/s | 51 kB 00:00 2024-04-13T05:00:55.827 INFO:teuthology.orchestra.run.smithi189.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 590 kB/s | 64 kB 00:00 2024-04-13T05:00:55.910 INFO:teuthology.orchestra.run.smithi057.stdout:(144/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.8 MB/s | 2.0 MB 00:01 2024-04-13T05:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 420 kB/s | 45 kB 00:00 2024-04-13T05:00:56.017 INFO:teuthology.orchestra.run.smithi057.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 456 kB/s | 90 kB 00:00 2024-04-13T05:00:56.062 INFO:teuthology.orchestra.run.smithi057.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 833 kB/s | 230 kB 00:00 2024-04-13T05:00:56.096 INFO:teuthology.orchestra.run.smithi057.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 258 kB/s | 20 kB 00:00 2024-04-13T05:00:56.192 INFO:teuthology.orchestra.run.smithi057.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-13T05:00:56.276 INFO:teuthology.orchestra.run.smithi057.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-04-13T05:00:56.335 INFO:teuthology.orchestra.run.smithi057.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-04-13T05:00:56.482 INFO:teuthology.orchestra.run.smithi083.stdout:(70/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 1.5 MB/s | 4.1 MB 00:02 2024-04-13T05:00:56.528 INFO:teuthology.orchestra.run.smithi189.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-04-13T05:00:56.862 INFO:teuthology.orchestra.run.smithi189.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-13T05:00:56.962 INFO:teuthology.orchestra.run.smithi189.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-13T05:00:56.977 INFO:teuthology.orchestra.run.smithi057.stdout:(151/174): librbd1-19.0.0-2884.g218a6914.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-13T05:00:56.990 INFO:teuthology.orchestra.run.smithi083.stdout:(71/174): postgresql-private-libs-13.14-1.el9.x 263 kB/s | 133 kB 00:00 2024-04-13T05:00:57.021 INFO:teuthology.orchestra.run.smithi189.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 338 kB/s | 19 kB 00:00 2024-04-13T05:00:57.140 INFO:teuthology.orchestra.run.smithi083.stdout:(72/174): postgresql-contrib-13.14-1.el9.x86_64 564 kB/s | 911 kB 00:01 2024-04-13T05:00:57.338 INFO:teuthology.orchestra.run.smithi189.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-13T05:00:57.404 INFO:teuthology.orchestra.run.smithi189.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 396 kB/s | 26 kB 00:00 2024-04-13T05:00:57.469 INFO:teuthology.orchestra.run.smithi057.stdout:(152/174): librados2-19.0.0-2884.g218a6914.el9. 2.7 MB/s | 3.4 MB 00:01 2024-04-13T05:00:57.471 INFO:teuthology.orchestra.run.smithi189.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 738 kB/s | 49 kB 00:00 2024-04-13T05:00:57.547 INFO:teuthology.orchestra.run.smithi189.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 908 kB/s | 67 kB 00:00 2024-04-13T05:00:57.622 INFO:teuthology.orchestra.run.smithi189.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-13T05:00:57.747 INFO:teuthology.orchestra.run.smithi189.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-04-13T05:00:57.816 INFO:teuthology.orchestra.run.smithi083.stdout:(73/174): openblas-openmp-0.3.21-2.el9.x86_64.r 1.2 MB/s | 4.7 MB 00:04 2024-04-13T05:00:57.864 INFO:teuthology.orchestra.run.smithi189.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-13T05:00:57.923 INFO:teuthology.orchestra.run.smithi189.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 753 kB/s | 43 kB 00:00 2024-04-13T05:00:57.982 INFO:teuthology.orchestra.run.smithi189.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 557 kB/s | 32 kB 00:00 2024-04-13T05:00:58.024 INFO:teuthology.orchestra.run.smithi189.stdout:(117/174): protobuf-compiler-3.14.0-13.el9.x86_ 411 kB/s | 863 kB 00:02 2024-04-13T05:00:58.051 INFO:teuthology.orchestra.run.smithi189.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-04-13T05:00:58.085 INFO:teuthology.orchestra.run.smithi189.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-04-13T05:00:58.160 INFO:teuthology.orchestra.run.smithi189.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-13T05:00:58.236 INFO:teuthology.orchestra.run.smithi189.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 476 kB/s | 35 kB 00:00 2024-04-13T05:00:58.250 INFO:teuthology.orchestra.run.smithi083.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 202 kB/s | 87 kB 00:00 2024-04-13T05:00:58.252 INFO:teuthology.orchestra.run.smithi057.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 54 kB/s | 42 kB 00:00 2024-04-13T05:00:58.294 INFO:teuthology.orchestra.run.smithi189.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-13T05:00:58.328 INFO:teuthology.orchestra.run.smithi189.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 1.3 MB/s | 358 kB 00:00 2024-04-13T05:00:58.437 INFO:teuthology.orchestra.run.smithi189.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-13T05:00:58.470 INFO:teuthology.orchestra.run.smithi189.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-13T05:00:58.487 INFO:teuthology.orchestra.run.smithi189.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 217 kB/s | 11 kB 00:00 2024-04-13T05:00:58.521 INFO:teuthology.orchestra.run.smithi189.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 356 kB/s | 18 kB 00:00 2024-04-13T05:00:58.538 INFO:teuthology.orchestra.run.smithi189.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 466 kB/s | 23 kB 00:00 2024-04-13T05:00:58.571 INFO:teuthology.orchestra.run.smithi189.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2024-04-13T05:00:58.588 INFO:teuthology.orchestra.run.smithi189.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 397 kB/s | 20 kB 00:00 2024-04-13T05:00:58.647 INFO:teuthology.orchestra.run.smithi189.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 804 kB/s | 46 kB 00:00 2024-04-13T05:00:58.722 INFO:teuthology.orchestra.run.smithi189.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-04-13T05:00:58.733 INFO:teuthology.orchestra.run.smithi083.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 649 kB/s | 1.0 MB 00:01 2024-04-13T05:00:58.748 INFO:teuthology.orchestra.run.smithi189.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 792 kB/s | 79 kB 00:00 2024-04-13T05:00:58.781 INFO:teuthology.orchestra.run.smithi189.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-04-13T05:00:58.815 INFO:teuthology.orchestra.run.smithi189.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-13T05:00:58.874 INFO:teuthology.orchestra.run.smithi189.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-04-13T05:00:58.899 INFO:teuthology.orchestra.run.smithi189.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-13T05:00:58.941 INFO:teuthology.orchestra.run.smithi189.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 427 kB/s | 28 kB 00:00 2024-04-13T05:00:58.974 INFO:teuthology.orchestra.run.smithi189.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-04-13T05:00:59.016 INFO:teuthology.orchestra.run.smithi189.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-13T05:00:59.042 INFO:teuthology.orchestra.run.smithi189.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 899 kB/s | 59 kB 00:00 2024-04-13T05:00:59.084 INFO:teuthology.orchestra.run.smithi189.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 540 kB/s | 36 kB 00:00 2024-04-13T05:00:59.109 INFO:teuthology.orchestra.run.smithi189.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 770 kB/s | 51 kB 00:00 2024-04-13T05:00:59.150 INFO:teuthology.orchestra.run.smithi189.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-13T05:00:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-13T05:00:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 347 kB/s | 20 kB 00:00 2024-04-13T05:00:59.241 INFO:teuthology.orchestra.run.smithi083.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 484 kB/s | 245 kB 00:00 2024-04-13T05:00:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-13T05:00:59.286 INFO:teuthology.orchestra.run.smithi057.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 242 kB/s | 249 kB 00:01 2024-04-13T05:00:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-13T05:00:59.411 INFO:teuthology.orchestra.run.smithi189.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-13T05:00:59.750 INFO:teuthology.orchestra.run.smithi083.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 488 kB/s | 248 kB 00:00 2024-04-13T05:01:00.025 INFO:teuthology.orchestra.run.smithi083.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 93 kB/s | 26 kB 00:00 2024-04-13T05:01:00.053 INFO:teuthology.orchestra.run.smithi057.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 93 kB/s | 71 kB 00:00 2024-04-13T05:01:00.236 INFO:teuthology.orchestra.run.smithi189.stdout:(150/174): librbd1-19.0.0-2884.g218a6914.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-13T05:01:00.292 INFO:teuthology.orchestra.run.smithi083.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-04-13T05:01:00.353 INFO:teuthology.orchestra.run.smithi189.stdout:(151/174): librados2-19.0.0-2884.g218a6914.el9. 3.3 MB/s | 3.4 MB 00:01 2024-04-13T05:01:00.420 INFO:teuthology.orchestra.run.smithi057.stdout:(156/174): NetworkManager-1.47.4-1.el9.x86_64.r 589 kB/s | 2.3 MB 00:04 2024-04-13T05:01:00.469 INFO:teuthology.orchestra.run.smithi057.stdout:(157/174): NetworkManager-libnm-1.47.4-1.el9.x8 536 kB/s | 1.8 MB 00:03 2024-04-13T05:01:00.554 INFO:teuthology.orchestra.run.smithi189.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 7.3 MB/s | 2.3 MB 00:00 2024-04-13T05:01:00.604 INFO:teuthology.orchestra.run.smithi189.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 853 kB/s | 42 kB 00:00 2024-04-13T05:01:00.662 INFO:teuthology.orchestra.run.smithi189.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-04-13T05:01:00.713 INFO:teuthology.orchestra.run.smithi189.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-04-13T05:01:00.785 INFO:teuthology.orchestra.run.smithi189.stdout:(156/174): elfutils-debuginfod-client-0.190-2.e 513 kB/s | 36 kB 00:00 2024-04-13T05:01:00.786 INFO:teuthology.orchestra.run.smithi057.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 50 kB/s | 36 kB 00:00 2024-04-13T05:01:00.846 INFO:teuthology.orchestra.run.smithi189.stdout:(157/174): NetworkManager-libnm-1.47.4-1.el9.x8 3.7 MB/s | 1.8 MB 00:00 2024-04-13T05:01:00.880 INFO:teuthology.orchestra.run.smithi189.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-04-13T05:01:00.884 INFO:teuthology.orchestra.run.smithi083.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 291 kB/s | 172 kB 00:00 2024-04-13T05:01:00.922 INFO:teuthology.orchestra.run.smithi189.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.4 MB/s | 256 kB 00:00 2024-04-13T05:01:00.947 INFO:teuthology.orchestra.run.smithi189.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 349 kB/s | 23 kB 00:00 2024-04-13T05:01:00.973 INFO:teuthology.orchestra.run.smithi189.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-13T05:01:01.006 INFO:teuthology.orchestra.run.smithi189.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-04-13T05:01:01.031 INFO:teuthology.orchestra.run.smithi189.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-04-13T05:01:01.057 INFO:teuthology.orchestra.run.smithi189.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-04-13T05:01:01.082 INFO:teuthology.orchestra.run.smithi189.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 578 kB/s | 29 kB 00:00 2024-04-13T05:01:01.149 INFO:teuthology.orchestra.run.smithi189.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-04-13T05:01:01.159 INFO:teuthology.orchestra.run.smithi083.stdout:(81/174): python3-markupsafe-1.1.1-12.el9.x86_6 127 kB/s | 35 kB 00:00 2024-04-13T05:01:01.283 INFO:teuthology.orchestra.run.smithi189.stdout:(167/174): util-linux-2.37.4-18.el9.x86_64.rpm 17 MB/s | 2.3 MB 00:00 2024-04-13T05:01:01.345 INFO:teuthology.orchestra.run.smithi057.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 208 kB/s | 192 kB 00:00 2024-04-13T05:01:01.358 INFO:teuthology.orchestra.run.smithi189.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 6.2 MB/s | 465 kB 00:00 2024-04-13T05:01:01.410 INFO:teuthology.orchestra.run.smithi083.stdout:(82/174): postgresql-server-13.14-1.el9.x86_64. 1.3 MB/s | 5.8 MB 00:04 2024-04-13T05:01:01.433 INFO:teuthology.orchestra.run.smithi189.stdout:(169/174): util-linux-user-2.37.4-18.el9.x86_64 421 kB/s | 31 kB 00:00 2024-04-13T05:01:01.437 INFO:teuthology.orchestra.run.smithi057.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 35 kB/s | 23 kB 00:00 2024-04-13T05:01:01.502 INFO:teuthology.orchestra.run.smithi189.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.2 MB/s | 2.2 MB 00:00 2024-04-13T05:01:01.534 INFO:teuthology.orchestra.run.smithi189.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-04-13T05:01:01.537 INFO:teuthology.orchestra.run.smithi057.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 240 kB/s | 256 kB 00:01 2024-04-13T05:01:01.642 INFO:teuthology.orchestra.run.smithi189.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-04-13T05:01:01.993 INFO:teuthology.orchestra.run.smithi083.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 430 kB/s | 250 kB 00:00 2024-04-13T05:01:01.993 INFO:teuthology.orchestra.run.smithi189.stdout:(173/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-04-13T05:01:02.018 INFO:teuthology.orchestra.run.smithi189.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 154 kB/s | 74 kB 00:00 2024-04-13T05:01:02.024 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-04-13T05:01:02.025 INFO:teuthology.orchestra.run.smithi189.stdout:Total 5.8 MB/s | 229 MB 00:39 2024-04-13T05:01:02.221 INFO:teuthology.orchestra.run.smithi057.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 124 kB/s | 108 kB 00:00 2024-04-13T05:01:02.555 INFO:teuthology.orchestra.run.smithi057.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 139 kB/s | 155 kB 00:01 2024-04-13T05:01:02.580 INFO:teuthology.orchestra.run.smithi057.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 130 kB/s | 136 kB 00:01 2024-04-13T05:01:02.585 INFO:teuthology.orchestra.run.smithi083.stdout:(84/174): python3-babel-2.9.1-2.el9.noarch.rpm 1.4 MB/s | 6.0 MB 00:04 2024-04-13T05:01:02.619 INFO:teuthology.orchestra.run.smithi083.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 356 kB/s | 222 kB 00:00 2024-04-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi057.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 87 kB/s | 64 kB 00:00 2024-04-13T05:01:02.961 INFO:teuthology.orchestra.run.smithi083.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 124 kB/s | 42 kB 00:00 2024-04-13T05:01:03.011 INFO:teuthology.orchestra.run.smithi083.stdout:(87/174): python3-packaging-20.9-5.el9.noarch.r 182 kB/s | 77 kB 00:00 2024-04-13T05:01:03.223 INFO:teuthology.orchestra.run.smithi057.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 43 kB/s | 29 kB 00:00 2024-04-13T05:01:03.478 INFO:teuthology.orchestra.run.smithi083.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 522 kB/s | 269 kB 00:00 2024-04-13T05:01:03.528 INFO:teuthology.orchestra.run.smithi083.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 307 kB/s | 159 kB 00:00 2024-04-13T05:01:03.912 INFO:teuthology.orchestra.run.smithi083.stdout:(90/174): python3-pytz-2021.1-5.el9.noarch.rpm 133 kB/s | 51 kB 00:00 2024-04-13T05:01:04.345 INFO:teuthology.orchestra.run.smithi083.stdout:(91/174): python3-requests-oauthlib-1.3.0-12.el 124 kB/s | 54 kB 00:00 2024-04-13T05:01:04.379 INFO:teuthology.orchestra.run.smithi083.stdout:(92/174): python3-pyasn1-modules-0.4.8-6.el9.no 310 kB/s | 279 kB 00:00 2024-04-13T05:01:04.679 INFO:teuthology.orchestra.run.smithi083.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 139 kB/s | 42 kB 00:00 2024-04-13T05:01:04.956 INFO:teuthology.orchestra.run.smithi057.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 522 kB/s | 1.2 MB 00:02 2024-04-13T05:01:05.105 INFO:teuthology.orchestra.run.smithi083.stdout:(94/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.3 MB/s | 5.1 MB 00:03 2024-04-13T05:01:05.130 INFO:teuthology.orchestra.run.smithi083.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 98 kB/s | 44 kB 00:00 2024-04-13T05:01:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:(96/174): qatlib-service-24.02.0-1.el9.x86_64.r 131 kB/s | 36 kB 00:00 2024-04-13T05:01:05.705 INFO:teuthology.orchestra.run.smithi083.stdout:(97/174): qatlib-24.02.0-1.el9.x86_64.rpm 368 kB/s | 221 kB 00:00 2024-04-13T05:01:05.728 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-04-13T05:01:05.739 INFO:teuthology.orchestra.run.smithi083.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 142 kB/s | 47 kB 00:00 2024-04-13T05:01:06.089 INFO:teuthology.orchestra.run.smithi083.stdout:(99/174): uuid-1.6.2-55.el9.x86_64.rpm 165 kB/s | 58 kB 00:00 2024-04-13T05:01:06.119 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-04-13T05:01:06.119 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-04-13T05:01:06.232 INFO:teuthology.orchestra.run.smithi057.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 365 kB/s | 465 kB 00:01 2024-04-13T05:01:06.248 INFO:teuthology.orchestra.run.smithi083.stdout:(100/174): socat-1.7.4.1-5.el9.x86_64.rpm 563 kB/s | 305 kB 00:00 2024-04-13T05:01:06.531 INFO:teuthology.orchestra.run.smithi083.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 212 kB/s | 94 kB 00:00 2024-04-13T05:01:06.607 INFO:teuthology.orchestra.run.smithi083.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 178 kB/s | 64 kB 00:00 2024-04-13T05:01:06.607 INFO:teuthology.orchestra.run.smithi057.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 605 kB/s | 2.2 MB 00:03 2024-04-13T05:01:06.690 INFO:teuthology.orchestra.run.smithi083.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-13T05:01:06.865 INFO:teuthology.orchestra.run.smithi083.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-13T05:01:06.891 INFO:teuthology.orchestra.run.smithi083.stdout:(105/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 126 kB/s | 45 kB 00:00 2024-04-13T05:01:06.923 INFO:teuthology.orchestra.run.smithi057.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 45 kB/s | 31 kB 00:00 2024-04-13T05:01:06.991 INFO:teuthology.orchestra.run.smithi057.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 633 kB/s | 2.3 MB 00:03 2024-04-13T05:01:07.299 INFO:teuthology.orchestra.run.smithi057.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 244 kB/s | 74 kB 00:00 2024-04-13T05:01:07.708 INFO:teuthology.orchestra.run.smithi083.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 651 kB/s | 548 kB 00:00 2024-04-13T05:01:07.741 INFO:teuthology.orchestra.run.smithi083.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 364 kB/s | 309 kB 00:00 2024-04-13T05:01:07.791 INFO:teuthology.orchestra.run.smithi057.stdout:(173/174): xfsdump-3.1.12-4.el9.x86_64.rpm 273 kB/s | 323 kB 00:01 2024-04-13T05:01:07.817 INFO:teuthology.orchestra.run.smithi083.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 181 kB/s | 19 kB 00:00 2024-04-13T05:01:07.925 INFO:teuthology.orchestra.run.smithi083.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-13T05:01:08.050 INFO:teuthology.orchestra.run.smithi083.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 391 kB/s | 49 kB 00:00 2024-04-13T05:01:08.192 INFO:teuthology.orchestra.run.smithi083.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 478 kB/s | 67 kB 00:00 2024-04-13T05:01:08.368 INFO:teuthology.orchestra.run.smithi083.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-13T05:01:08.401 INFO:teuthology.orchestra.run.smithi083.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 728 kB/s | 151 kB 00:00 2024-04-13T05:01:08.502 INFO:teuthology.orchestra.run.smithi083.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 838 kB 00:00 2024-04-13T05:01:08.577 INFO:teuthology.orchestra.run.smithi083.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-04-13T05:01:08.611 INFO:teuthology.orchestra.run.smithi083.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-04-13T05:01:08.683 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-04-13T05:01:08.684 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-04-13T05:01:08.694 INFO:teuthology.orchestra.run.smithi083.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-04-13T05:01:08.719 INFO:teuthology.orchestra.run.smithi083.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-04-13T05:01:08.828 INFO:teuthology.orchestra.run.smithi083.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-13T05:01:08.890 INFO:teuthology.orchestra.run.smithi083.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-13T05:01:08.977 INFO:teuthology.orchestra.run.smithi083.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-13T05:01:09.003 INFO:teuthology.orchestra.run.smithi083.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 316 kB/s | 35 kB 00:00 2024-04-13T05:01:09.103 INFO:teuthology.orchestra.run.smithi083.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-04-13T05:01:09.166 INFO:teuthology.orchestra.run.smithi057.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 490 kB/s | 1.1 MB 00:02 2024-04-13T05:01:09.175 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-04-13T05:01:09.175 INFO:teuthology.orchestra.run.smithi057.stdout:Total 4.7 MB/s | 229 MB 00:48 2024-04-13T05:01:09.195 INFO:teuthology.orchestra.run.smithi083.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-13T05:01:09.220 INFO:teuthology.orchestra.run.smithi083.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-13T05:01:09.312 INFO:teuthology.orchestra.run.smithi083.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-13T05:01:09.338 INFO:teuthology.orchestra.run.smithi083.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-13T05:01:09.421 INFO:teuthology.orchestra.run.smithi083.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-04-13T05:01:09.447 INFO:teuthology.orchestra.run.smithi083.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-04-13T05:01:09.530 INFO:teuthology.orchestra.run.smithi083.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-13T05:01:09.639 INFO:teuthology.orchestra.run.smithi083.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 431 kB/s | 46 kB 00:00 2024-04-13T05:01:09.681 INFO:teuthology.orchestra.run.smithi083.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-04-13T05:01:09.748 INFO:teuthology.orchestra.run.smithi083.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 731 kB/s | 79 kB 00:00 2024-04-13T05:01:09.790 INFO:teuthology.orchestra.run.smithi083.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 535 kB/s | 58 kB 00:00 2024-04-13T05:01:09.873 INFO:teuthology.orchestra.run.smithi083.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-13T05:01:09.932 INFO:teuthology.orchestra.run.smithi083.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-04-13T05:01:09.991 INFO:teuthology.orchestra.run.smithi083.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 774 kB/s | 90 kB 00:00 2024-04-13T05:01:10.049 INFO:teuthology.orchestra.run.smithi083.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 243 kB/s | 28 kB 00:00 2024-04-13T05:01:10.108 INFO:teuthology.orchestra.run.smithi083.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-04-13T05:01:10.167 INFO:teuthology.orchestra.run.smithi083.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-13T05:01:10.217 INFO:teuthology.orchestra.run.smithi083.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 548 kB/s | 59 kB 00:00 2024-04-13T05:01:10.434 INFO:teuthology.orchestra.run.smithi083.stdout:(142/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.7 MB/s | 16 MB 00:06 2024-04-13T05:01:10.451 INFO:teuthology.orchestra.run.smithi083.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 126 kB/s | 36 kB 00:00 2024-04-13T05:01:10.468 INFO:teuthology.orchestra.run.smithi083.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 204 kB/s | 51 kB 00:00 2024-04-13T05:01:10.610 INFO:teuthology.orchestra.run.smithi083.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 568 kB/s | 90 kB 00:00 2024-04-13T05:01:10.644 INFO:teuthology.orchestra.run.smithi083.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-13T05:01:10.719 INFO:teuthology.orchestra.run.smithi083.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-13T05:01:10.752 INFO:teuthology.orchestra.run.smithi083.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-13T05:01:10.903 INFO:teuthology.orchestra.run.smithi083.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-13T05:01:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:(150/174): python3-webob-1.8.7-6.el9.noarch.rpm 296 kB/s | 230 kB 00:00 2024-04-13T05:01:12.345 INFO:teuthology.orchestra.run.smithi083.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2024-04-13T05:01:12.562 INFO:teuthology.orchestra.run.smithi083.stdout:(152/174): librbd1-19.0.0-2884.g218a6914.el9.x8 1.9 MB/s | 3.1 MB 00:01 2024-04-13T05:01:12.629 INFO:teuthology.orchestra.run.smithi083.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 6.5 MB/s | 1.8 MB 00:00 2024-04-13T05:01:12.754 INFO:teuthology.orchestra.run.smithi083.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 2.0 MB/s | 249 kB 00:00 2024-04-13T05:01:12.871 INFO:teuthology.orchestra.run.smithi083.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 616 kB/s | 71 kB 00:00 2024-04-13T05:01:12.955 INFO:teuthology.orchestra.run.smithi083.stdout:(156/174): NetworkManager-team-1.47.4-1.el9.x86 107 kB/s | 42 kB 00:00 2024-04-13T05:01:13.005 INFO:teuthology.orchestra.run.smithi083.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 273 kB/s | 36 kB 00:00 2024-04-13T05:01:13.012 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-04-13T05:01:13.164 INFO:teuthology.orchestra.run.smithi083.stdout:(158/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.6 MB/s | 256 kB 00:00 2024-04-13T05:01:13.280 INFO:teuthology.orchestra.run.smithi083.stdout:(159/174): libacl-2.3.1-4.el9.x86_64.rpm 199 kB/s | 23 kB 00:00 2024-04-13T05:01:13.314 INFO:teuthology.orchestra.run.smithi083.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 537 kB/s | 192 kB 00:00 2024-04-13T05:01:13.390 INFO:teuthology.orchestra.run.smithi083.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 108 kB 00:00 2024-04-13T05:01:13.420 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-04-13T05:01:13.420 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-04-13T05:01:13.506 INFO:teuthology.orchestra.run.smithi083.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 809 kB/s | 155 kB 00:00 2024-04-13T05:01:13.532 INFO:teuthology.orchestra.run.smithi083.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 998 kB/s | 136 kB 00:00 2024-04-13T05:01:13.615 INFO:teuthology.orchestra.run.smithi083.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 593 kB/s | 64 kB 00:00 2024-04-13T05:01:13.699 INFO:teuthology.orchestra.run.smithi083.stdout:(165/174): librados2-19.0.0-2884.g218a6914.el9. 1.2 MB/s | 3.4 MB 00:02 2024-04-13T05:01:13.724 INFO:teuthology.orchestra.run.smithi083.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 149 kB/s | 29 kB 00:00 2024-04-13T05:01:14.013 INFO:teuthology.orchestra.run.smithi083.stdout:(167/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.0 MB/s | 2.3 MB 00:00 2024-04-13T05:01:14.067 INFO:teuthology.orchestra.run.smithi083.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 2.7 MB/s | 1.2 MB 00:00 2024-04-13T05:01:14.175 INFO:teuthology.orchestra.run.smithi083.stdout:(169/174): util-linux-user-2.37.4-18.el9.x86_64 289 kB/s | 31 kB 00:00 2024-04-13T05:01:14.301 INFO:teuthology.orchestra.run.smithi083.stdout:(170/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-04-13T05:01:14.461 INFO:teuthology.orchestra.run.smithi083.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-04-13T05:01:14.786 INFO:teuthology.orchestra.run.smithi083.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 602 kB/s | 465 kB 00:00 2024-04-13T05:01:14.803 INFO:teuthology.orchestra.run.smithi083.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 218 kB/s | 74 kB 00:00 2024-04-13T05:01:14.853 INFO:teuthology.orchestra.run.smithi083.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 1.9 MB/s | 2.2 MB 00:01 2024-04-13T05:01:14.861 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-13T05:01:14.862 INFO:teuthology.orchestra.run.smithi083.stdout:Total 4.3 MB/s | 229 MB 00:52 2024-04-13T05:01:16.099 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-04-13T05:01:16.099 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-04-13T05:01:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-04-13T05:01:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-13T05:01:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-13T05:01:16.329 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T05:01:16.374 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T05:01:16.465 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-13T05:01:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-13T05:01:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-13T05:01:16.652 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-13T05:01:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-13T05:01:17.181 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-13T05:01:17.233 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:01:17.395 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:01:17.445 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:01:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:01:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T05:01:17.663 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-13T05:01:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-13T05:01:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-13T05:01:18.059 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:01:18.200 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:01:18.225 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T05:01:18.299 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T05:01:18.352 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-13T05:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T05:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T05:01:18.572 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-13T05:01:18.743 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T05:01:18.795 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T05:01:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-13T05:01:18.966 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-13T05:01:18.966 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-13T05:01:19.122 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-13T05:01:19.285 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T05:01:19.389 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T05:01:19.556 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:19.613 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:19.960 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:20.147 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-13T05:01:20.264 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-13T05:01:20.302 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-13T05:01:20.350 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-13T05:01:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-13T05:01:20.488 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-13T05:01:20.549 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-13T05:01:20.597 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-13T05:01:20.641 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-13T05:01:20.741 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-13T05:01:20.803 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-13T05:01:20.835 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-13T05:01:20.890 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-13T05:01:20.998 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-13T05:01:21.039 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 39/198 2024-04-13T05:01:21.087 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 40/198 2024-04-13T05:01:21.120 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 41/198 2024-04-13T05:01:21.215 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-13T05:01:21.289 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-13T05:01:21.377 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-13T05:01:21.489 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-13T05:01:21.490 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-13T05:01:21.554 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-13T05:01:21.619 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-13T05:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-13T05:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-13T05:01:22.526 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-13T05:01:22.608 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T05:01:22.637 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T05:01:22.702 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-13T05:01:22.769 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-13T05:01:22.825 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 53/198 2024-04-13T05:01:22.875 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-13T05:01:23.285 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-13T05:01:23.511 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-13T05:01:23.612 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-13T05:01:23.667 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-13T05:01:23.799 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-04-13T05:01:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-13T05:01:23.915 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-13T05:01:23.925 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-13T05:01:23.964 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-13T05:01:23.997 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-13T05:01:24.010 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T05:01:24.056 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T05:01:24.080 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-13T05:01:24.115 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-13T05:01:24.153 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-13T05:01:24.153 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-13T05:01:24.217 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-13T05:01:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-13T05:01:24.274 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-13T05:01:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-13T05:01:24.331 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-13T05:01:24.377 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-13T05:01:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-13T05:01:24.469 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-13T05:01:24.583 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-13T05:01:24.635 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-13T05:01:24.673 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-13T05:01:24.731 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-13T05:01:24.873 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-13T05:01:24.885 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-13T05:01:24.922 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-13T05:01:24.925 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:01:24.967 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-13T05:01:25.004 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-13T05:01:25.046 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-13T05:01:25.086 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:01:25.118 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:01:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-13T05:01:25.245 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:01:25.278 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T05:01:25.349 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-13T05:01:25.419 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-13T05:01:25.460 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-13T05:01:25.497 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-13T05:01:25.534 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-13T05:01:25.587 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-13T05:01:25.635 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-13T05:01:25.670 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-13T05:01:25.675 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-13T05:01:25.707 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-13T05:01:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-13T05:01:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:01:25.773 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-13T05:01:25.841 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-13T05:01:25.884 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-13T05:01:25.930 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:01:25.979 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T05:01:26.027 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-13T05:01:26.082 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T05:01:26.088 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-13T05:01:26.128 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-13T05:01:26.130 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-13T05:01:26.159 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T05:01:26.192 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-13T05:01:26.222 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T05:01:26.339 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-13T05:01:26.429 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-13T05:01:26.535 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T05:01:26.585 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T05:01:26.653 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-13T05:01:27.400 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-13T05:01:27.424 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-13T05:01:27.477 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-13T05:01:27.510 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-13T05:01:27.633 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T05:01:27.634 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-13T05:01:27.713 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-13T05:01:27.722 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T05:01:27.811 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-13T05:01:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-13T05:01:27.930 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:28.010 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:28.079 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-13T05:01:28.109 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-13T05:01:28.195 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-13T05:01:28.341 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:28.655 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-13T05:01:28.791 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-13T05:01:28.825 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-13T05:01:28.856 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-13T05:01:28.865 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-13T05:01:28.960 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-13T05:01:28.964 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-13T05:01:29.007 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-13T05:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-13T05:01:29.012 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-13T05:01:29.055 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T05:01:29.086 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-13T05:01:29.100 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-13T05:01:29.135 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-13T05:01:29.146 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-13T05:01:29.196 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-13T05:01:29.197 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-13T05:01:29.294 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-13T05:01:29.349 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-13T05:01:29.352 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-13T05:01:29.389 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-13T05:01:29.431 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-13T05:01:29.442 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-13T05:01:29.468 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-13T05:01:29.506 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-13T05:01:29.653 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-13T05:01:29.653 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 39/198 2024-04-13T05:01:29.694 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 40/198 2024-04-13T05:01:29.727 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 41/198 2024-04-13T05:01:29.813 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-13T05:01:29.848 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-13T05:01:29.854 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:01:29.916 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-13T05:01:30.012 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:01:30.029 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-13T05:01:30.047 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:01:30.177 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:01:30.219 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-13T05:01:30.220 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T05:01:30.285 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-13T05:01:30.294 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-13T05:01:30.585 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-13T05:01:30.644 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-13T05:01:30.669 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:01:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-13T05:01:30.814 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-13T05:01:30.819 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:01:30.845 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T05:01:30.869 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-13T05:01:30.938 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 17/198 2024-04-13T05:01:30.989 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-13T05:01:31.007 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-13T05:01:31.014 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T05:01:31.077 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-13T05:01:31.108 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-13T05:01:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-13T05:01:31.245 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-13T05:01:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T05:01:31.314 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T05:01:31.346 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T05:01:31.361 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T05:01:31.381 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-13T05:01:31.388 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T05:01:31.406 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-13T05:01:31.415 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 113/198 2024-04-13T05:01:31.454 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-13T05:01:31.465 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-13T05:01:31.473 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-13T05:01:31.507 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 53/198 2024-04-13T05:01:31.556 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-13T05:01:31.784 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-13T05:01:31.955 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T05:01:31.977 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-13T05:01:32.021 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T05:01:32.034 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-13T05:01:32.053 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T05:01:32.210 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-13T05:01:32.211 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T05:01:32.238 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 117/198 2024-04-13T05:01:32.248 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:32.294 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-13T05:01:32.307 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:32.370 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-13T05:01:32.596 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-13T05:01:32.648 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-13T05:01:32.681 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-13T05:01:32.719 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-13T05:01:32.810 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-13T05:01:32.832 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-13T05:01:32.855 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-13T05:01:32.884 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-13T05:01:32.944 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-13T05:01:32.967 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-13T05:01:32.986 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-13T05:01:33.006 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-13T05:01:33.053 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-13T05:01:33.064 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-13T05:01:33.127 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-13T05:01:33.144 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-13T05:01:33.201 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-13T05:01:33.272 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-13T05:01:33.323 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-13T05:01:33.332 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-13T05:01:33.362 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-13T05:01:33.383 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-13T05:01:33.421 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-13T05:01:33.471 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-13T05:01:33.480 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-13T05:01:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-13T05:01:33.589 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-13T05:01:33.591 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-13T05:01:33.628 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-13T05:01:33.646 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-13T05:01:33.689 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-13T05:01:33.725 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-13T05:01:33.754 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-13T05:01:33.758 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-13T05:01:33.802 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 39/198 2024-04-13T05:01:33.853 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 40/198 2024-04-13T05:01:33.883 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 41/198 2024-04-13T05:01:33.938 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-13T05:01:33.975 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-13T05:01:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-13T05:01:34.116 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-13T05:01:34.138 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-13T05:01:34.156 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-13T05:01:34.185 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-13T05:01:34.211 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-13T05:01:34.261 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-13T05:01:34.318 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-13T05:01:34.330 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-13T05:01:34.377 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-13T05:01:34.391 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-13T05:01:34.414 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-13T05:01:34.454 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-13T05:01:34.519 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-13T05:01:34.561 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-13T05:01:34.708 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-13T05:01:34.769 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-13T05:01:34.810 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-13T05:01:34.874 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-13T05:01:35.032 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-13T05:01:35.059 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-13T05:01:35.069 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:35.100 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-13T05:01:35.111 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:35.162 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-13T05:01:35.295 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-13T05:01:35.372 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T05:01:35.401 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-13T05:01:35.455 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-13T05:01:35.542 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-13T05:01:35.600 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 53/198 2024-04-13T05:01:35.648 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-13T05:01:35.740 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:35.781 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T05:01:35.994 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T05:01:35.994 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-13T05:01:35.994 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-13T05:01:35.994 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:36.040 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T05:01:36.052 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-13T05:01:36.115 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-13T05:01:36.173 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-13T05:01:36.232 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-13T05:01:36.275 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-13T05:01:36.359 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-13T05:01:36.379 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-13T05:01:36.424 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-13T05:01:36.445 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-13T05:01:36.560 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-13T05:01:36.600 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-13T05:01:36.610 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-13T05:01:36.707 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-13T05:01:36.782 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-13T05:01:36.851 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-13T05:01:36.878 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-13T05:01:36.888 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-13T05:01:36.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-13T05:01:36.961 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-13T05:01:36.997 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-13T05:01:37.022 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-13T05:01:37.060 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-13T05:01:37.113 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-13T05:01:37.171 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-13T05:01:37.375 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-13T05:01:37.427 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-13T05:01:37.464 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-13T05:01:37.514 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-13T05:01:37.671 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-13T05:01:37.713 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-13T05:01:37.760 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-13T05:01:37.796 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-13T05:01:37.828 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-13T05:01:37.869 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-13T05:01:38.001 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-13T05:01:38.007 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-13T05:01:38.179 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-13T05:01:38.219 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-13T05:01:38.256 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-13T05:01:38.298 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-13T05:01:38.344 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-13T05:01:38.389 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-13T05:01:38.440 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-13T05:01:38.485 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-13T05:01:38.529 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-13T05:01:38.595 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-13T05:01:38.645 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-13T05:01:38.788 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-13T05:01:38.851 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-13T05:01:38.893 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-13T05:01:38.956 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-13T05:01:39.109 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-13T05:01:39.174 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-13T05:01:39.646 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-13T05:01:39.744 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-13T05:01:39.781 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-13T05:01:40.164 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-13T05:01:40.205 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T05:01:40.215 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-13T05:01:40.265 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-13T05:01:40.298 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-13T05:01:40.336 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T05:01:40.361 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 113/198 2024-04-13T05:01:40.413 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-13T05:01:40.421 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-13T05:01:40.477 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-13T05:01:40.572 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-13T05:01:40.636 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-13T05:01:40.873 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-13T05:01:40.905 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-13T05:01:40.977 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T05:01:41.007 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T05:01:41.014 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-13T05:01:41.107 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T05:01:41.134 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 117/198 2024-04-13T05:01:41.825 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-13T05:01:41.959 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-13T05:01:43.510 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-13T05:01:43.596 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-13T05:01:43.630 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-13T05:01:43.800 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:43.839 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-13T05:01:44.025 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T05:01:44.130 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 112/198 2024-04-13T05:01:44.164 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 113/198 2024-04-13T05:01:44.221 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-13T05:01:44.502 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:44.567 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T05:01:44.742 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T05:01:44.742 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-13T05:01:44.742 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-13T05:01:44.742 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:01:44.785 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T05:01:44.789 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T05:01:44.828 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T05:01:44.944 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 116/198 2024-04-13T05:01:44.971 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 117/198 2024-04-13T05:01:47.616 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:47.651 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:48.280 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 118/198 2024-04-13T05:01:48.330 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T05:01:48.494 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 119/198 2024-04-13T05:01:48.494 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-13T05:01:48.495 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-13T05:01:48.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:01:48.539 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T05:01:48.681 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-04-13T05:01:48.682 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:48.909 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T05:01:48.956 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T05:01:48.956 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:01:48.956 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-13T05:01:48.956 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:50.562 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T05:01:50.644 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T05:01:50.645 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:01:50.645 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-13T05:01:50.645 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T05:01:50.645 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T05:01:50.645 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:50.895 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T05:01:50.967 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T05:01:50.968 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:01:50.968 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-13T05:01:50.968 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T05:01:50.968 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T05:01:50.968 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T05:01:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T05:01:51.565 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:01:51.565 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-13T05:01:51.565 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T05:01:51.566 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T05:01:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-13T05:01:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-13T05:01:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-13T05:01:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-13T05:01:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-13T05:01:52.244 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-13T05:01:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-13T05:01:52.520 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-13T05:01:52.566 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-13T05:01:52.608 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-13T05:01:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-13T05:01:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-13T05:01:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-13T05:01:53.389 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T05:01:53.429 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T05:01:53.511 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 139/198 2024-04-13T05:01:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 140/198 2024-04-13T05:01:53.639 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T05:01:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T05:01:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T05:01:56.085 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T05:01:56.115 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T05:01:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T05:01:56.438 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 144/198 2024-04-13T05:01:56.475 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T05:01:56.543 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T05:01:56.543 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:01:56.543 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-13T05:01:56.543 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T05:01:56.543 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T05:01:56.543 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:01:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T05:01:56.638 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T05:01:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 147/198 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /root 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2024-04-13T05:01:58.191 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:01:58.412 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T05:01:58.462 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T05:01:58.463 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:01:58.463 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-13T05:01:58.463 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:00.147 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T05:02:00.229 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T05:02:00.229 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:00.229 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-13T05:02:00.229 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T05:02:00.229 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T05:02:00.229 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:00.489 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T05:02:00.592 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T05:02:00.592 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:00.592 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-13T05:02:00.592 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T05:02:00.592 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T05:02:00.592 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:01.073 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T05:02:01.149 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T05:02:01.149 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:01.149 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-13T05:02:01.149 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T05:02:01.149 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T05:02:01.149 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:01.292 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-13T05:02:01.366 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-13T05:02:01.451 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-13T05:02:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 120/198 2024-04-13T05:02:01.496 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-13T05:02:01.496 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-13T05:02:01.496 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-13T05:02:01.496 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-13T05:02:01.496 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-13T05:02:01.497 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-13T05:02:01.497 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-13T05:02:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:01.710 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-13T05:02:01.720 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T05:02:01.744 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 148/198 2024-04-13T05:02:01.761 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-13T05:02:01.769 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 121/198 2024-04-13T05:02:01.769 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:01.770 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-13T05:02:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:01.852 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-13T05:02:01.855 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 149/198 2024-04-13T05:02:01.969 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-13T05:02:02.159 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-13T05:02:02.234 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-13T05:02:02.277 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-13T05:02:02.318 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-13T05:02:02.366 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-13T05:02:02.402 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-13T05:02:02.801 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-13T05:02:02.841 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:02.919 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:02.919 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:02.919 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-13T05:02:02.920 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T05:02:02.920 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T05:02:02.920 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T05:02:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T05:02:03.035 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:03.035 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-13T05:02:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:03.086 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T05:02:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:03.122 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T05:02:03.182 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 139/198 2024-04-13T05:02:03.184 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:03.184 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:03.184 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-13T05:02:03.184 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T05:02:03.184 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T05:02:03.184 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:03.292 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 140/198 2024-04-13T05:02:03.323 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T05:02:03.432 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T05:02:03.510 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 122/198 2024-04-13T05:02:03.511 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:03.511 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-13T05:02:03.511 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T05:02:03.511 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-13T05:02:03.511 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T05:02:03.610 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T05:02:03.610 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:03.610 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-13T05:02:03.610 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T05:02:03.610 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T05:02:03.610 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:03.669 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 155/198 2024-04-13T05:02:03.717 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 156/198 2024-04-13T05:02:03.755 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-13T05:02:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T05:02:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 123/198 2024-04-13T05:02:03.825 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:03.825 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-13T05:02:03.825 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T05:02:03.825 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-13T05:02:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:04.041 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 158/198 2024-04-13T05:02:04.115 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-13T05:02:04.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-13T05:02:04.235 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-13T05:02:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-13T05:02:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T05:02:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 163/198 2024-04-13T05:02:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 124/198 2024-04-13T05:02:04.377 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:04.377 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-13T05:02:04.377 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T05:02:04.377 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-13T05:02:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-13T05:02:04.447 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-13T05:02:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-13T05:02:04.533 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-13T05:02:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-13T05:02:04.637 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-13T05:02:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-13T05:02:04.678 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-13T05:02:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T05:02:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T05:02:04.890 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-13T05:02:04.979 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-13T05:02:05.349 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-13T05:02:05.386 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-13T05:02:05.435 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-13T05:02:05.466 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-13T05:02:05.505 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-13T05:02:05.536 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-13T05:02:05.899 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:05.903 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T05:02:05.933 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T05:02:05.951 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:06.084 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T05:02:06.214 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T05:02:06.248 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 138/198 2024-04-13T05:02:06.272 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 144/198 2024-04-13T05:02:06.301 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T05:02:06.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 139/198 2024-04-13T05:02:06.377 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 140/198 2024-04-13T05:02:06.385 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T05:02:06.385 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:06.385 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-13T05:02:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T05:02:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T05:02:06.386 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:06.408 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T05:02:06.444 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T05:02:06.485 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T05:02:06.598 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 147/198 2024-04-13T05:02:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 170/198 2024-04-13T05:02:07.519 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-13T05:02:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-13T05:02:07.682 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-13T05:02:07.691 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 141/198 2024-04-13T05:02:07.707 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-13T05:02:07.733 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-13T05:02:07.735 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T05:02:07.775 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-13T05:02:07.818 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-13T05:02:07.818 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T05:02:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T05:02:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-13T05:02:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T05:02:08.143 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T05:02:08.196 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-13T05:02:08.242 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-13T05:02:08.291 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-13T05:02:08.324 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-13T05:02:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-13T05:02:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-13T05:02:08.504 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-13T05:02:08.535 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-13T05:02:08.578 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-13T05:02:08.621 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-13T05:02:08.654 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:08.838 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 142/198 2024-04-13T05:02:08.868 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T05:02:09.015 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 143/198 2024-04-13T05:02:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 144/198 2024-04-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 145/198 2024-04-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:09.319 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-13T05:02:09.363 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T05:02:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-13T05:02:09.399 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 146/198 2024-04-13T05:02:09.486 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T05:02:09.505 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 147/198 2024-04-13T05:02:09.727 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T05:02:09.727 INFO:teuthology.orchestra.run.smithi189.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-13T05:02:09.727 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:11.735 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 148/198 2024-04-13T05:02:11.843 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 149/198 2024-04-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-13T05:02:12.892 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:12.968 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:12.968 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:12.968 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-13T05:02:12.968 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T05:02:12.968 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T05:02:12.968 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:13.018 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T05:02:13.098 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T05:02:13.099 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:13.099 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-13T05:02:13.099 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:13.169 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:13.262 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:13.262 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:13.262 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-13T05:02:13.262 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T05:02:13.262 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T05:02:13.262 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:13.644 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T05:02:13.722 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T05:02:13.722 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:13.722 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-13T05:02:13.722 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T05:02:13.722 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T05:02:13.722 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:13.794 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 155/198 2024-04-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 156/198 2024-04-13T05:02:13.866 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-13T05:02:14.172 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 158/198 2024-04-13T05:02:14.242 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-13T05:02:14.303 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-13T05:02:14.394 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 148/198 2024-04-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 163/198 2024-04-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 149/198 2024-04-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-13T05:02:14.574 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-13T05:02:14.672 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-13T05:02:14.790 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-13T05:02:14.830 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-13T05:02:15.164 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 1/198 2024-04-13T05:02:15.164 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2/198 2024-04-13T05:02:15.164 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 3/198 2024-04-13T05:02:15.164 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 4/198 2024-04-13T05:02:15.164 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 5/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 6/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 7/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 8/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 9/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 13/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 14/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 15/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 17/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 18/198 2024-04-13T05:02:15.165 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 19/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 20/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 21/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 22/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 23/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 24/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 25/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 26/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 27/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 28/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 29/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 30/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 31/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 32/198 2024-04-13T05:02:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 33/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 34/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 35/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 36/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 37/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 38/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 39/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-13T05:02:15.167 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-13T05:02:15.168 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-13T05:02:15.169 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-13T05:02:15.170 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-13T05:02:15.171 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-13T05:02:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-13T05:02:15.173 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-13T05:02:15.174 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-13T05:02:15.175 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-13T05:02:15.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-13T05:02:15.177 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-13T05:02:15.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-13T05:02:15.458 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-13T05:02:15.498 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:15.567 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:15.567 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:15.567 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-13T05:02:15.567 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T05:02:15.567 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-13T05:02:15.567 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:15.615 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T05:02:15.707 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2884.g218a6 152/198 2024-04-13T05:02:15.707 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:15.707 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-13T05:02:15.707 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:15.781 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:15.782 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:15.826 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:15.857 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:15.857 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:15.857 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-13T05:02:15.857 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T05:02:15.857 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-13T05:02:15.857 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:15.885 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:16.242 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T05:02:16.313 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 154/198 2024-04-13T05:02:16.313 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-13T05:02:16.313 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-13T05:02:16.314 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T05:02:16.314 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-13T05:02:16.314 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:16.372 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 155/198 2024-04-13T05:02:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 156/198 2024-04-13T05:02:16.447 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-13T05:02:16.729 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 158/198 2024-04-13T05:02:16.795 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-13T05:02:16.855 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-13T05:02:16.917 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-13T05:02:16.970 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-13T05:02:17.005 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 163/198 2024-04-13T05:02:17.050 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-13T05:02:17.103 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-13T05:02:17.194 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-13T05:02:17.312 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 170/198 2024-04-13T05:02:17.337 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-13T05:02:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-13T05:02:17.429 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-13T05:02:17.559 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-13T05:02:17.675 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-13T05:02:17.709 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-13T05:02:17.752 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-13T05:02:17.752 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:17.775 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: acl-2.3.1-4.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.776 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.777 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.778 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-13T05:02:17.779 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: jq-1.6-16.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-13T05:02:17.780 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-13T05:02:17.781 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.14-1.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-13T05:02:17.782 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-13T05:02:17.783 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-13T05:02:17.784 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-13T05:02:17.785 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-13T05:02:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-13T05:02:17.787 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-3.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-13T05:02:17.788 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-13T05:02:17.789 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-13T05:02:17.789 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:02:17.789 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-04-13T05:02:17.888 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T05:02:17.931 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-13T05:02:17.931 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T05:02:18.068 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T05:02:18.112 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-13T05:02:18.274 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:18.315 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:18.328 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-13T05:02:18.371 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-13T05:02:18.414 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-13T05:02:18.462 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-13T05:02:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-13T05:02:18.501 DEBUG:teuthology.parallel:result is None 2024-04-13T05:02:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-13T05:02:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-13T05:02:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T05:02:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T05:02:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-13T05:02:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-13T05:02:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T05:02:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T05:02:19.696 INFO:teuthology.orchestra.run.smithi057.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-13T05:02:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:19.727 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 170/198 2024-04-13T05:02:19.850 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-13T05:02:19.959 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-13T05:02:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-13T05:02:20.022 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-13T05:02:20.047 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-13T05:02:20.082 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-13T05:02:20.124 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-13T05:02:20.124 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T05:02:20.270 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-13T05:02:20.337 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-13T05:02:20.337 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T05:02:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-13T05:02:20.527 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-13T05:02:20.574 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-13T05:02:20.615 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-13T05:02:20.647 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-13T05:02:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-13T05:02:20.796 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-13T05:02:20.844 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-13T05:02:20.877 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-13T05:02:20.911 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-13T05:02:20.945 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-13T05:02:20.978 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-13T05:02:21.014 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:21.014 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-13T05:02:21.370 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T05:02:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-13T05:02:21.539 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-13T05:02:21.593 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-13T05:02:21.626 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-13T05:02:21.674 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-13T05:02:21.794 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T05:02:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-13T05:02:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-13T05:02:22.041 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:25.206 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 1/198 2024-04-13T05:02:25.206 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2/198 2024-04-13T05:02:25.206 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 3/198 2024-04-13T05:02:25.206 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 4/198 2024-04-13T05:02:25.206 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 5/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 6/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 7/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 8/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 9/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 13/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 14/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 15/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 17/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 18/198 2024-04-13T05:02:25.207 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 19/198 2024-04-13T05:02:25.208 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 20/198 2024-04-13T05:02:25.208 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 21/198 2024-04-13T05:02:25.208 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 22/198 2024-04-13T05:02:25.208 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 23/198 2024-04-13T05:02:25.208 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 24/198 2024-04-13T05:02:25.208 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 25/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 26/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 27/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 28/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 29/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 30/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 31/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 32/198 2024-04-13T05:02:25.224 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 33/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 34/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 35/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 36/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 37/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 38/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 39/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-13T05:02:25.225 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-13T05:02:25.226 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-13T05:02:25.227 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-13T05:02:25.228 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-13T05:02:25.229 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-13T05:02:25.230 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-13T05:02:25.231 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-13T05:02:25.232 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-13T05:02:25.233 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-13T05:02:25.234 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-13T05:02:25.235 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-13T05:02:25.236 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-13T05:02:27.716 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-2884.g218a6914.el9.x86_64 1/198 2024-04-13T05:02:27.716 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2/198 2024-04-13T05:02:27.716 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 3/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 4/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2884.g218a6 5/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 6/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 7/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 8/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 9/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 10/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 11/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 12/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 13/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 14/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_ 15/198 2024-04-13T05:02:27.717 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 16/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 17/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-2884.g218a6914.el9.x86_6 18/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86 19/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 20/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2884.g218a6914.el 21/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-2884.g218a6914.el9. 22/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_6 23/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 24/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 25/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 26/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 27/198 2024-04-13T05:02:27.718 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 28/198 2024-04-13T05:02:27.733 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 29/198 2024-04-13T05:02:27.733 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2884.g218a6914. 30/198 2024-04-13T05:02:27.733 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noa 31/198 2024-04-13T05:02:27.733 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.n 32/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2884.g218 33/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el 34/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 35/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.e 36/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 37/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-2884.g218a6914.el9.noarch 38/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 39/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-13T05:02:27.734 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-13T05:02:27.737 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-13T05:02:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-13T05:02:27.739 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-13T05:02:27.740 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-13T05:02:27.741 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-2884.g218a6914.el9.x86_64 151/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 153/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-13T05:02:27.742 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-13T05:02:27.743 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-13T05:02:27.744 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-13T05:02:27.745 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-13T05:02:27.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-13T05:02:27.876 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:27.876 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2024-04-13T05:02:27.876 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:27.876 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: acl-2.3.1-4.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.877 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: automake-1.16.2-8.el9.noarch 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: bison-3.7.4-5.el9.x86_64 2024-04-13T05:02:27.878 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: flex-2.6.4-9.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-13T05:02:27.880 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: jq-1.6-16.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-13T05:02:27.881 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: lua-5.4.4-4.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-13T05:02:27.882 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-13.14-1.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-13T05:02:27.883 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-13T05:02:27.885 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply-3.11-14.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-13T05:02:27.886 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-13T05:02:27.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: re2-1:20211101-3.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-13T05:02:27.888 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-13T05:02:27.889 INFO:teuthology.orchestra.run.smithi057.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-13T05:02:27.889 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-13T05:02:27.889 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-13T05:02:27.889 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-13T05:02:27.889 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:02:27.889 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-04-13T05:02:28.778 DEBUG:teuthology.parallel:result is None 2024-04-13T05:02:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-13T05:02:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:30.244 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-13T05:02:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:30.244 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: acl-2.3.1-4.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi083.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-13T05:02:30.246 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.247 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-2884.g218a6914.el9.noarch 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-13T05:02:30.248 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: jq-1.6-16.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-13T05:02:30.249 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-13T05:02:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-13T05:02:30.251 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-13T05:02:30.252 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-13T05:02:30.253 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-13T05:02:30.254 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-13T05:02:30.255 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.256 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-2884.g218a6914.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-13T05:02:30.257 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-13T05:02:31.017 DEBUG:teuthology.parallel:result is None 2024-04-13T05:02:31.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:02:31.233 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi057.stdout:19.0.0-2884.g218a6914.el9 2024-04-13T05:02:31.288 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2884.g218a6914.el9 2024-04-13T05:02:31.288 INFO:teuthology.task.install:The correct ceph version 19.0.0-2884.g218a6914 is installed. 2024-04-13T05:02:31.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:02:31.577 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-13T05:02:31.633 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-2884.g218a6914.el9 2024-04-13T05:02:31.633 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2884.g218a6914.el9 2024-04-13T05:02:31.633 INFO:teuthology.task.install:The correct ceph version 19.0.0-2884.g218a6914 is installed. 2024-04-13T05:02:31.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:02:31.858 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-13T05:02:31.931 INFO:teuthology.orchestra.run.smithi189.stdout:19.0.0-2884.g218a6914.el9 2024-04-13T05:02:31.932 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2884.g218a6914.el9 2024-04-13T05:02:31.932 INFO:teuthology.task.install:The correct ceph version 19.0.0-2884.g218a6914 is installed. 2024-04-13T05:02:31.934 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-13T05:02:31.934 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:02:31.934 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-13T05:02:32.017 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:02:32.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-13T05:02:32.095 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:02:32.095 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-13T05:02:32.200 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-13T05:02:32.200 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:02:32.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2024-04-13T05:02:32.283 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-13T05:02:32.405 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:02:32.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-13T05:02:32.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-13T05:02:32.592 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:02:32.593 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-04-13T05:02:32.689 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-13T05:02:32.829 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-13T05:02:32.829 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:02:32.829 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-13T05:02:32.937 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-13T05:02:33.084 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:02:33.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-13T05:02:33.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-13T05:02:33.270 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:02:33.270 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-13T05:02:33.360 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-13T05:02:33.487 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-13T05:02:33.488 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:02:33.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2024-04-13T05:02:33.591 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-13T05:02:33.724 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:02:33.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-13T05:02:33.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-13T05:02:33.909 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:02:33.909 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-04-13T05:02:34.004 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-13T05:02:34.093 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-13T05:02:34.197 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'subvols': {'create': 2}} 2024-04-13T05:02:34.197 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:02:34.198 INFO:tasks.cephadm:Cluster fsid is 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:02:34.198 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-13T05:02:34.198 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.57', 'mon.b': '172.21.15.83', 'mon.c': '172.21.15.189'} 2024-04-13T05:02:34.198 INFO:tasks.cephadm:First mon is mon.a on smithi057 2024-04-13T05:02:34.198 INFO:tasks.cephadm:First mgr is x 2024-04-13T05:02:34.198 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-13T05:02:34.198 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2024-04-13T05:02:34.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-04-13T05:02:34.403 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-04-13T05:02:34.497 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-13T05:02:34.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:02:34.638 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '218a691417ef57c5d22a8d5cf3c5f41f7b542838', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78520/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2884-g218a6914', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2884.g218a6914'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-12 10:42:09.634437', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240412.042930', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}] 2024-04-13T05:02:34.852 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20240412.042930, sha1 218a691417ef57c5d22a8d5cf3c5f41f7b542838 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:02:34.854 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/x86_64/flavors/default/cephadm 2024-04-13T05:02:34.854 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:35.989 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-r--r--. 1 ubuntu ubuntu 781652 Apr 13 05:02 /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:35.990 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 781652 Apr 13 05:02 /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:37.191 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240412.042930/218a691417ef57c5d22a8d5cf3c5f41f7b542838/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:38.654 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 781652 Apr 13 05:02 /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:38.654 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:38.719 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:38.777 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-13T05:02:38.857 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 on all hosts... 2024-04-13T05:02:38.857 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 pull 2024-04-13T05:02:38.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 pull 2024-04-13T05:02:38.871 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 pull 2024-04-13T05:02:40.095 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838... 2024-04-13T05:02:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838... 2024-04-13T05:02:40.253 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838... 2024-04-13T05:03:24.395 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:03:24.395 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev)", 2024-04-13T05:03:24.395 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05", 2024-04-13T05:03:24.396 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2024-04-13T05:03:24.396 INFO:teuthology.orchestra.run.smithi057.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa" 2024-04-13T05:03:24.396 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:03:24.396 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev)", 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05", 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa" 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-13T05:03:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-13T05:03:39.723 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-13T05:03:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev)", 2024-04-13T05:03:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05", 2024-04-13T05:03:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-04-13T05:03:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa" 2024-04-13T05:03:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-04-13T05:03:39.724 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-13T05:03:39.777 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2024-04-13T05:03:39.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-04-13T05:03:39.992 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-04-13T05:03:40.093 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2024-04-13T05:03:40.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-04-13T05:03:40.302 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-04-13T05:03:40.400 INFO:tasks.cephadm:Writing seed config... 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-13T05:03:40.402 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-13T05:03:40.403 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-13T05:03:40.404 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-13T05:03:40.405 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-13T05:03:40.405 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-13T05:03:40.405 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-13T05:03:40.405 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:03:40.405 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-13T05:03:40.464 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0a178454-f953-11ee-bc8d-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-13T05:03:40.464 DEBUG:teuthology.orchestra.run.smithi057:mon.a> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service 2024-04-13T05:03:40.511 DEBUG:teuthology.orchestra.run.smithi057:mgr.x> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x.service 2024-04-13T05:03:40.557 INFO:tasks.cephadm:Bootstrapping... 2024-04-13T05:03:40.557 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 -v bootstrap --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.57 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:03:40.853 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-04-13T05:03:40.853 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838', '-v', 'bootstrap', '--fsid', '0a178454-f953-11ee-bc8d-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.57', '--skip-admin-label'] 2024-04-13T05:03:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-13T05:03:40.854 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying podman|docker is present... 2024-04-13T05:03:40.915 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-13T05:03:40.916 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying lvm2 is present... 2024-04-13T05:03:40.916 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying time synchronization is in place... 2024-04-13T05:03:40.934 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-13T05:03:40.934 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-13T05:03:40.951 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-13T05:03:40.952 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-04-13T05:03:40.972 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-04-13T05:03:40.991 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-04-13T05:03:40.992 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-04-13T05:03:40.992 INFO:teuthology.orchestra.run.smithi057.stdout:Repeating the final host check... 2024-04-13T05:03:41.061 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-13T05:03:41.062 INFO:teuthology.orchestra.run.smithi057.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-13T05:03:41.062 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl is present 2024-04-13T05:03:41.062 INFO:teuthology.orchestra.run.smithi057.stdout:lvcreate is present 2024-04-13T05:03:41.085 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-13T05:03:41.085 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-04-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-04-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-04-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-04-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi057.stdout:Host looks OK 2024-04-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi057.stdout:Cluster fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi057.stdout:Acquiring lock 140071809880992 on /run/cephadm/0a178454-f953-11ee-bc8d-c7b262605968.lock 2024-04-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi057.stdout:Lock 140071809880992 acquired on /run/cephadm/0a178454-f953-11ee-bc8d-c7b262605968.lock 2024-04-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 3300 ... 2024-04-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 6789 ... 2024-04-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi057.stdout:Base mon IP(s) is [172.21.15.57:3300, 172.21.15.57:6789], mon addrv is [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-04-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.57 metric 100 2024-04-13T05:03:41.178 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2024-04-13T05:03:41.191 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-13T05:03:41.191 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-13T05:03:41.191 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-13T05:03:41.203 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-13T05:03:41.203 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-13T05:03:41.203 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-13T05:03:41.204 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-13T05:03:41.204 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1562/64 scope link noprefixroute 2024-04-13T05:03:41.204 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-13T05:03:41.207 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-04-13T05:03:41.207 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-04-13T05:03:41.207 INFO:teuthology.orchestra.run.smithi057.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-13T05:03:41.210 INFO:teuthology.orchestra.run.smithi057.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-13T05:03:41.212 INFO:teuthology.orchestra.run.smithi057.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838... 2024-04-13T05:03:41.941 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 2024-04-13T05:03:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838... 2024-04-13T05:03:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Getting image source signatures 2024-04-13T05:03:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:9793b5b5582798bde394f4383d142e9f248a25e4ebfd72ee05ea9ec6251a877d 2024-04-13T05:03:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:bc174130f2d13ad2c3f0a0364f095cc7358abb3528a6563dc6f800a0c20467de 2024-04-13T05:03:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying config sha256:a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 2024-04-13T05:03:41.942 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-13T05:03:51.915 INFO:teuthology.orchestra.run.smithi057.stdout:ceph: stdout ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev) 2024-04-13T05:03:51.916 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph version: ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev) 2024-04-13T05:03:51.916 INFO:teuthology.orchestra.run.smithi057.stdout:Extracting ceph user uid/gid from container image... 2024-04-13T05:03:53.028 INFO:teuthology.orchestra.run.smithi057.stdout:stat: stdout 167 167 2024-04-13T05:03:53.028 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial keys... 2024-04-13T05:03:54.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQA5EhpmR2nSHhAAeXajuS4aFvM6qJXLCe0uZQ== 2024-04-13T05:03:55.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQA6EhpmA5tkJxAA8T5pjMWOTJbMInbGGHC91Q== 2024-04-13T05:03:56.497 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQA7EhpmV79GLxAAuGrol685eBfF32ORMSKpqg== 2024-04-13T05:03:56.498 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial monmap... 2024-04-13T05:03:57.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-13T05:03:57.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-13T05:03:57.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool for a [v2:172.21.15.57:3300,v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = quincy 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: set fsid to 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:03:57.601 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mon... 2024-04-13T05:03:58.748 INFO:teuthology.orchestra.run.smithi057.stdout:create mon.a on 2024-04-13T05:03:59.121 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-13T05:03:59.374 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-13T05:03:59.669 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0a178454-f953-11ee-bc8d-c7b262605968.target → /etc/systemd/system/ceph-0a178454-f953-11ee-bc8d-c7b262605968.target. 2024-04-13T05:03:59.669 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0a178454-f953-11ee-bc8d-c7b262605968.target → /etc/systemd/system/ceph-0a178454-f953-11ee-bc8d-c7b262605968.target. 2024-04-13T05:04:00.099 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a 2024-04-13T05:04:00.099 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service: Unit ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service not loaded. 2024-04-13T05:04:00.410 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0a178454-f953-11ee-bc8d-c7b262605968.target.wants/ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0a178454-f953-11ee-bc8d-c7b262605968@.service. 2024-04-13T05:04:01.154 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:00 smithi057 podman[31947]: 2024-04-13 05:04:00.88503817 +0000 UTC m=+0.173730091 container create 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-13T05:04:01.154 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:01 smithi057 podman[31947]: 2024-04-13 05:04:01.155391328 +0000 UTC m=+0.444083284 container init 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:04:01.278 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-13T05:04:01.278 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-04-13T05:04:01.294 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-13T05:04:01.294 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-04-13T05:04:01.294 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-04-13T05:04:01.294 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-04-13T05:04:01.294 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon to start... 2024-04-13T05:04:01.295 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon... 2024-04-13T05:04:01.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:01 smithi057 podman[31947]: 2024-04-13 05:04:01.164986722 +0000 UTC m=+0.453678640 container start 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:04:01.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:01 smithi057 bash[31947]: 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a 2024-04-13T05:04:01.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:01 smithi057 systemd[1]: Started Ceph mon.a for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:04:01.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:01 smithi057 ceph-mon[31966]: mkfs 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:01.437 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:01 smithi057 ceph-mon[31966]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-13T05:04:02.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-13T05:04:02.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: monmap epoch 1 2024-04-13T05:04:02.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: last_changed 2024-04-13T05:03:56.948897+0000 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: min_mon_release 19 (squid) 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: election_strategy: 1 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: fsmap 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: osdmap e1: 0 total, 0 up, 0 in 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: mgrmap e1: no daemons active 2024-04-13T05:04:02.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:02 smithi057 ceph-mon[31966]: from='client.? 172.21.15.57:0/2054159536' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-13T05:04:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout cluster: 2024-04-13T05:04:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout id: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:02.636 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-13T05:04:02.637 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:02.637 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout services: 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.718863s) 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout data: 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-13T05:04:02.640 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pgs: 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.869+0000 7f8972e00700 1 Processor -- start 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.870+0000 7f8972e00700 1 -- start start 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.870+0000 7f8972e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c10a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.870+0000 7f8972e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f896c10ae40 con 0x7f896c10a450 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.872+0000 7f896be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c10a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.872+0000 7f896be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c10a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33994/0 (socket says 172.21.15.57:33994) 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.872+0000 7f896be00700 1 -- 172.21.15.57:0/2214339705 learned_addr learned my addr 172.21.15.57:0/2214339705 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.872+0000 7f896be00700 1 -- 172.21.15.57:0/2214339705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c10b600 con 0x7f896c10a450 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.873+0000 7f896be00700 1 --2- 172.21.15.57:0/2214339705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c10a870 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f895c0098a0 tx=0x7f895c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166a2c9f4f86dd40 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.873+0000 7f896aa00700 1 -- 172.21.15.57:0/2214339705 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c004160 con 0x7f896c10a450 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.873+0000 7f896aa00700 1 -- 172.21.15.57:0/2214339705 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c004300 con 0x7f896c10a450 2024-04-13T05:04:02.641 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.873+0000 7f896aa00700 1 -- 172.21.15.57:0/2214339705 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c0036a0 con 0x7f896c10a450 2024-04-13T05:04:02.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.874+0000 7f8972e00700 1 -- 172.21.15.57:0/2214339705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 msgr2=0x7f896c10a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:02.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.874+0000 7f8972e00700 1 --2- 172.21.15.57:0/2214339705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c10a870 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f895c0098a0 tx=0x7f895c009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:02.642 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.874+0000 7f8972e00700 1 -- 172.21.15.57:0/2214339705 shutdown_connections 2024-04-13T05:04:02.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.874+0000 7f8972e00700 1 --2- 172.21.15.57:0/2214339705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c10a870 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:02.645 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.874+0000 7f8972e00700 1 -- 172.21.15.57:0/2214339705 >> 172.21.15.57:0/2214339705 conn(0x7f896c07db30 msgr2=0x7f896c108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.874+0000 7f8972e00700 1 -- 172.21.15.57:0/2214339705 shutdown_connections 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.875+0000 7f8972e00700 1 -- 172.21.15.57:0/2214339705 wait complete. 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.876+0000 7f8972e00700 1 Processor -- start 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.876+0000 7f8972e00700 1 -- start start 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.876+0000 7f8972e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.877+0000 7f8972e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f896c10bb30 con 0x7f896c10a450 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.877+0000 7f896be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.877+0000 7f896be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33998/0 (socket says 172.21.15.57:33998) 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.877+0000 7f896be00700 1 -- 172.21.15.57:0/2054159536 learned_addr learned my addr 172.21.15.57:0/2054159536 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.877+0000 7f896be00700 1 -- 172.21.15.57:0/2054159536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c19dfb0 con 0x7f896c10a450 2024-04-13T05:04:02.646 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f896be00700 1 --2- 172.21.15.57:0/2054159536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c19da70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f895c000c00 tx=0x7f895c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c003db0 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c011630 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c19e240 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f895c011950 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1a0f30 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.878+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f895c01a460 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.879+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f895c01b780 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.879+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896c064110 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.883+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f895c01b980 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.964+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f896c02e1f0 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.965+0000 7f8968c00700 1 -- 172.21.15.57:0/2054159536 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f895c01bb60 con 0x7f896c10a450 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.966+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 msgr2=0x7f896c19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:02.647 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.966+0000 7f8972e00700 1 --2- 172.21.15.57:0/2054159536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c19da70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f895c000c00 tx=0x7f895c004750 comp rx=0 tx=0).stop 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.966+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 shutdown_connections 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.966+0000 7f8972e00700 1 --2- 172.21.15.57:0/2054159536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f896c10a450 0x7f896c19da70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.966+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 >> 172.21.15.57:0/2054159536 conn(0x7f896c07db30 msgr2=0x7f896c194710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.967+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 shutdown_connections 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:01.967+0000 7f8972e00700 1 -- 172.21.15.57:0/2054159536 wait complete. 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:mon is available 2024-04-13T05:04:02.648 INFO:teuthology.orchestra.run.smithi057.stdout:Assimilating anything we can from ceph.conf... 2024-04-13T05:04:03.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:03 smithi057 ceph-mon[31966]: from='client.? 172.21.15.57:0/3403783033' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-13T05:04:03.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:03 smithi057 ceph-mon[31966]: from='client.? 172.21.15.57:0/3403783033' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-13T05:04:04.034 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:04.034 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-04-13T05:04:04.035 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:04.035 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-04-13T05:04:04.035 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-13T05:04:04.035 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-13T05:04:04.035 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-13T05:04:04.035 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-13T05:04:04.036 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:04.036 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-13T05:04:04.036 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-13T05:04:04.036 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:04.037 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-04-13T05:04:04.037 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-13T05:04:04.037 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-13T05:04:04.038 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-13T05:04:04.038 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-13T05:04:04.038 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-13T05:04:04.039 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.276+0000 7f7986000700 1 Processor -- start 2024-04-13T05:04:04.039 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.276+0000 7f7986000700 1 -- start start 2024-04-13T05:04:04.039 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.277+0000 7f7986000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f798010b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:04.039 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.277+0000 7f7986000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f798007be60 con 0x7f7980108cc0 2024-04-13T05:04:04.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.277+0000 7f797f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f798010b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:04.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.278+0000 7f797f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f798010b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34000/0 (socket says 172.21.15.57:34000) 2024-04-13T05:04:04.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.278+0000 7f797f400700 1 -- 172.21.15.57:0/3891145838 learned_addr learned my addr 172.21.15.57:0/3891145838 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:04.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.278+0000 7f797f400700 1 -- 172.21.15.57:0/3891145838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798010b5f0 con 0x7f7980108cc0 2024-04-13T05:04:04.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.278+0000 7f797f400700 1 --2- 172.21.15.57:0/3891145838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f798010b0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f79700098a0 tx=0x7f7970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c33ea39bb7b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.279+0000 7f797e000700 1 -- 172.21.15.57:0/3891145838 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7970004160 con 0x7f7980108cc0 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.279+0000 7f797e000700 1 -- 172.21.15.57:0/3891145838 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7970004300 con 0x7f7980108cc0 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.279+0000 7f797e000700 1 -- 172.21.15.57:0/3891145838 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79700036a0 con 0x7f7980108cc0 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.279+0000 7f7986000700 1 -- 172.21.15.57:0/3891145838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 msgr2=0x7f798010b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.279+0000 7f7986000700 1 --2- 172.21.15.57:0/3891145838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f798010b0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f79700098a0 tx=0x7f7970009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.280+0000 7f7986000700 1 -- 172.21.15.57:0/3891145838 shutdown_connections 2024-04-13T05:04:04.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.280+0000 7f7986000700 1 --2- 172.21.15.57:0/3891145838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f798010b0b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:04.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.280+0000 7f7986000700 1 -- 172.21.15.57:0/3891145838 >> 172.21.15.57:0/3891145838 conn(0x7f7980102730 msgr2=0x7f7980104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:04.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.280+0000 7f7986000700 1 -- 172.21.15.57:0/3891145838 shutdown_connections 2024-04-13T05:04:04.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.280+0000 7f7986000700 1 -- 172.21.15.57:0/3891145838 wait complete. 2024-04-13T05:04:04.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.281+0000 7f7986000700 1 Processor -- start 2024-04-13T05:04:04.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.281+0000 7f7986000700 1 -- start start 2024-04-13T05:04:04.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.281+0000 7f7986000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f7980108540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:04.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.281+0000 7f7986000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f798010bac0 con 0x7f7980108cc0 2024-04-13T05:04:04.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.282+0000 7f797f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f7980108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:04.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.282+0000 7f797f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f7980108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34004/0 (socket says 172.21.15.57:34004) 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.282+0000 7f797f400700 1 -- 172.21.15.57:0/3403783033 learned_addr learned my addr 172.21.15.57:0/3403783033 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.282+0000 7f797f400700 1 -- 172.21.15.57:0/3403783033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7980108a80 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.282+0000 7f797f400700 1 --2- 172.21.15.57:0/3403783033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f7980108540 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7970000c00 tx=0x7f7970004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7970003db0 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7970011630 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7970011950 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7980106c70 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801071b0 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f797001a460 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.283+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f797001b780 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.284+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7980064110 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.287+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f797001b980 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.367+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7980107650 con 0x7f7980108cc0 2024-04-13T05:04:04.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.372+0000 7f7977e00700 1 -- 172.21.15.57:0/3403783033 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f797001bb60 con 0x7f7980108cc0 2024-04-13T05:04:04.047 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.373+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 msgr2=0x7f7980108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.373+0000 7f7986000700 1 --2- 172.21.15.57:0/3403783033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f7980108540 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7970000c00 tx=0x7f7970004750 comp rx=0 tx=0).stop 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.374+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 shutdown_connections 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.374+0000 7f7986000700 1 --2- 172.21.15.57:0/3403783033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7980108cc0 0x7f7980108540 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.374+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 >> 172.21.15.57:0/3403783033 conn(0x7f7980102730 msgr2=0x7f79801034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.374+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 shutdown_connections 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:03.374+0000 7f7986000700 1 -- 172.21.15.57:0/3403783033 wait complete. 2024-04-13T05:04:04.048 INFO:teuthology.orchestra.run.smithi057.stdout:Generating new minimal ceph.conf... 2024-04-13T05:04:05.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:04 smithi057 ceph-mon[31966]: from='client.? 172.21.15.57:0/616760759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:04:05.343 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.649+0000 7f0eaa600700 1 Processor -- start 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.650+0000 7f0eaa600700 1 -- start start 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.650+0000 7f0eaa600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea4108680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.650+0000 7f0eaa600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4108c50 con 0x7f0ea4108260 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.651+0000 7f0ea3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea4108680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.651+0000 7f0ea3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea4108680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34018/0 (socket says 172.21.15.57:34018) 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.651+0000 7f0ea3400700 1 -- 172.21.15.57:0/4048509445 learned_addr learned my addr 172.21.15.57:0/4048509445 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.651+0000 7f0ea3400700 1 -- 172.21.15.57:0/4048509445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea4109410 con 0x7f0ea4108260 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.652+0000 7f0ea3400700 1 --2- 172.21.15.57:0/4048509445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea4108680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e940098a0 tx=0x7f0e94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3badb530a3ef33ae server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.652+0000 7f0ea2a00700 1 -- 172.21.15.57:0/4048509445 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e94004160 con 0x7f0ea4108260 2024-04-13T05:04:05.344 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.652+0000 7f0ea2a00700 1 -- 172.21.15.57:0/4048509445 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e94004300 con 0x7f0ea4108260 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.652+0000 7f0ea2a00700 1 -- 172.21.15.57:0/4048509445 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e94011a80 con 0x7f0ea4108260 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 -- 172.21.15.57:0/4048509445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 msgr2=0x7f0ea4108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 --2- 172.21.15.57:0/4048509445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea4108680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e940098a0 tx=0x7f0e94009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 -- 172.21.15.57:0/4048509445 shutdown_connections 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 --2- 172.21.15.57:0/4048509445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea4108680 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 -- 172.21.15.57:0/4048509445 >> 172.21.15.57:0/4048509445 conn(0x7f0ea4103860 msgr2=0x7f0ea4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 -- 172.21.15.57:0/4048509445 shutdown_connections 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.653+0000 7f0eaa600700 1 -- 172.21.15.57:0/4048509445 wait complete. 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.654+0000 7f0eaa600700 1 Processor -- start 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.655+0000 7f0eaa600700 1 -- start start 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.655+0000 7f0eaa600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.655+0000 7f0eaa600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea410c7c0 con 0x7f0ea4108260 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.655+0000 7f0ea3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:05.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.655+0000 7f0ea3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea419c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34024/0 (socket says 172.21.15.57:34024) 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.655+0000 7f0ea3400700 1 -- 172.21.15.57:0/616760759 learned_addr learned my addr 172.21.15.57:0/616760759 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.656+0000 7f0ea3400700 1 -- 172.21.15.57:0/616760759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea419c670 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.656+0000 7f0ea3400700 1 --2- 172.21.15.57:0/616760759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea419c130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c00 tx=0x7f0e94004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.656+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e940039c0 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.656+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e94003b60 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.656+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9401a6f0 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.657+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea419c900 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.657+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea419f5f0 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.657+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e9401ac20 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.657+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e94019700 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.658+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea4064110 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.661+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0e94004300 con 0x7f0ea4108260 2024-04-13T05:04:05.346 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.739+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0ea419f7c0 con 0x7f0ea4108260 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.739+0000 7f0ea0c00700 1 -- 172.21.15.57:0/616760759 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0e94032030 con 0x7f0ea4108260 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.741+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 msgr2=0x7f0ea419c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.741+0000 7f0eaa600700 1 --2- 172.21.15.57:0/616760759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea419c130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c00 tx=0x7f0e94004290 comp rx=0 tx=0).stop 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.741+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 shutdown_connections 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.741+0000 7f0eaa600700 1 --2- 172.21.15.57:0/616760759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ea4108260 0x7f0ea419c130 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.741+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 >> 172.21.15.57:0/616760759 conn(0x7f0ea4103860 msgr2=0x7f0ea4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.742+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 shutdown_connections 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:04.742+0000 7f0eaa600700 1 -- 172.21.15.57:0/616760759 wait complete. 2024-04-13T05:04:05.347 INFO:teuthology.orchestra.run.smithi057.stdout:Restarting the monitor... 2024-04-13T05:04:05.637 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:05 smithi057 systemd[1]: Stopping Ceph mon.a for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:04:05.954 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:05 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a[31962]: 2024-04-13T05:04:05.800+0000 7fa161a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:04:05.954 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:05 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a[31962]: 2024-04-13T05:04:05.800+0000 7fa161a00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-13T05:04:06.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:05 smithi057 podman[32207]: 2024-04-13 05:04:05.956110035 +0000 UTC m=+0.266969716 container died 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-13T05:04:06.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:06 smithi057 podman[32207]: 2024-04-13 05:04:06.37701195 +0000 UTC m=+0.687871628 container cleanup 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-13T05:04:06.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:06 smithi057 bash[32207]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a 2024-04-13T05:04:06.878 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:06 smithi057 podman[32221]: 2024-04-13 05:04:06.62836593 +0000 UTC m=+0.660244377 container remove 4a06222cbee4c4134aee28f695d8f001c6f809a70f528349adc807ec07bf542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:04:07.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-13T05:04:07.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 systemd[1]: Stopped Ceph mon.a for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:04:07.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service: Consumed 2.002s CPU time. 2024-04-13T05:04:07.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 systemd[1]: Starting Ceph mon.a for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:04:07.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 podman[32313]: 2024-04-13 05:04:07.473346127 +0000 UTC m=+0.040364509 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:04:07.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 podman[32313]: 2024-04-13 05:04:07.566594125 +0000 UTC m=+0.133612518 container create 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux ) 2024-04-13T05:04:07.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 podman[32313]: 2024-04-13 05:04:07.807465052 +0000 UTC m=+0.374483484 container init 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:04:07.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 podman[32313]: 2024-04-13 05:04:07.818339054 +0000 UTC m=+0.385357470 container start 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:04:07.933 INFO:teuthology.orchestra.run.smithi057.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: set uid:gid to 167:167 (ceph:ceph) 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev), process ceph-mon, pid 2 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: pidfile_write: ignore empty --pid-file 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: load: jerasure load: lrc 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: RocksDB version: 7.9.2 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Git sha 0 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Compile date 2024-04-12 04:47:35 2024-04-13T05:04:08.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: DB SUMMARY 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: DB Session ID: TC92TP1V6ON2I7Q5UPPR 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: CURRENT file: CURRENT 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: IDENTITY file: IDENTITY 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103508 ; 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.error_if_exists: 0 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.create_if_missing: 0 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.paranoid_checks: 1 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-13T05:04:08.087 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.env: 0x55b7b74b1cc0 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.fs: PosixFileSystem 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.info_log: 0x55b7cafa9c40 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_file_opening_threads: 16 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.statistics: (nil) 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.use_fsync: 0 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_log_file_size: 0 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.keep_log_file_num: 1000 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.recycle_log_file_num: 0 2024-04-13T05:04:08.088 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.allow_fallocate: 1 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.allow_mmap_reads: 0 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.allow_mmap_writes: 0 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.use_direct_reads: 0 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.create_missing_column_families: 0 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.db_log_dir: 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.wal_dir: 2024-04-13T05:04:08.089 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.advise_random_on_open: 1 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.db_write_buffer_size: 0 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.write_buffer_manager: 0x55b7caf7a1e0 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-13T05:04:08.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.rate_limiter: (nil) 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.wal_recovery_mode: 2 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.enable_thread_tracking: 0 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.enable_pipelined_write: 0 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.unordered_write: 0 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.row_cache: None 2024-04-13T05:04:08.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.wal_filter: None 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.allow_ingest_behind: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.two_write_queues: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.manual_wal_flush: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.wal_compression: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.atomic_flush: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.log_readahead_size: 0 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-13T05:04:08.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.best_efforts_recovery: 0 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.allow_data_in_errors: 0 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.db_host_id: __hostname__ 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_background_jobs: 2 2024-04-13T05:04:08.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_background_compactions: -1 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_subcompactions: 1 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_total_wal_size: 0 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-13T05:04:08.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_open_files: -1 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bytes_per_sync: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_readahead_size: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_background_flushes: -1 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Compression algorithms supported: 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kZSTD supported: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kXpressCompression supported: 0 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kLZ4HCCompression supported: 1 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kZlibCompression supported: 1 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kSnappyCompression supported: 1 2024-04-13T05:04:08.095 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kLZ4Compression supported: 1 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: kBZip2Compression supported: 0 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.merge_operator: 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_filter: None 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_filter_factory: None 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.sst_partitioner_factory: None 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-13T05:04:08.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b7cafa9e00) 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x55b7cafad090 2024-04-13T05:04:08.097 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2024-04-13T05:04:08.098 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2024-04-13T05:04:08.099 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.write_buffer_size: 33554432 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_write_buffer_number: 2 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression: NoCompression 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression: Disabled 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.prefix_extractor: nullptr 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.num_levels: 7 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-13T05:04:08.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-13T05:04:08.101 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.level: 32767 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.strategy: 0 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-13T05:04:08.102 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.enabled: false 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.target_file_size_base: 67108864 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-13T05:04:08.103 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.arena_block_size: 1048576 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-13T05:04:08.104 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.disable_auto_compactions: 0 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-13T05:04:08.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.inplace_update_support: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.bloom_locality: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.max_successive_merges: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.paranoid_file_checks: 0 2024-04-13T05:04:08.106 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.force_consistency_checks: 1 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.report_bg_io_stats: 0 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.ttl: 2592000 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.enable_blob_files: false 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.min_blob_size: 0 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.blob_file_size: 268435456 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-13T05:04:08.107 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.blob_file_starting_level: 0 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4651acf7-5ced-42a5-9d1c-0c6fbe680f08 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984647876894, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984647878704, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98804, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96747, "index_size": 313, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13788, "raw_average_key_size": 49, "raw_value_size": 89195, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712984647, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4651acf7-5ced-42a5-9d1c-0c6fbe680f08", "db_session_id": "TC92TP1V6ON2I7Q5UPPR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984647878870, "job": 1, "event": "recovery_finished"} 2024-04-13T05:04:08.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b7cb09c000 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: DB pointer 0x55b7cb086000 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-13T05:04:08.109 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 98.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 98.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-13T05:04:08.110 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 80.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 8.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 8.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x55b7cafad090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-04-13T05:04:08.111 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???) e1 preinit fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).mds e1 new map 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).mds e1 print_map 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-13T05:04:08.112 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 bash[32313]: 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 systemd[1]: Started Ceph mon.a for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: monmap epoch 1 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: last_changed 2024-04-13T05:03:56.948897+0000 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: min_mon_release 19 (squid) 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: election_strategy: 1 2024-04-13T05:04:08.113 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:04:08.114 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: fsmap 2024-04-13T05:04:08.114 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: osdmap e1: 0 total, 0 up, 0 in 2024-04-13T05:04:08.114 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:07 smithi057 ceph-mon[32331]: mgrmap e1: no daemons active 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.637+0000 7f4ddf000700 1 Processor -- start 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.637+0000 7f4ddf000700 1 -- start start 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.638+0000 7f4ddf000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd8111240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.638+0000 7f4ddf000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8111810 con 0x7f4dd810ee50 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.638+0000 7f4dddc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd8111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.638+0000 7f4dddc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd8111240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49006/0 (socket says 172.21.15.57:49006) 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.638+0000 7f4dddc00700 1 -- 172.21.15.57:0/1022830527 learned_addr learned my addr 172.21.15.57:0/1022830527 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.639+0000 7f4dddc00700 1 -- 172.21.15.57:0/1022830527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd8111990 con 0x7f4dd810ee50 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.639+0000 7f4dddc00700 1 --2- 172.21.15.57:0/1022830527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd8111240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0098a0 tx=0x7f4dcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfecf70ebceee965 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4dd7e00700 1 -- 172.21.15.57:0/1022830527 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dcc004160 con 0x7f4dd810ee50 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4dd7e00700 1 -- 172.21.15.57:0/1022830527 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4dcc004300 con 0x7f4dd810ee50 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4dd7e00700 1 -- 172.21.15.57:0/1022830527 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dcc011a80 con 0x7f4dd810ee50 2024-04-13T05:04:09.351 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4ddf000700 1 -- 172.21.15.57:0/1022830527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 msgr2=0x7f4dd8111240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4ddf000700 1 --2- 172.21.15.57:0/1022830527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd8111240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0098a0 tx=0x7f4dcc009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4ddf000700 1 -- 172.21.15.57:0/1022830527 shutdown_connections 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4ddf000700 1 --2- 172.21.15.57:0/1022830527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd8111240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.640+0000 7f4ddf000700 1 -- 172.21.15.57:0/1022830527 >> 172.21.15.57:0/1022830527 conn(0x7f4dd807d9a0 msgr2=0x7f4dd807fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.641+0000 7f4ddf000700 1 -- 172.21.15.57:0/1022830527 shutdown_connections 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.641+0000 7f4ddf000700 1 -- 172.21.15.57:0/1022830527 wait complete. 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4ddf000700 1 Processor -- start 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4ddf000700 1 -- start start 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4ddf000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd81a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4ddf000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81129b0 con 0x7f4dd810ee50 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4dddc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd81a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4dddc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd81a24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49018/0 (socket says 172.21.15.57:49018) 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.642+0000 7f4dddc00700 1 -- 172.21.15.57:0/2848156298 learned_addr learned my addr 172.21.15.57:0/2848156298 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:09.352 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.643+0000 7f4dddc00700 1 -- 172.21.15.57:0/2848156298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd81a29e0 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.643+0000 7f4dddc00700 1 --2- 172.21.15.57:0/2848156298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd81a24a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000c00 tx=0x7f4dcc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.643+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dcc0039f0 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.643+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4dcc003b90 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.643+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd81a2c70 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.643+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dcc01a740 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.644+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd81a5960 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.644+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4dcc02d3f0 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.644+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4dcc019700 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.645+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd8064110 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.648+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4dcc024af0 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.726+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f4dd81a5b30 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.730+0000 7f4dd6000700 1 -- 172.21.15.57:0/2848156298 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4dcc024440 con 0x7f4dd810ee50 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 msgr2=0x7f4dd81a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 --2- 172.21.15.57:0/2848156298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd81a24a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000c00 tx=0x7f4dcc011b00 comp rx=0 tx=0).stop 2024-04-13T05:04:09.353 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 shutdown_connections 2024-04-13T05:04:09.354 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 --2- 172.21.15.57:0/2848156298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dd810ee50 0x7f4dd81a24a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:09.354 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 >> 172.21.15.57:0/2848156298 conn(0x7f4dd807d9a0 msgr2=0x7f4dd807f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:09.354 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 shutdown_connections 2024-04-13T05:04:09.354 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:08.731+0000 7f4ddf000700 1 -- 172.21.15.57:0/2848156298 wait complete. 2024-04-13T05:04:09.354 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-13T05:04:09.377 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:04:09.377 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mgr... 2024-04-13T05:04:09.378 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-13T05:04:09.380 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-13T05:04:09.834 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x 2024-04-13T05:04:09.834 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x.service: Unit ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x.service not loaded. 2024-04-13T05:04:10.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2848156298' entity='client.admin' 2024-04-13T05:04:10.095 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0a178454-f953-11ee-bc8d-c7b262605968.target.wants/ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0a178454-f953-11ee-bc8d-c7b262605968@.service. 2024-04-13T05:04:10.512 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 systemd[1]: Starting Ceph mgr.x for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:04:10.780 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 podman[32544]: 2024-04-13 05:04:10.418944817 +0000 UTC m=+0.036524517 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:04:10.780 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 podman[32544]: 2024-04-13 05:04:10.549832042 +0000 UTC m=+0.167411772 container create 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-13T05:04:10.927 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-13T05:04:10.927 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-04-13T05:04:10.945 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-13T05:04:10.946 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-04-13T05:04:10.946 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-04-13T05:04:10.946 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-04-13T05:04:10.969 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-13T05:04:10.969 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-04-13T05:04:10.985 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-13T05:04:10.986 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-04-13T05:04:10.986 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-04-13T05:04:10.986 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-13T05:04:10.986 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr to start... 2024-04-13T05:04:10.986 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr... 2024-04-13T05:04:11.147 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 podman[32544]: 2024-04-13 05:04:10.794588102 +0000 UTC m=+0.412167839 container init 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:04:11.147 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 podman[32544]: 2024-04-13 05:04:10.805666442 +0000 UTC m=+0.423246175 container start 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-13T05:04:11.147 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 bash[32544]: 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f 2024-04-13T05:04:11.147 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:10 smithi057 systemd[1]: Started Ceph mgr.x for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:04:11.763 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:11.632+0000 7ffaa1e62200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T05:04:11.763 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:11.761+0000 7ffaa1e62200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T05:04:12.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:11 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/83309097' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-13T05:04:12.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:12.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:04:12.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:04:12.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-04-13T05:04:12.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-13T05:04:12.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-13T05:04:12.523 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-13T05:04:12.524 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-13T05:04:12.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-13T05:04:12.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-04-13T05:04:01.250728+0000", 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.672+0000 7ff4fda00700 1 Processor -- start 2024-04-13T05:04:12.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.672+0000 7ff4fda00700 1 -- start start 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.672+0000 7ff4fda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f8108680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.672+0000 7ff4fda00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8108bc0 con 0x7ff4f8108260 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.673+0000 7ff4f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f8108680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.673+0000 7ff4f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f8108680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49034/0 (socket says 172.21.15.57:49034) 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.673+0000 7ff4f6a00700 1 -- 172.21.15.57:0/2257239791 learned_addr learned my addr 172.21.15.57:0/2257239791 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.673+0000 7ff4f6a00700 1 -- 172.21.15.57:0/2257239791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f8108d40 con 0x7ff4f8108260 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.674+0000 7ff4f6a00700 1 --2- 172.21.15.57:0/2257239791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f8108680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80098a0 tx=0x7ff4e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eef032edfb9b29f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.674+0000 7ff4f5600700 1 -- 172.21.15.57:0/2257239791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e8004160 con 0x7ff4f8108260 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.674+0000 7ff4f5600700 1 -- 172.21.15.57:0/2257239791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff4e8004300 con 0x7ff4f8108260 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.674+0000 7ff4f5600700 1 -- 172.21.15.57:0/2257239791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e8011a80 con 0x7ff4f8108260 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 -- 172.21.15.57:0/2257239791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 msgr2=0x7ff4f8108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 --2- 172.21.15.57:0/2257239791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f8108680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80098a0 tx=0x7ff4e8009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 -- 172.21.15.57:0/2257239791 shutdown_connections 2024-04-13T05:04:12.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 --2- 172.21.15.57:0/2257239791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f8108680 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 -- 172.21.15.57:0/2257239791 >> 172.21.15.57:0/2257239791 conn(0x7ff4f8103940 msgr2=0x7ff4f8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 -- 172.21.15.57:0/2257239791 shutdown_connections 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.675+0000 7ff4fda00700 1 -- 172.21.15.57:0/2257239791 wait complete. 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.676+0000 7ff4fda00700 1 Processor -- start 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.676+0000 7ff4fda00700 1 -- start start 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.676+0000 7ff4fda00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.676+0000 7ff4fda00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810a260 con 0x7ff4f8108260 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.677+0000 7ff4f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.677+0000 7ff4f6a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49048/0 (socket says 172.21.15.57:49048) 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.677+0000 7ff4f6a00700 1 -- 172.21.15.57:0/83309097 learned_addr learned my addr 172.21.15.57:0/83309097 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.677+0000 7ff4f6a00700 1 -- 172.21.15.57:0/83309097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f819e800 con 0x7ff4f8108260 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.677+0000 7ff4f6a00700 1 --2- 172.21.15.57:0/83309097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f819e2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8011a50 tx=0x7ff4e8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.678+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e80039f0 con 0x7ff4f8108260 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.678+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff4e8003b90 con 0x7ff4f8108260 2024-04-13T05:04:12.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.678+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f819ea90 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.678+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4e801a600 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.678+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4e801ac80 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.678+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a1780 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.679+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4e8024780 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.680+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f8064110 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.682+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff4e8024a40 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.762+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4f81a1d20 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.762+0000 7ff4ef400700 1 -- 172.21.15.57:0/83309097 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff4e800b740 con 0x7ff4f8108260 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 msgr2=0x7ff4f819e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 --2- 172.21.15.57:0/83309097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f819e2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8011a50 tx=0x7ff4e8004290 comp rx=0 tx=0).stop 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 shutdown_connections 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 --2- 172.21.15.57:0/83309097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4f8108260 0x7ff4f819e2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 >> 172.21.15.57:0/83309097 conn(0x7ff4f8103940 msgr2=0x7ff4f8104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:12.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 shutdown_connections 2024-04-13T05:04:12.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:11.764+0000 7ff4fda00700 1 -- 172.21.15.57:0/83309097 wait complete. 2024-04-13T05:04:12.539 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (1/15)... 2024-04-13T05:04:12.770 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:12 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:12.645+0000 7ffaa1e62200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T05:04:12.770 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:12 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:12.772+0000 7ffaa1e62200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T05:04:13.028 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:12 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:12.957+0000 7ffaa1e62200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T05:04:13.528 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:13.087+0000 7ffaa1e62200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T05:04:13.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:13.201+0000 7ffaa1e62200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T05:04:13.824 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:13.561+0000 7ffaa1e62200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T05:04:13.824 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:13.708+0000 7ffaa1e62200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T05:04:13.824 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:13.826+0000 7ffaa1e62200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T05:04:14.180 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:13.956+0000 7ffaa1e62200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T05:04:14.180 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:14 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:14.182+0000 7ffaa1e62200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T05:04:15.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:15 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2886267684' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-13T05:04:15.863 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:15.581+0000 7ffaa1e62200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T05:04:15.863 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:15.764+0000 7ffaa1e62200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.883 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-13T05:04:15.884 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-13T05:04:15.885 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-04-13T05:04:15.886 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-04-13T05:04:01.250728+0000", 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.107+0000 7ff056800700 1 Processor -- start 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.107+0000 7ff056800700 1 -- start start 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.108+0000 7ff056800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.108+0000 7ff056800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff05007c880 con 0x7ff05007de00 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.108+0000 7ff04f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:15.887 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.108+0000 7ff04f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56184/0 (socket says 172.21.15.57:56184) 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.108+0000 7ff04f400700 1 -- 172.21.15.57:0/546046073 learned_addr learned my addr 172.21.15.57:0/546046073 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.108+0000 7ff04f400700 1 -- 172.21.15.57:0/546046073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05007ca00 con 0x7ff05007de00 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.109+0000 7ff04f400700 1 --2- 172.21.15.57:0/546046073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05007c2b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098a0 tx=0x7ff038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3921147a715a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.109+0000 7ff04e000700 1 -- 172.21.15.57:0/546046073 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff038004160 con 0x7ff05007de00 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.109+0000 7ff04e000700 1 -- 172.21.15.57:0/546046073 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff038004300 con 0x7ff05007de00 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.109+0000 7ff04e000700 1 -- 172.21.15.57:0/546046073 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff038011a80 con 0x7ff05007de00 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 -- 172.21.15.57:0/546046073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 msgr2=0x7ff05007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 --2- 172.21.15.57:0/546046073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05007c2b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098a0 tx=0x7ff038009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 -- 172.21.15.57:0/546046073 shutdown_connections 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 --2- 172.21.15.57:0/546046073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05007c2b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 -- 172.21.15.57:0/546046073 >> 172.21.15.57:0/546046073 conn(0x7ff050103860 msgr2=0x7ff050105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 -- 172.21.15.57:0/546046073 shutdown_connections 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.110+0000 7ff056800700 1 -- 172.21.15.57:0/546046073 wait complete. 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.112+0000 7ff056800700 1 Processor -- start 2024-04-13T05:04:15.888 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.112+0000 7ff056800700 1 -- start start 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.112+0000 7ff056800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05019e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.112+0000 7ff056800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff05010e7b0 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.112+0000 7ff04f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05019e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.113+0000 7ff04f400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05019e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56188/0 (socket says 172.21.15.57:56188) 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.113+0000 7ff04f400700 1 -- 172.21.15.57:0/2886267684 learned_addr learned my addr 172.21.15.57:0/2886267684 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.113+0000 7ff04f400700 1 -- 172.21.15.57:0/2886267684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05019e820 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.113+0000 7ff04f400700 1 --2- 172.21.15.57:0/2886267684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05019e2e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff038011a50 tx=0x7ff038004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.113+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0380039f0 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.114+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff038003b90 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.114+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05019eab0 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.114+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03801a600 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.114+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501a17a0 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.114+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff03801a8b0 con 0x7ff05007de00 2024-04-13T05:04:15.889 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.115+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff03802ddb0 con 0x7ff05007de00 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.115+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff050064110 con 0x7ff05007de00 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.117+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff038024a40 con 0x7ff05007de00 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.197+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff0501a1970 con 0x7ff05007de00 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.198+0000 7ff047e00700 1 -- 172.21.15.57:0/2886267684 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff038032030 con 0x7ff05007de00 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.200+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 msgr2=0x7ff05019e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.200+0000 7ff056800700 1 --2- 172.21.15.57:0/2886267684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05019e2e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff038011a50 tx=0x7ff038004290 comp rx=0 tx=0).stop 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.200+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 shutdown_connections 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.200+0000 7ff056800700 1 --2- 172.21.15.57:0/2886267684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff05007de00 0x7ff05019e2e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.200+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 >> 172.21.15.57:0/2886267684 conn(0x7ff050103860 msgr2=0x7ff050104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.200+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 shutdown_connections 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:15.201+0000 7ff056800700 1 -- 172.21.15.57:0/2886267684 wait complete. 2024-04-13T05:04:15.890 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (2/15)... 2024-04-13T05:04:16.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:16.202+0000 7ffaa1e62200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T05:04:16.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:16.327+0000 7ffaa1e62200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T05:04:16.852 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:16.596+0000 7ffaa1e62200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:04:17.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:16.854+0000 7ffaa1e62200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T05:04:17.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:17.252+0000 7ffaa1e62200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:04:17.900 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:17.578+0000 7ffaa1e62200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T05:04:18.167 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:17.902+0000 7ffaa1e62200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T05:04:18.778 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:18 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3234588672' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-13T05:04:19.238 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:19.035+0000 7ffaa1e62200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T05:04:19.249 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:19.249 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:04:19.249 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:04:19.249 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-04-13T05:04:19.249 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-13T05:04:19.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-13T05:04:19.251 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-13T05:04:19.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-13T05:04:19.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-13T05:04:19.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.258 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.259 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:19.260 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-04-13T05:04:01.250728+0000", 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.466+0000 7f9e34e00700 1 Processor -- start 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.467+0000 7f9e34e00700 1 -- start start 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.467+0000 7f9e34e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e30108680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.467+0000 7f9e34e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30108c50 con 0x7f9e30108260 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.467+0000 7f9e2de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e30108680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.468+0000 7f9e2de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e30108680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56200/0 (socket says 172.21.15.57:56200) 2024-04-13T05:04:19.261 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.468+0000 7f9e2de00700 1 -- 172.21.15.57:0/451550058 learned_addr learned my addr 172.21.15.57:0/451550058 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.468+0000 7f9e2de00700 1 -- 172.21.15.57:0/451550058 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e30109410 con 0x7f9e30108260 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.468+0000 7f9e2de00700 1 --2- 172.21.15.57:0/451550058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e30108680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e200098a0 tx=0x7f9e20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c90555c1d696e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e2ca00700 1 -- 172.21.15.57:0/451550058 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e20004160 con 0x7f9e30108260 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e2ca00700 1 -- 172.21.15.57:0/451550058 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9e20004300 con 0x7f9e30108260 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e2ca00700 1 -- 172.21.15.57:0/451550058 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e20011a80 con 0x7f9e30108260 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e34e00700 1 -- 172.21.15.57:0/451550058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 msgr2=0x7f9e30108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e34e00700 1 --2- 172.21.15.57:0/451550058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e30108680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e200098a0 tx=0x7f9e20009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e34e00700 1 -- 172.21.15.57:0/451550058 shutdown_connections 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e34e00700 1 --2- 172.21.15.57:0/451550058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e30108680 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.469+0000 7f9e34e00700 1 -- 172.21.15.57:0/451550058 >> 172.21.15.57:0/451550058 conn(0x7f9e30103860 msgr2=0x7f9e30105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:19.262 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.470+0000 7f9e34e00700 1 -- 172.21.15.57:0/451550058 shutdown_connections 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.470+0000 7f9e34e00700 1 -- 172.21.15.57:0/451550058 wait complete. 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e34e00700 1 Processor -- start 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e34e00700 1 -- start start 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e34e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e3019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e34e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3010c7c0 con 0x7f9e30108260 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e2de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e3019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e2de00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e3019e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56216/0 (socket says 172.21.15.57:56216) 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.471+0000 7f9e2de00700 1 -- 172.21.15.57:0/3234588672 learned_addr learned my addr 172.21.15.57:0/3234588672 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.472+0000 7f9e2de00700 1 -- 172.21.15.57:0/3234588672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3019e850 con 0x7f9e30108260 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.472+0000 7f9e2de00700 1 --2- 172.21.15.57:0/3234588672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e3019e310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9e20000c00 tx=0x7f9e20004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.472+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e200039c0 con 0x7f9e30108260 2024-04-13T05:04:19.263 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.472+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9e20003b60 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.472+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e2001a6f0 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.472+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3019eae0 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.473+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e301a17d0 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.473+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e2001ac20 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.473+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e20019700 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.474+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e30064110 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.476+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9e20004300 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.555+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9e301a19a0 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.556+0000 7f9e26a00700 1 -- 172.21.15.57:0/3234588672 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f9e20032030 con 0x7f9e30108260 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.557+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 msgr2=0x7f9e3019e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.557+0000 7f9e34e00700 1 --2- 172.21.15.57:0/3234588672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e3019e310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9e20000c00 tx=0x7f9e20004290 comp rx=0 tx=0).stop 2024-04-13T05:04:19.264 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.558+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 shutdown_connections 2024-04-13T05:04:19.265 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.558+0000 7f9e34e00700 1 --2- 172.21.15.57:0/3234588672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e30108260 0x7f9e3019e310 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:19.265 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.558+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 >> 172.21.15.57:0/3234588672 conn(0x7f9e30103860 msgr2=0x7f9e30104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:19.265 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.558+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 shutdown_connections 2024-04-13T05:04:19.265 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:18.558+0000 7f9e34e00700 1 -- 172.21.15.57:0/3234588672 wait complete. 2024-04-13T05:04:19.265 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (3/15)... 2024-04-13T05:04:20.029 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:19.763+0000 7ffaa1e62200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T05:04:20.029 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:19.899+0000 7ffaa1e62200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T05:04:20.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:20.192+0000 7ffaa1e62200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T05:04:20.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: Activating manager daemon x 2024-04-13T05:04:20.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: mgrmap e2: x(active, starting, since 0.0122629s) 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: Manager daemon x is now available 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-13T05:04:20.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' 2024-04-13T05:04:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:20 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-13T05:04:21.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:21 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' 2024-04-13T05:04:21.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:21 smithi057 ceph-mon[32331]: from='mgr.14100 172.21.15.57:0/3081477426' entity='mgr.x' 2024-04-13T05:04:21.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:21 smithi057 ceph-mon[32331]: mgrmap e3: x(active, since 1.0219s) 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-13T05:04:23.124 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-13T05:04:23.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-13T05:04:23.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-13T05:04:23.134 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-13T05:04:23.134 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-13T05:04:23.134 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-13T05:04:23.134 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.134 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-13T05:04:23.134 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-04-13T05:04:01.250728+0000", 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-04-13T05:04:23.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.984+0000 7fc9ce000700 1 Processor -- start 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.984+0000 7fc9ce000700 1 -- start start 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.984+0000 7fc9ce000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c810b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.984+0000 7fc9ce000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c807be00 con 0x7fc9c8108d20 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.985+0000 7fc9c7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c810b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.985+0000 7fc9c7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c810b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56268/0 (socket says 172.21.15.57:56268) 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.985+0000 7fc9c7400700 1 -- 172.21.15.57:0/1158702141 learned_addr learned my addr 172.21.15.57:0/1158702141 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.985+0000 7fc9c7400700 1 -- 172.21.15.57:0/1158702141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c810b650 con 0x7fc9c8108d20 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.985+0000 7fc9c7400700 1 --2- 172.21.15.57:0/1158702141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c810b110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80098a0 tx=0x7fc9b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4444d1d2f346769 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.986+0000 7fc9c6000700 1 -- 172.21.15.57:0/1158702141 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8004160 con 0x7fc9c8108d20 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.986+0000 7fc9c6000700 1 -- 172.21.15.57:0/1158702141 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc9b8004300 con 0x7fc9c8108d20 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.986+0000 7fc9c6000700 1 -- 172.21.15.57:0/1158702141 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8011a80 con 0x7fc9c8108d20 2024-04-13T05:04:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 -- 172.21.15.57:0/1158702141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 msgr2=0x7fc9c810b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 --2- 172.21.15.57:0/1158702141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c810b110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80098a0 tx=0x7fc9b8009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 -- 172.21.15.57:0/1158702141 shutdown_connections 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 --2- 172.21.15.57:0/1158702141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c810b110 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 -- 172.21.15.57:0/1158702141 >> 172.21.15.57:0/1158702141 conn(0x7fc9c8102730 msgr2=0x7fc9c8104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 -- 172.21.15.57:0/1158702141 shutdown_connections 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.987+0000 7fc9ce000700 1 -- 172.21.15.57:0/1158702141 wait complete. 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.988+0000 7fc9ce000700 1 Processor -- start 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.988+0000 7fc9ce000700 1 -- start start 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.989+0000 7fc9ce000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.989+0000 7fc9ce000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c810c5c0 con 0x7fc9c8108d20 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.989+0000 7fc9c7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.989+0000 7fc9c7400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c819c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56270/0 (socket says 172.21.15.57:56270) 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.989+0000 7fc9c7400700 1 -- 172.21.15.57:0/3769930376 learned_addr learned my addr 172.21.15.57:0/3769930376 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.990+0000 7fc9c7400700 1 -- 172.21.15.57:0/3769930376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c819c640 con 0x7fc9c8108d20 2024-04-13T05:04:23.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.990+0000 7fc9c7400700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c819c100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011a50 tx=0x7fc9b8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.990+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b80039f0 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.990+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c819c8d0 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.990+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc9b8003b90 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.990+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b801a600 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.991+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c819f5c0 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.992+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7fc9b8024440 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.992+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c8064110 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.992+0000 7fc9bfe00700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc9ac039550 0x7fc9ac03ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.992+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc9b804d870 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.993+0000 7fc9c6a00700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc9ac039550 0x7fc9ac03ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.993+0000 7fc9c6a00700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc9ac039550 0x7fc9ac03ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4006fd0 tx=0x7fc9b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:21.999+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9b800f2e0 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.261+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc9b804fd00 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.311+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc9c819f790 con 0x7fc9c8108d20 2024-04-13T05:04:23.139 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.312+0000 7fc9bfe00700 1 -- 172.21.15.57:0/3769930376 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc9c819f790 con 0x7fc9c8108d20 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.316+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc9ac039550 msgr2=0x7fc9ac03ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.316+0000 7fc9ce000700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc9ac039550 0x7fc9ac03ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4006fd0 tx=0x7fc9b4006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.316+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 msgr2=0x7fc9c819c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.316+0000 7fc9ce000700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c819c100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011a50 tx=0x7fc9b8004290 comp rx=0 tx=0).stop 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.317+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 shutdown_connections 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.317+0000 7fc9ce000700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc9ac039550 0x7fc9ac03ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.317+0000 7fc9ce000700 1 --2- 172.21.15.57:0/3769930376 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc9c8108d20 0x7fc9c819c100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.317+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 >> 172.21.15.57:0/3769930376 conn(0x7fc9c8102730 msgr2=0x7fc9c8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.317+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 shutdown_connections 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:22.317+0000 7fc9ce000700 1 -- 172.21.15.57:0/3769930376 wait complete. 2024-04-13T05:04:23.140 INFO:teuthology.orchestra.run.smithi057.stdout:mgr is available 2024-04-13T05:04:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:23 smithi057 ceph-mon[32331]: mgrmap e4: x(active, since 2s) 2024-04-13T05:04:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:23 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3769930376' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-13T05:04:24.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:24 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1599828167' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-13T05:04:24.735 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.800+0000 7f9945400700 1 Processor -- start 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.801+0000 7f9945400700 1 -- start start 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.801+0000 7f9945400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f994007ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:24.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.801+0000 7f9945400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f994007f200 con 0x7f994007e810 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.801+0000 7f993e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f994007ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.801+0000 7f993e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f994007ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56280/0 (socket says 172.21.15.57:56280) 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.801+0000 7f993e800700 1 -- 172.21.15.57:0/494990022 learned_addr learned my addr 172.21.15.57:0/494990022 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.802+0000 7f993e800700 1 -- 172.21.15.57:0/494990022 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994007fa10 con 0x7f994007e810 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.802+0000 7f993e800700 1 --2- 172.21.15.57:0/494990022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f994007ec30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99300098a0 tx=0x7f9930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0244fc3ff54231e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.802+0000 7f993d400700 1 -- 172.21.15.57:0/494990022 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930004160 con 0x7f994007e810 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.802+0000 7f993d400700 1 -- 172.21.15.57:0/494990022 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9930004300 con 0x7f994007e810 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.802+0000 7f993d400700 1 -- 172.21.15.57:0/494990022 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930011a80 con 0x7f994007e810 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 -- 172.21.15.57:0/494990022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 msgr2=0x7f994007ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 --2- 172.21.15.57:0/494990022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f994007ec30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99300098a0 tx=0x7f9930009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 -- 172.21.15.57:0/494990022 shutdown_connections 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 --2- 172.21.15.57:0/494990022 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f994007ec30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 -- 172.21.15.57:0/494990022 >> 172.21.15.57:0/494990022 conn(0x7f994007d820 msgr2=0x7f9940108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 -- 172.21.15.57:0/494990022 shutdown_connections 2024-04-13T05:04:24.737 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.803+0000 7f9945400700 1 -- 172.21.15.57:0/494990022 wait complete. 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.804+0000 7f9945400700 1 Processor -- start 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.804+0000 7f9945400700 1 -- start start 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.805+0000 7f9945400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f99401a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.805+0000 7f9945400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9940080dd0 con 0x7f994007e810 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.805+0000 7f993e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f99401a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.805+0000 7f993e800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f99401a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56290/0 (socket says 172.21.15.57:56290) 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.805+0000 7f993e800700 1 -- 172.21.15.57:0/1599828167 learned_addr learned my addr 172.21.15.57:0/1599828167 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.805+0000 7f993e800700 1 -- 172.21.15.57:0/1599828167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99401a2c00 con 0x7f994007e810 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.806+0000 7f993e800700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f99401a26c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9930011a50 tx=0x7f9930004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.806+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99300039f0 con 0x7f994007e810 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.806+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9930003b90 con 0x7f994007e810 2024-04-13T05:04:24.738 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.806+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99401a2e90 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.806+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993001a600 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.806+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a5b80 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.807+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f9930024a40 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.807+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9940064110 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.808+0000 7f9937400700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7f9924039670 0x7f992403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.808+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f993004da90 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.808+0000 7f993de00700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7f9924039670 0x7f992403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.809+0000 7f993de00700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7f9924039670 0x7f992403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:23.813+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f993004d4b0 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.044+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f99400008d0 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.046+0000 7f9937400700 1 -- 172.21.15.57:0/1599828167 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f993002d770 con 0x7f994007e810 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7f9924039670 msgr2=0x7f992403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7f9924039670 0x7f992403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 msgr2=0x7f99401a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:24.739 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f99401a26c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9930011a50 tx=0x7f9930004290 comp rx=0 tx=0).stop 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 shutdown_connections 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7f9924039670 0x7f992403bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 --2- 172.21.15.57:0/1599828167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f994007e810 0x7f99401a26c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 >> 172.21.15.57:0/1599828167 conn(0x7f994007d820 msgr2=0x7f99401993e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 shutdown_connections 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:24.049+0000 7f9945400700 1 -- 172.21.15.57:0/1599828167 wait complete. 2024-04-13T05:04:24.740 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling cephadm module... 2024-04-13T05:04:26.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:25 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2668350524' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-13T05:04:26.029 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: ignoring --setuser ceph since I am not root 2024-04-13T05:04:26.029 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: ignoring --setgroup ceph since I am not root 2024-04-13T05:04:26.418 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.314+0000 7ff8eea00700 1 Processor -- start 2024-04-13T05:04:26.423 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.314+0000 7ff8eea00700 1 -- start start 2024-04-13T05:04:26.423 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.315+0000 7ff8eea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:26.423 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.315+0000 7ff8eea00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e807c7d0 con 0x7ff8e807dde0 2024-04-13T05:04:26.423 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.315+0000 7ff8e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:26.423 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.316+0000 7ff8e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60534/0 (socket says 172.21.15.57:60534) 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.316+0000 7ff8e7e00700 1 -- 172.21.15.57:0/1334162345 learned_addr learned my addr 172.21.15.57:0/1334162345 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.316+0000 7ff8e7e00700 1 -- 172.21.15.57:0/1334162345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e807c950 con 0x7ff8e807dde0 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.316+0000 7ff8e7e00700 1 --2- 172.21.15.57:0/1334162345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e807c290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80098a0 tx=0x7ff8d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb50c4ff9a2b1165 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8e6a00700 1 -- 172.21.15.57:0/1334162345 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d8004160 con 0x7ff8e807dde0 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8e6a00700 1 -- 172.21.15.57:0/1334162345 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff8d8004300 con 0x7ff8e807dde0 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8e6a00700 1 -- 172.21.15.57:0/1334162345 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d8011a80 con 0x7ff8e807dde0 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8eea00700 1 -- 172.21.15.57:0/1334162345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 msgr2=0x7ff8e807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8eea00700 1 --2- 172.21.15.57:0/1334162345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e807c290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80098a0 tx=0x7ff8d8009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8eea00700 1 -- 172.21.15.57:0/1334162345 shutdown_connections 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8eea00700 1 --2- 172.21.15.57:0/1334162345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e807c290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.317+0000 7ff8eea00700 1 -- 172.21.15.57:0/1334162345 >> 172.21.15.57:0/1334162345 conn(0x7ff8e8103860 msgr2=0x7ff8e8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.318+0000 7ff8eea00700 1 -- 172.21.15.57:0/1334162345 shutdown_connections 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.318+0000 7ff8eea00700 1 -- 172.21.15.57:0/1334162345 wait complete. 2024-04-13T05:04:26.424 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.318+0000 7ff8eea00700 1 Processor -- start 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.319+0000 7ff8eea00700 1 -- start start 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.319+0000 7ff8eea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.319+0000 7ff8eea00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e810e7d0 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.319+0000 7ff8e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.319+0000 7ff8e7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60540/0 (socket says 172.21.15.57:60540) 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.319+0000 7ff8e7e00700 1 -- 172.21.15.57:0/2668350524 learned_addr learned my addr 172.21.15.57:0/2668350524 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8e7e00700 1 -- 172.21.15.57:0/2668350524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e819e840 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8e7e00700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e819e300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8011a50 tx=0x7ff8d8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d80039f0 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff8d8003b90 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d801a600 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e819ead0 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.320+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e81a17c0 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.321+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e8064110 con 0x7ff8e807dde0 2024-04-13T05:04:26.425 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.321+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff8d8024a40 con 0x7ff8e807dde0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.322+0000 7ff8e4c00700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff8cc039670 0x7ff8cc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.322+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8d804da80 con 0x7ff8e807dde0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.322+0000 7ff8e7400700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff8cc039670 0x7ff8cc03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.323+0000 7ff8e7400700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff8cc039670 0x7ff8cc03bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4006fd0 tx=0x7ff8d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.327+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8d804d4a0 con 0x7ff8e807dde0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.612+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff8e81a1990 con 0x7ff8e807dde0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.665+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff8e81a1990 con 0x7ff8e807dde0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.668+0000 7ff8e4c00700 1 -- 172.21.15.57:0/2668350524 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff8d802d770 con 0x7ff8e807dde0 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff8cc039670 msgr2=0x7ff8cc03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff8cc039670 0x7ff8cc03bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4006fd0 tx=0x7ff8d4006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 msgr2=0x7ff8e819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e819e300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8011a50 tx=0x7ff8d8004290 comp rx=0 tx=0).stop 2024-04-13T05:04:26.426 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 shutdown_connections 2024-04-13T05:04:26.427 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff8cc039670 0x7ff8cc03bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:26.427 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 --2- 172.21.15.57:0/2668350524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8e807dde0 0x7ff8e819e300 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:26.427 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.669+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 >> 172.21.15.57:0/2668350524 conn(0x7ff8e8103860 msgr2=0x7ff8e8104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:26.427 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.670+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 shutdown_connections 2024-04-13T05:04:26.427 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:25.670+0000 7ff8eea00700 1 -- 172.21.15.57:0/2668350524 wait complete. 2024-04-13T05:04:26.672 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:26.478+0000 7f6005885200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T05:04:26.672 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:26.604+0000 7f6005885200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T05:04:26.969 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:26 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2668350524' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-13T05:04:26.969 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:26 smithi057 ceph-mon[32331]: mgrmap e5: x(active, since 5s) 2024-04-13T05:04:27.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:27.442+0000 7f6005885200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T05:04:27.886 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:27 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3811798483' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-13T05:04:27.886 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:27.569+0000 7f6005885200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T05:04:27.886 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:27.756+0000 7f6005885200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T05:04:27.886 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:27.886+0000 7f6005885200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T05:04:28.041 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:04:28.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-13T05:04:28.042 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-13T05:04:28.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-13T05:04:28.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-13T05:04:28.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:04:28.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.019+0000 7fc4ef800700 1 Processor -- start 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.019+0000 7fc4ef800700 1 -- start start 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.020+0000 7fc4ef800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e807c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.020+0000 7fc4ef800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e807c7b0 con 0x7fc4e807ddc0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.020+0000 7fc4ece00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.020+0000 7fc4ece00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e807c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60566/0 (socket says 172.21.15.57:60566) 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.020+0000 7fc4ece00700 1 -- 172.21.15.57:0/1616718855 learned_addr learned my addr 172.21.15.57:0/1616718855 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.021+0000 7fc4ece00700 1 -- 172.21.15.57:0/1616718855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e807c930 con 0x7fc4e807ddc0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.021+0000 7fc4ece00700 1 --2- 172.21.15.57:0/1616718855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e807c270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0098a0 tx=0x7fc4dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fef8c8f30696a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.021+0000 7fc4e7400700 1 -- 172.21.15.57:0/1616718855 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc004160 con 0x7fc4e807ddc0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.021+0000 7fc4e7400700 1 -- 172.21.15.57:0/1616718855 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc4dc004300 con 0x7fc4e807ddc0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.021+0000 7fc4e7400700 1 -- 172.21.15.57:0/1616718855 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc011a80 con 0x7fc4e807ddc0 2024-04-13T05:04:28.045 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.022+0000 7fc4ef800700 1 -- 172.21.15.57:0/1616718855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 msgr2=0x7fc4e807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:28.046 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.022+0000 7fc4ef800700 1 --2- 172.21.15.57:0/1616718855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e807c270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0098a0 tx=0x7fc4dc009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:28.050 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.022+0000 7fc4ef800700 1 -- 172.21.15.57:0/1616718855 shutdown_connections 2024-04-13T05:04:28.050 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.022+0000 7fc4ef800700 1 --2- 172.21.15.57:0/1616718855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e807c270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:28.050 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.022+0000 7fc4ef800700 1 -- 172.21.15.57:0/1616718855 >> 172.21.15.57:0/1616718855 conn(0x7fc4e8103860 msgr2=0x7fc4e8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:28.050 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.023+0000 7fc4ef800700 1 -- 172.21.15.57:0/1616718855 shutdown_connections 2024-04-13T05:04:28.050 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.023+0000 7fc4ef800700 1 -- 172.21.15.57:0/1616718855 wait complete. 2024-04-13T05:04:28.050 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.024+0000 7fc4ef800700 1 Processor -- start 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.024+0000 7fc4ef800700 1 -- start start 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.025+0000 7fc4ef800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.025+0000 7fc4ef800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e810e7f0 con 0x7fc4e807ddc0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.025+0000 7fc4ece00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.025+0000 7fc4ece00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e819e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60578/0 (socket says 172.21.15.57:60578) 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.025+0000 7fc4ece00700 1 -- 172.21.15.57:0/3811798483 learned_addr learned my addr 172.21.15.57:0/3811798483 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4ece00700 1 -- 172.21.15.57:0/3811798483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e819e820 con 0x7fc4e807ddc0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4ece00700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e819e2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc011a50 tx=0x7fc4dc004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc0039f0 con 0x7fc4e807ddc0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc4dc003b90 con 0x7fc4e807ddc0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc01a600 con 0x7fc4e807ddc0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e819eab0 con 0x7fc4e807ddc0 2024-04-13T05:04:28.051 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.026+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e81a17a0 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.028+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc4dc024a40 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.028+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e8064110 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.028+0000 7fc4e5600700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 0x7fc4d003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.028+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4dc04db80 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.028+0000 7fc4e7e00700 1 -- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 msgr2=0x7fc4d003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.028+0000 7fc4e7e00700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 0x7fc4d003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.033+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4dc04d5a0 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.229+0000 7fc4e7e00700 1 -- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 msgr2=0x7fc4d003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.229+0000 7fc4e7e00700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 0x7fc4d003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.318+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc4e81a1970 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.319+0000 7fc4e5600700 1 -- 172.21.15.57:0/3811798483 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fc4e81a1970 con 0x7fc4e807ddc0 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 msgr2=0x7fc4d003bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 0x7fc4d003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 msgr2=0x7fc4e819e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:28.052 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e819e2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc011a50 tx=0x7fc4dc004290 comp rx=0 tx=0).stop 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 shutdown_connections 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7fc4d0039670 0x7fc4d003bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 --2- 172.21.15.57:0/3811798483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4e807ddc0 0x7fc4e819e2e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 >> 172.21.15.57:0/3811798483 conn(0x7fc4e8103860 msgr2=0x7fc4e8104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 shutdown_connections 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:27.322+0000 7fc4ef800700 1 -- 172.21.15.57:0/3811798483 wait complete. 2024-04-13T05:04:28.053 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-04-13T05:04:28.054 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 5... 2024-04-13T05:04:28.201 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:28.002+0000 7f6005885200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T05:04:28.501 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:28.355+0000 7f6005885200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T05:04:28.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:28.502+0000 7f6005885200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T05:04:28.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:28.621+0000 7f6005885200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T05:04:28.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:28.752+0000 7f6005885200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T05:04:29.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:28.997+0000 7f6005885200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T05:04:30.515 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:30.335+0000 7f6005885200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T05:04:30.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:30.516+0000 7f6005885200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T05:04:31.278 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:30.946+0000 7f6005885200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T05:04:31.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:31.070+0000 7f6005885200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T05:04:31.593 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:31.338+0000 7f6005885200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:04:31.989 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:31.593+0000 7f6005885200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T05:04:32.278 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:31.990+0000 7f6005885200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:04:32.634 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:32.315+0000 7f6005885200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T05:04:33.028 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:32.634+0000 7f6005885200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T05:04:34.028 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:33.709+0000 7f6005885200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T05:04:34.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:34.433+0000 7f6005885200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T05:04:34.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:34.567+0000 7f6005885200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:04:34.861+0000 7f6005885200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: Active manager daemon x restarted 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: Activating manager daemon x 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: osdmap e2: 0 total, 0 up, 0 in 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: mgrmap e6: x(active, starting, since 0.0139707s) 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-13T05:04:35.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-13T05:04:35.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-13T05:04:35.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-13T05:04:35.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:34 smithi057 ceph-mon[32331]: Manager daemon x is now available 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: Found migration_current of "None". Setting to last migration. 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-13T05:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:35 smithi057 ceph-mon[32331]: mgrmap e7: x(active, since 1.01755s) 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.697+0000 7ff3eba00700 1 Processor -- start 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.697+0000 7ff3eba00700 1 -- start start 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.698+0000 7ff3eba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e407c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.698+0000 7ff3eba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e407c7b0 con 0x7ff3e407dd70 2024-04-13T05:04:36.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.698+0000 7ff3e9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.698+0000 7ff3e9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e407c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60582/0 (socket says 172.21.15.57:60582) 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.698+0000 7ff3e9000700 1 -- 172.21.15.57:0/335695821 learned_addr learned my addr 172.21.15.57:0/335695821 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.699+0000 7ff3e9000700 1 -- 172.21.15.57:0/335695821 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e407c930 con 0x7ff3e407dd70 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.699+0000 7ff3e9000700 1 --2- 172.21.15.57:0/335695821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e407c270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80098a0 tx=0x7ff3d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bd85c0383d46df server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.699+0000 7ff3e3400700 1 -- 172.21.15.57:0/335695821 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3d8004160 con 0x7ff3e407dd70 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3e3400700 1 -- 172.21.15.57:0/335695821 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff3d8004300 con 0x7ff3e407dd70 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3e3400700 1 -- 172.21.15.57:0/335695821 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3d8011a80 con 0x7ff3e407dd70 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3eba00700 1 -- 172.21.15.57:0/335695821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 msgr2=0x7ff3e407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3eba00700 1 --2- 172.21.15.57:0/335695821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e407c270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80098a0 tx=0x7ff3d8009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3eba00700 1 -- 172.21.15.57:0/335695821 shutdown_connections 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3eba00700 1 --2- 172.21.15.57:0/335695821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e407c270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3eba00700 1 -- 172.21.15.57:0/335695821 >> 172.21.15.57:0/335695821 conn(0x7ff3e4103870 msgr2=0x7ff3e4105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.700+0000 7ff3eba00700 1 -- 172.21.15.57:0/335695821 shutdown_connections 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.701+0000 7ff3eba00700 1 -- 172.21.15.57:0/335695821 wait complete. 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3eba00700 1 Processor -- start 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3eba00700 1 -- start start 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3eba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3eba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410e7e0 con 0x7ff3e407dd70 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3e9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3e9000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e419e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60590/0 (socket says 172.21.15.57:60590) 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.702+0000 7ff3e9000700 1 -- 172.21.15.57:0/3596117408 learned_addr learned my addr 172.21.15.57:0/3596117408 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:36.582 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.703+0000 7ff3e9000700 1 -- 172.21.15.57:0/3596117408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e419e800 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.703+0000 7ff3e9000700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e419e2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006af0 tx=0x7ff3d8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.703+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3d8003930 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.703+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e419ea90 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.704+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff3d8003ad0 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.704+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3d80195e0 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.704+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e41a1780 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.705+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff3d8019910 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.705+0000 7ff3e1600700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.705+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 --> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff3cc03c240 con 0x7ff3cc039670 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.705+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff3d804dbf0 con 0x7ff3e407dd70 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.705+0000 7ff3e3e00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 msgr2=0x7ff3cc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:36.583 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.705+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.906+0000 7ff3e3e00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 msgr2=0x7ff3cc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:28.906+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:29.307+0000 7ff3e3e00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 msgr2=0x7ff3cc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:29.307+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:30.108+0000 7ff3e3e00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 msgr2=0x7ff3cc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:30.108+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:31.710+0000 7ff3e3e00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 msgr2=0x7ff3cc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/570139496 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:31.710+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:34.876+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7ff3d8019c50 con 0x7ff3e407dd70 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:34.876+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 msgr2=0x7ff3cc03bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:34.876+0000 7ff3e1600700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.584 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.876+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7ff3d804f9d0 con 0x7ff3e407dd70 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.876+0000 7ff3e1600700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff3cc03cf60 0x7ff3cc03f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.876+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff3d8008ea0 con 0x7ff3cc03cf60 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.877+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff3cc03cf60 0x7ff3cc03f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.881+0000 7ff3e3e00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff3cc03cf60 0x7ff3cc03f350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0003a10 tx=0x7ff3d00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.882+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff3d8008ea0 con 0x7ff3cc03cf60 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.890+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff3e4064110 con 0x7ff3cc03cf60 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.891+0000 7ff3e1600700 1 -- 172.21.15.57:0/3596117408 <== mgr.14120 v2:172.21.15.57:6800/3670387557 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff3e4064110 con 0x7ff3cc03cf60 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.891+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff3cc03cf60 msgr2=0x7ff3cc03f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.891+0000 7ff3eba00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff3cc03cf60 0x7ff3cc03f350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0003a10 tx=0x7ff3d00092b0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.891+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 msgr2=0x7ff3e419e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.891+0000 7ff3eba00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e419e2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006af0 tx=0x7ff3d8011b70 comp rx=0 tx=0).stop 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 shutdown_connections 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff3cc03cf60 0x7ff3cc03f350 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3e407dd70 0x7ff3e419e2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 --2- 172.21.15.57:0/3596117408 >> [v2:172.21.15.57:6800/570139496,v1:172.21.15.57:6801/570139496] conn(0x7ff3cc039670 0x7ff3cc03bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 >> 172.21.15.57:0/3596117408 conn(0x7ff3e4103870 msgr2=0x7ff3e41042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 shutdown_connections 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:35.892+0000 7ff3eba00700 1 -- 172.21.15.57:0/3596117408 wait complete. 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 5 is available 2024-04-13T05:04:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:Setting orchestrator backend to cephadm... 2024-04-13T05:04:37.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:36 smithi057 ceph-mon[32331]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-13T05:04:37.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:36 smithi057 ceph-mon[32331]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-13T05:04:38.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:37 smithi057 ceph-mon[32331]: mgrmap e8: x(active, since 2s) 2024-04-13T05:04:38.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:37 smithi057 ceph-mon[32331]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:38.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:37 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:38.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:37 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:04:38.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:37 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:38.162 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:37 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:38.173 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.193+0000 7f325b200700 1 Processor -- start 2024-04-13T05:04:38.173 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.193+0000 7f325b200700 1 -- start start 2024-04-13T05:04:38.174 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.193+0000 7f325b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325410b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.193+0000 7f325b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f325407be60 con 0x7f3254108cc0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.194+0000 7f3253e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325410b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.194+0000 7f3253e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325410b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55724/0 (socket says 172.21.15.57:55724) 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.194+0000 7f3253e00700 1 -- 172.21.15.57:0/316573527 learned_addr learned my addr 172.21.15.57:0/316573527 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.194+0000 7f3253e00700 1 -- 172.21.15.57:0/316573527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325410b5f0 con 0x7f3254108cc0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.195+0000 7f3253e00700 1 --2- 172.21.15.57:0/316573527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325410b0b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32440098a0 tx=0x7f3244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781b4ec5b135daae server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.195+0000 7f3252a00700 1 -- 172.21.15.57:0/316573527 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3244004160 con 0x7f3254108cc0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.195+0000 7f3252a00700 1 -- 172.21.15.57:0/316573527 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3244004300 con 0x7f3254108cc0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.195+0000 7f3252a00700 1 -- 172.21.15.57:0/316573527 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3244011a80 con 0x7f3254108cc0 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 -- 172.21.15.57:0/316573527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 msgr2=0x7f325410b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 --2- 172.21.15.57:0/316573527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325410b0b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32440098a0 tx=0x7f3244009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:38.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 -- 172.21.15.57:0/316573527 shutdown_connections 2024-04-13T05:04:38.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 --2- 172.21.15.57:0/316573527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325410b0b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:38.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 -- 172.21.15.57:0/316573527 >> 172.21.15.57:0/316573527 conn(0x7f3254102730 msgr2=0x7f3254104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:38.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 -- 172.21.15.57:0/316573527 shutdown_connections 2024-04-13T05:04:38.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.196+0000 7f325b200700 1 -- 172.21.15.57:0/316573527 wait complete. 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.197+0000 7f325b200700 1 Processor -- start 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.198+0000 7f325b200700 1 -- start start 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.198+0000 7f325b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.198+0000 7f325b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f325410c5c0 con 0x7f3254108cc0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.198+0000 7f3253e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.198+0000 7f3253e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325419c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55728/0 (socket says 172.21.15.57:55728) 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.198+0000 7f3253e00700 1 -- 172.21.15.57:0/2903574613 learned_addr learned my addr 172.21.15.57:0/2903574613 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.199+0000 7f3253e00700 1 -- 172.21.15.57:0/2903574613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325419c640 con 0x7f3254108cc0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.199+0000 7f3253e00700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325419c100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3244011a50 tx=0x7f3244004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.199+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32440039f0 con 0x7f3254108cc0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.199+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3244003b90 con 0x7f3254108cc0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.199+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325419c8d0 con 0x7f3254108cc0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.199+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f324401a600 con 0x7f3254108cc0 2024-04-13T05:04:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.200+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325419f5c0 con 0x7f3254108cc0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.201+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3254064110 con 0x7f3254108cc0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.201+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3244024a40 con 0x7f3254108cc0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.201+0000 7f3250c00700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f32380396c0 0x7f323803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.201+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f324404dbb0 con 0x7f3254108cc0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.202+0000 7f3253400700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f32380396c0 0x7f323803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.202+0000 7f3253400700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f32380396c0 0x7f323803bb80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3240006fd0 tx=0x7f3240006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.207+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32440172b0 con 0x7f3254108cc0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.450+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f32541038b0 con 0x7f32380396c0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.461+0000 7f3250c00700 1 -- 172.21.15.57:0/2903574613 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f32541038b0 con 0x7f32380396c0 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f32380396c0 msgr2=0x7f323803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f32380396c0 0x7f323803bb80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3240006fd0 tx=0x7f3240006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 msgr2=0x7f325419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325419c100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3244011a50 tx=0x7f3244004290 comp rx=0 tx=0).stop 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 shutdown_connections 2024-04-13T05:04:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f32380396c0 0x7f323803bb80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:38.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 --2- 172.21.15.57:0/2903574613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254108cc0 0x7f325419c100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:38.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 >> 172.21.15.57:0/2903574613 conn(0x7f3254102730 msgr2=0x7f32541031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:38.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 shutdown_connections 2024-04-13T05:04:38.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:37.464+0000 7f325b200700 1 -- 172.21.15.57:0/2903574613 wait complete. 2024-04-13T05:04:39.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:38 smithi057 ceph-mon[32331]: [13/Apr/2024:05:04:38] ENGINE Bus STARTING 2024-04-13T05:04:39.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:38 smithi057 ceph-mon[32331]: [13/Apr/2024:05:04:38] ENGINE Serving on http://172.21.15.57:8765 2024-04-13T05:04:39.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:38 smithi057 ceph-mon[32331]: [13/Apr/2024:05:04:38] ENGINE Serving on https://172.21.15.57:7150 2024-04-13T05:04:39.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:38 smithi057 ceph-mon[32331]: [13/Apr/2024:05:04:38] ENGINE Bus STARTED 2024-04-13T05:04:39.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:38 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:04:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-13T05:04:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.780+0000 7f2f63200700 1 Processor -- start 2024-04-13T05:04:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.780+0000 7f2f63200700 1 -- start start 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.781+0000 7f2f63200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f64111240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.781+0000 7f2f63200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64111810 con 0x7f2f6410ee50 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.781+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f64111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.781+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f64111240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55738/0 (socket says 172.21.15.57:55738) 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.781+0000 7f2f61e00700 1 -- 172.21.15.57:0/2634524777 learned_addr learned my addr 172.21.15.57:0/2634524777 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.781+0000 7f2f61e00700 1 -- 172.21.15.57:0/2634524777 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f64111990 con 0x7f2f6410ee50 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.782+0000 7f2f61e00700 1 --2- 172.21.15.57:0/2634524777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f64111240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee18f31d9b556f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.782+0000 7f2f60a00700 1 -- 172.21.15.57:0/2634524777 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f54004160 con 0x7f2f6410ee50 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.782+0000 7f2f60a00700 1 -- 172.21.15.57:0/2634524777 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2f54004300 con 0x7f2f6410ee50 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.782+0000 7f2f60a00700 1 -- 172.21.15.57:0/2634524777 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f54011a80 con 0x7f2f6410ee50 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 -- 172.21.15.57:0/2634524777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 msgr2=0x7f2f64111240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 --2- 172.21.15.57:0/2634524777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f64111240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 -- 172.21.15.57:0/2634524777 shutdown_connections 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 --2- 172.21.15.57:0/2634524777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f64111240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 -- 172.21.15.57:0/2634524777 >> 172.21.15.57:0/2634524777 conn(0x7f2f6407d9a0 msgr2=0x7f2f6407fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 -- 172.21.15.57:0/2634524777 shutdown_connections 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.783+0000 7f2f63200700 1 -- 172.21.15.57:0/2634524777 wait complete. 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.784+0000 7f2f63200700 1 Processor -- start 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.784+0000 7f2f63200700 1 -- start start 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.785+0000 7f2f63200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f641a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.785+0000 7f2f63200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641129b0 con 0x7f2f6410ee50 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.785+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f641a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.785+0000 7f2f61e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f641a24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55740/0 (socket says 172.21.15.57:55740) 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.785+0000 7f2f61e00700 1 -- 172.21.15.57:0/1860145857 learned_addr learned my addr 172.21.15.57:0/1860145857 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.785+0000 7f2f61e00700 1 -- 172.21.15.57:0/1860145857 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f641a29e0 con 0x7f2f6410ee50 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.786+0000 7f2f61e00700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f641a24a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f54000c00 tx=0x7f2f54011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:39.670 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.786+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f540039f0 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.786+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2f54003b90 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.786+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f641a2c70 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.786+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f5401a740 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.786+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f641a5960 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.787+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f64064110 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.787+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2f54024af0 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.788+0000 7f2f5aa00700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2f48039670 0x7f2f4803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.788+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f5404db50 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.788+0000 7f2f61400700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2f48039670 0x7f2f4803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.789+0000 7f2f61400700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2f48039670 0x7f2f4803bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c006fd0 tx=0x7f2f4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:38.795+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f5400f2e0 con 0x7f2f6410ee50 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.027+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2f6407fae0 con 0x7f2f48039670 2024-04-13T05:04:39.671 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.029+0000 7f2f5aa00700 1 -- 172.21.15.57:0/1860145857 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2f6407fae0 con 0x7f2f48039670 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2f48039670 msgr2=0x7f2f4803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2f48039670 0x7f2f4803bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c006fd0 tx=0x7f2f4c006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 msgr2=0x7f2f641a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f641a24a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f54000c00 tx=0x7f2f54011b00 comp rx=0 tx=0).stop 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 shutdown_connections 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2f48039670 0x7f2f4803bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 --2- 172.21.15.57:0/1860145857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f6410ee50 0x7f2f641a24a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 >> 172.21.15.57:0/1860145857 conn(0x7f2f6407d9a0 msgr2=0x7f2f6407f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.032+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 shutdown_connections 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:39.033+0000 7f2f63200700 1 -- 172.21.15.57:0/1860145857 wait complete. 2024-04-13T05:04:39.672 INFO:teuthology.orchestra.run.smithi057.stdout:Generating ssh key... 2024-04-13T05:04:40.264 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:39 smithi057 ceph-mon[32331]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:41.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:40 smithi057 ceph-mon[32331]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:41.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:40 smithi057 ceph-mon[32331]: Generating ssh key... 2024-04-13T05:04:41.682 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: Generating public/private rsa key pair. 2024-04-13T05:04:41.682 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: Your identification has been saved in /tmp/tmpbsytyfuk/key. 2024-04-13T05:04:41.682 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: Your public key has been saved in /tmp/tmpbsytyfuk/key.pub. 2024-04-13T05:04:41.682 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: The key fingerprint is: 2024-04-13T05:04:41.682 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: SHA256:XXI/d9AYeuaQWbX4eZ+RciHSKveAwp8Fvg2Ls4bAQMg ceph-0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:41.682 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: The key's randomart image is: 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: +---[RSA 3072]----+ 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: |o o..| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: |.E .=.+.| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: |. ...B+=o.| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: | . . ..o+o*o.+| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: | o oS=.= .+*+| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: | o + X o oo*| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: | . .o = . . ..| 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: | . .o | 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: | .. | 2024-04-13T05:04:41.683 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:04:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: +----[SHA256]-----+ 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.265+0000 7f0bbb200700 1 Processor -- start 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.265+0000 7f0bbb200700 1 -- start start 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.266+0000 7f0bbb200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.266+0000 7f0bbb200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb407c7f0 con 0x7f0bb407dd40 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.266+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.266+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb407c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55744/0 (socket says 172.21.15.57:55744) 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.266+0000 7f0bb3e00700 1 -- 172.21.15.57:0/2499516994 learned_addr learned my addr 172.21.15.57:0/2499516994 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.267+0000 7f0bb3e00700 1 -- 172.21.15.57:0/2499516994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb407c970 con 0x7f0bb407dd40 2024-04-13T05:04:42.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.267+0000 7f0bb3e00700 1 --2- 172.21.15.57:0/2499516994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb407c2b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40098a0 tx=0x7f0ba4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ac1ebed57d0b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.267+0000 7f0bb2a00700 1 -- 172.21.15.57:0/2499516994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba4004160 con 0x7f0bb407dd40 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.267+0000 7f0bb2a00700 1 -- 172.21.15.57:0/2499516994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0ba4004300 con 0x7f0bb407dd40 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.267+0000 7f0bb2a00700 1 -- 172.21.15.57:0/2499516994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba4011a80 con 0x7f0bb407dd40 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 -- 172.21.15.57:0/2499516994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 msgr2=0x7f0bb407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 --2- 172.21.15.57:0/2499516994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb407c2b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40098a0 tx=0x7f0ba4009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 -- 172.21.15.57:0/2499516994 shutdown_connections 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 --2- 172.21.15.57:0/2499516994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb407c2b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 -- 172.21.15.57:0/2499516994 >> 172.21.15.57:0/2499516994 conn(0x7f0bb41038a0 msgr2=0x7f0bb4105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 -- 172.21.15.57:0/2499516994 shutdown_connections 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.268+0000 7f0bbb200700 1 -- 172.21.15.57:0/2499516994 wait complete. 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.269+0000 7f0bbb200700 1 Processor -- start 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.269+0000 7f0bbb200700 1 -- start start 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.270+0000 7f0bbb200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:42.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.270+0000 7f0bbb200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb410e6d0 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.270+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.270+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb419e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55752/0 (socket says 172.21.15.57:55752) 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.270+0000 7f0bb3e00700 1 -- 172.21.15.57:0/10429094 learned_addr learned my addr 172.21.15.57:0/10429094 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.270+0000 7f0bb3e00700 1 -- 172.21.15.57:0/10429094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb419e850 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.271+0000 7f0bb3e00700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb419e310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4006af0 tx=0x7f0ba4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.271+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba4003930 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.271+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0ba4003ad0 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.271+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb419eae0 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.271+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba40195e0 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.271+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb41a17d0 con 0x7f0bb407dd40 2024-04-13T05:04:42.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.272+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb4064110 con 0x7f0bb407dd40 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.273+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0ba40198e0 con 0x7f0bb407dd40 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.273+0000 7f0bb0c00700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f0b98039620 0x7f0b9803bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.273+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0ba404dce0 con 0x7f0bb407dd40 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.273+0000 7f0bb3400700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f0b98039620 0x7f0b9803bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.274+0000 7f0bb3400700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f0b98039620 0x7f0b9803bae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0006fd0 tx=0x7f0ba0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.278+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ba4023e00 con 0x7f0bb407dd40 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:40.506+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0bb4104ad0 con 0x7f0b98039620 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.314+0000 7f0bb0c00700 1 -- 172.21.15.57:0/10429094 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0bb4104ad0 con 0x7f0b98039620 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f0b98039620 msgr2=0x7f0b9803bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f0b98039620 0x7f0b9803bae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0006fd0 tx=0x7f0ba0006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 msgr2=0x7f0bb419e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb419e310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4006af0 tx=0x7f0ba4011b70 comp rx=0 tx=0).stop 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 shutdown_connections 2024-04-13T05:04:42.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f0b98039620 0x7f0b9803bae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:42.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 --2- 172.21.15.57:0/10429094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bb407dd40 0x7f0bb419e310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:42.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.316+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 >> 172.21.15.57:0/10429094 conn(0x7f0bb41038a0 msgr2=0x7f0bb41043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:42.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.317+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 shutdown_connections 2024-04-13T05:04:42.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:41.317+0000 7f0bbb200700 1 -- 172.21.15.57:0/10429094 wait complete. 2024-04-13T05:04:42.562 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:42 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:42.563 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:42 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:43.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:04:43.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.616+0000 7f9b63c00700 1 Processor -- start 2024-04-13T05:04:43.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.616+0000 7f9b63c00700 1 -- start start 2024-04-13T05:04:43.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.616+0000 7f9b63c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:43.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.616+0000 7f9b63c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c07c880 con 0x7f9b5c07de00 2024-04-13T05:04:43.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.617+0000 7f9b61200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:43.513 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.617+0000 7f9b61200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55762/0 (socket says 172.21.15.57:55762) 2024-04-13T05:04:43.513 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.617+0000 7f9b61200700 1 -- 172.21.15.57:0/1411024593 learned_addr learned my addr 172.21.15.57:0/1411024593 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:43.513 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.617+0000 7f9b61200700 1 -- 172.21.15.57:0/1411024593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c07ca00 con 0x7f9b5c07de00 2024-04-13T05:04:43.513 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.618+0000 7f9b61200700 1 --2- 172.21.15.57:0/1411024593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c07c2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b500098a0 tx=0x7f9b50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b379ebe9450605 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:43.513 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.618+0000 7f9b5b400700 1 -- 172.21.15.57:0/1411024593 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50004160 con 0x7f9b5c07de00 2024-04-13T05:04:43.513 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.618+0000 7f9b5b400700 1 -- 172.21.15.57:0/1411024593 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b50004300 con 0x7f9b5c07de00 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.618+0000 7f9b5b400700 1 -- 172.21.15.57:0/1411024593 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50011a80 con 0x7f9b5c07de00 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 -- 172.21.15.57:0/1411024593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 msgr2=0x7f9b5c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 --2- 172.21.15.57:0/1411024593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c07c2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b500098a0 tx=0x7f9b50009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 -- 172.21.15.57:0/1411024593 shutdown_connections 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 --2- 172.21.15.57:0/1411024593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c07c2b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 -- 172.21.15.57:0/1411024593 >> 172.21.15.57:0/1411024593 conn(0x7f9b5c103860 msgr2=0x7f9b5c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 -- 172.21.15.57:0/1411024593 shutdown_connections 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.619+0000 7f9b63c00700 1 -- 172.21.15.57:0/1411024593 wait complete. 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.620+0000 7f9b63c00700 1 Processor -- start 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.620+0000 7f9b63c00700 1 -- start start 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.620+0000 7f9b63c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c10b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.620+0000 7f9b63c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10e7b0 con 0x7f9b5c07de00 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.621+0000 7f9b61200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c10b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:43.514 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.621+0000 7f9b61200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c10b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55770/0 (socket says 172.21.15.57:55770) 2024-04-13T05:04:43.515 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.621+0000 7f9b61200700 1 -- 172.21.15.57:0/4095728239 learned_addr learned my addr 172.21.15.57:0/4095728239 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:43.515 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.621+0000 7f9b61200700 1 -- 172.21.15.57:0/4095728239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c10ba30 con 0x7f9b5c07de00 2024-04-13T05:04:43.517 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.621+0000 7f9b61200700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c10b4f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b50011a50 tx=0x7f9b50004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:43.517 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.621+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b500039f0 con 0x7f9b5c07de00 2024-04-13T05:04:43.517 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.622+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b50003b90 con 0x7f9b5c07de00 2024-04-13T05:04:43.517 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.622+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b5001a600 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.622+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c10bcc0 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.622+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c108280 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.623+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9b50024a40 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.623+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c10be90 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.623+0000 7f9b59600700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f9b440396c0 0x7f9b4403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.623+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b5004dbe0 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.624+0000 7f9b5be00700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f9b440396c0 0x7f9b4403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.624+0000 7f9b5be00700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f9b440396c0 0x7f9b4403bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006fd0 tx=0x7f9b48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.629+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b5c10be90 con 0x7f9b5c07de00 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.859+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9b5c1089d0 con 0x7f9b440396c0 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.860+0000 7f9b59600700 1 -- 172.21.15.57:0/4095728239 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9b5c1089d0 con 0x7f9b440396c0 2024-04-13T05:04:43.518 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f9b440396c0 msgr2=0x7f9b4403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f9b440396c0 0x7f9b4403bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006fd0 tx=0x7f9b48006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 msgr2=0x7f9b5c10b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c10b4f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b50011a50 tx=0x7f9b50004290 comp rx=0 tx=0).stop 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 shutdown_connections 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f9b440396c0 0x7f9b4403bb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 --2- 172.21.15.57:0/4095728239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b5c07de00 0x7f9b5c10b4f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.863+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 >> 172.21.15.57:0/4095728239 conn(0x7f9b5c103860 msgr2=0x7f9b5c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.864+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 shutdown_connections 2024-04-13T05:04:43.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:42.864+0000 7f9b63c00700 1 -- 172.21.15.57:0/4095728239 wait complete. 2024-04-13T05:04:43.520 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-13T05:04:43.520 INFO:teuthology.orchestra.run.smithi057.stdout:Adding key to root@localhost authorized_keys... 2024-04-13T05:04:43.520 INFO:teuthology.orchestra.run.smithi057.stdout:Adding host smithi057... 2024-04-13T05:04:44.115 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:43 smithi057 ceph-mon[32331]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:45.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:44 smithi057 ceph-mon[32331]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:47.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:46 smithi057 ceph-mon[32331]: Deploying cephadm binary to smithi057 2024-04-13T05:04:52.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Added host 'smithi057' with addr '172.21.15.57' 2024-04-13T05:04:52.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.116+0000 7f05fa800700 1 Processor -- start 2024-04-13T05:04:52.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.116+0000 7f05fa800700 1 -- start start 2024-04-13T05:04:52.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.117+0000 7f05fa800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f410a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:52.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.117+0000 7f05fa800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410ae40 con 0x7f05f410a450 2024-04-13T05:04:52.128 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.117+0000 7f05f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f410a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:52.129 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.117+0000 7f05f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f410a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55780/0 (socket says 172.21.15.57:55780) 2024-04-13T05:04:52.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.117+0000 7f05f3400700 1 -- 172.21.15.57:0/1020706592 learned_addr learned my addr 172.21.15.57:0/1020706592 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:52.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.118+0000 7f05f3400700 1 -- 172.21.15.57:0/1020706592 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f410b600 con 0x7f05f410a450 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.118+0000 7f05f3400700 1 --2- 172.21.15.57:0/1020706592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f410a870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05e40098a0 tx=0x7f05e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5846c779bcb3803 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.118+0000 7f05f2000700 1 -- 172.21.15.57:0/1020706592 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4004160 con 0x7f05f410a450 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05f2000700 1 -- 172.21.15.57:0/1020706592 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f05e4004300 con 0x7f05f410a450 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05f2000700 1 -- 172.21.15.57:0/1020706592 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4011a80 con 0x7f05f410a450 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05fa800700 1 -- 172.21.15.57:0/1020706592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 msgr2=0x7f05f410a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05fa800700 1 --2- 172.21.15.57:0/1020706592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f410a870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05e40098a0 tx=0x7f05e4009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05fa800700 1 -- 172.21.15.57:0/1020706592 shutdown_connections 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05fa800700 1 --2- 172.21.15.57:0/1020706592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f410a870 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05fa800700 1 -- 172.21.15.57:0/1020706592 >> 172.21.15.57:0/1020706592 conn(0x7f05f407db30 msgr2=0x7f05f4108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.119+0000 7f05fa800700 1 -- 172.21.15.57:0/1020706592 shutdown_connections 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.120+0000 7f05fa800700 1 -- 172.21.15.57:0/1020706592 wait complete. 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.121+0000 7f05fa800700 1 Processor -- start 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.121+0000 7f05fa800700 1 -- start start 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.121+0000 7f05fa800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:52.132 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.121+0000 7f05fa800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410e9b0 con 0x7f05f410a450 2024-04-13T05:04:52.133 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.121+0000 7f05f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:52.133 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.122+0000 7f05f3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55796/0 (socket says 172.21.15.57:55796) 2024-04-13T05:04:52.133 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.122+0000 7f05f3400700 1 -- 172.21.15.57:0/3873080576 learned_addr learned my addr 172.21.15.57:0/3873080576 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:52.133 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.122+0000 7f05f3400700 1 -- 172.21.15.57:0/3873080576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419e7e0 con 0x7f05f410a450 2024-04-13T05:04:52.133 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.122+0000 7f05f3400700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f419e2a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006af0 tx=0x7f05e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:52.133 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.122+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4003930 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.123+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f05e4003ad0 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.123+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e40195e0 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.123+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f419ea70 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.123+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f41a1760 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.124+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f05e4019890 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.124+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f4064110 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.124+0000 7f05ebe00700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f05d8039670 0x7f05d803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.125+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05e404db90 con 0x7f05f410a450 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.125+0000 7f05f2a00700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f05d8039670 0x7f05d803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.125+0000 7f05f2a00700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f05d8039670 0x7f05d803bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006fd0 tx=0x7f05e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:52.135 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.130+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e400fda0 con 0x7f05f410a450 2024-04-13T05:04:52.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:44.365+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}) -- 0x7f05f4108230 con 0x7f05d8039670 2024-04-13T05:04:52.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.220+0000 7f05ebe00700 1 -- 172.21.15.57:0/3873080576 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f05f4108230 con 0x7f05d8039670 2024-04-13T05:04:52.136 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.224+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f05d8039670 msgr2=0x7f05d803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:52.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f05d8039670 0x7f05d803bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006fd0 tx=0x7f05e0006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:52.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 msgr2=0x7f05f419e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:52.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f419e2a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006af0 tx=0x7f05e4011b70 comp rx=0 tx=0).stop 2024-04-13T05:04:52.137 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 shutdown_connections 2024-04-13T05:04:52.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f05d8039670 0x7f05d803bb30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:52.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 --2- 172.21.15.57:0/3873080576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f410a450 0x7f05f419e2a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:52.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.225+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 >> 172.21.15.57:0/3873080576 conn(0x7f05f407db30 msgr2=0x7f05f4107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:52.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.226+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 shutdown_connections 2024-04-13T05:04:52.138 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:51.226+0000 7f05fa800700 1 -- 172.21.15.57:0/3873080576 wait complete. 2024-04-13T05:04:52.138 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mon service... 2024-04-13T05:04:52.464 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:52 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:52.464 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:52 smithi057 ceph-mon[32331]: Added host smithi057 2024-04-13T05:04:52.464 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:52 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:04:54.405 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:54 smithi057 ceph-mon[32331]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:54.405 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:54 smithi057 ceph-mon[32331]: Saving service mon spec with placement count:5 2024-04-13T05:04:54.405 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:54 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.820+0000 7fd74ae00700 1 Processor -- start 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.821+0000 7fd74ae00700 1 -- start start 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.821+0000 7fd74ae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd744111240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.821+0000 7fd74ae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd744111810 con 0x7fd74410ee50 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.822+0000 7fd749a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd744111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.822+0000 7fd749a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd744111240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57080/0 (socket says 172.21.15.57:57080) 2024-04-13T05:04:54.719 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.822+0000 7fd749a00700 1 -- 172.21.15.57:0/3062229975 learned_addr learned my addr 172.21.15.57:0/3062229975 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.822+0000 7fd749a00700 1 -- 172.21.15.57:0/3062229975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd744111990 con 0x7fd74410ee50 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.822+0000 7fd749a00700 1 --2- 172.21.15.57:0/3062229975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd744111240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7380098a0 tx=0x7fd738009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3486677bf7273c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.823+0000 7fd743e00700 1 -- 172.21.15.57:0/3062229975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd738004160 con 0x7fd74410ee50 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.823+0000 7fd743e00700 1 -- 172.21.15.57:0/3062229975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd738004300 con 0x7fd74410ee50 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.823+0000 7fd743e00700 1 -- 172.21.15.57:0/3062229975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd738011a80 con 0x7fd74410ee50 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.823+0000 7fd74ae00700 1 -- 172.21.15.57:0/3062229975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 msgr2=0x7fd744111240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.823+0000 7fd74ae00700 1 --2- 172.21.15.57:0/3062229975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd744111240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7380098a0 tx=0x7fd738009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.824+0000 7fd74ae00700 1 -- 172.21.15.57:0/3062229975 shutdown_connections 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.824+0000 7fd74ae00700 1 --2- 172.21.15.57:0/3062229975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd744111240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.824+0000 7fd74ae00700 1 -- 172.21.15.57:0/3062229975 >> 172.21.15.57:0/3062229975 conn(0x7fd74407d6a0 msgr2=0x7fd74407fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.824+0000 7fd74ae00700 1 -- 172.21.15.57:0/3062229975 shutdown_connections 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.824+0000 7fd74ae00700 1 -- 172.21.15.57:0/3062229975 wait complete. 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.825+0000 7fd74ae00700 1 Processor -- start 2024-04-13T05:04:54.720 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.825+0000 7fd74ae00700 1 -- start start 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.825+0000 7fd74ae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd7441a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.825+0000 7fd74ae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd744112a10 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.826+0000 7fd749a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd7441a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.826+0000 7fd749a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd7441a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57096/0 (socket says 172.21.15.57:57096) 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.826+0000 7fd749a00700 1 -- 172.21.15.57:0/680084535 learned_addr learned my addr 172.21.15.57:0/680084535 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.826+0000 7fd749a00700 1 -- 172.21.15.57:0/680084535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441a2a80 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.826+0000 7fd749a00700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd7441a2540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd738000c00 tx=0x7fd738004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.827+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7380039c0 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.827+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd738003b60 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.827+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd73801a6f0 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.827+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7441a2d10 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.827+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441a5a00 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.828+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd738024a40 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.828+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd744064110 con 0x7fd74410ee50 2024-04-13T05:04:54.721 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.829+0000 7fd742000700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7fd72c0396c0 0x7fd72c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.829+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd73804daa0 con 0x7fd74410ee50 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.829+0000 7fd749000700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7fd72c0396c0 0x7fd72c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.830+0000 7fd749000700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7fd72c0396c0 0x7fd72c03bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd730006fd0 tx=0x7fd730006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:52.835+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd73800f2e0 con 0x7fd74410ee50 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.091+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd74407e830 con 0x7fd72c0396c0 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.100+0000 7fd742000700 1 -- 172.21.15.57:0/680084535 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd74407e830 con 0x7fd72c0396c0 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7fd72c0396c0 msgr2=0x7fd72c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7fd72c0396c0 0x7fd72c03bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd730006fd0 tx=0x7fd730006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 msgr2=0x7fd7441a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd7441a2540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd738000c00 tx=0x7fd738004290 comp rx=0 tx=0).stop 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 shutdown_connections 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7fd72c0396c0 0x7fd72c03bb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:54.722 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 --2- 172.21.15.57:0/680084535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd74410ee50 0x7fd7441a2540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:54.723 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 >> 172.21.15.57:0/680084535 conn(0x7fd74407d6a0 msgr2=0x7fd74407e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:54.723 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 shutdown_connections 2024-04-13T05:04:54.723 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:53.103+0000 7fd74ae00700 1 -- 172.21.15.57:0/680084535 wait complete. 2024-04-13T05:04:54.723 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mgr service... 2024-04-13T05:04:55.596 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:55 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:55.596 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:55 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.326+0000 7f2c4a400700 1 Processor -- start 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.326+0000 7f2c4a400700 1 -- start start 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.326+0000 7f2c4a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c44108680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.326+0000 7f2c4a400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44108c50 con 0x7f2c44108260 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.327+0000 7f2c43400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c44108680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.327+0000 7f2c43400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c44108680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38846/0 (socket says 172.21.15.57:38846) 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.327+0000 7f2c43400700 1 -- 172.21.15.57:0/76024962 learned_addr learned my addr 172.21.15.57:0/76024962 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:56.328 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.328+0000 7f2c43400700 1 -- 172.21.15.57:0/76024962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c44109410 con 0x7f2c44108260 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.328+0000 7f2c43400700 1 --2- 172.21.15.57:0/76024962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c44108680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e4261f6b788b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.328+0000 7f2c42000700 1 -- 172.21.15.57:0/76024962 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34004160 con 0x7f2c44108260 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.328+0000 7f2c42000700 1 -- 172.21.15.57:0/76024962 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c34004300 con 0x7f2c44108260 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.328+0000 7f2c42000700 1 -- 172.21.15.57:0/76024962 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34011a50 con 0x7f2c44108260 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.329+0000 7f2c4a400700 1 -- 172.21.15.57:0/76024962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 msgr2=0x7f2c44108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.329+0000 7f2c4a400700 1 --2- 172.21.15.57:0/76024962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c44108680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).stop 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.329+0000 7f2c4a400700 1 -- 172.21.15.57:0/76024962 shutdown_connections 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.329+0000 7f2c4a400700 1 --2- 172.21.15.57:0/76024962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c44108680 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.329+0000 7f2c4a400700 1 -- 172.21.15.57:0/76024962 >> 172.21.15.57:0/76024962 conn(0x7f2c44103860 msgr2=0x7f2c44105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.330+0000 7f2c4a400700 1 -- 172.21.15.57:0/76024962 shutdown_connections 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.330+0000 7f2c4a400700 1 -- 172.21.15.57:0/76024962 wait complete. 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.331+0000 7f2c4a400700 1 Processor -- start 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.331+0000 7f2c4a400700 1 -- start start 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.332+0000 7f2c4a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c4407db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:56.329 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.332+0000 7f2c4a400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410c7c0 con 0x7f2c44108260 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.332+0000 7f2c43400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c4407db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.332+0000 7f2c43400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c4407db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38862/0 (socket says 172.21.15.57:38862) 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.332+0000 7f2c43400700 1 -- 172.21.15.57:0/180401460 learned_addr learned my addr 172.21.15.57:0/180401460 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.332+0000 7f2c43400700 1 -- 172.21.15.57:0/180401460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4407e060 con 0x7f2c44108260 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.333+0000 7f2c43400700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c4407db20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011a20 tx=0x7f2c34011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.333+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34003af0 con 0x7f2c44108260 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.333+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c3401a8e0 con 0x7f2c44108260 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.333+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34024860 con 0x7f2c44108260 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.333+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4407c250 con 0x7f2c44108260 2024-04-13T05:04:56.330 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.333+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4407c790 con 0x7f2c44108260 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.335+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2c34022020 con 0x7f2c44108260 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.335+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c44064110 con 0x7f2c44108260 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.335+0000 7f2c3be00700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2c280396c0 0x7f2c2803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.335+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c3404da60 con 0x7f2c44108260 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.335+0000 7f2c42a00700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2c280396c0 0x7f2c2803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.336+0000 7f2c42a00700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2c280396c0 0x7f2c2803bb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c30006fd0 tx=0x7f2c30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.342+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c3400f2e0 con 0x7f2c44108260 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.596+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2c4407d0a0 con 0x7f2c280396c0 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.604+0000 7f2c3be00700 1 -- 172.21.15.57:0/180401460 <== mgr.14120 v2:172.21.15.57:6800/3670387557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2c4407d0a0 con 0x7f2c280396c0 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.607+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2c280396c0 msgr2=0x7f2c2803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.607+0000 7f2c4a400700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2c280396c0 0x7f2c2803bb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c30006fd0 tx=0x7f2c30006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.607+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 msgr2=0x7f2c4407db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:56.331 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.607+0000 7f2c4a400700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c4407db20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011a20 tx=0x7f2c34011b00 comp rx=0 tx=0).stop 2024-04-13T05:04:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.608+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 shutdown_connections 2024-04-13T05:04:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.608+0000 7f2c4a400700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f2c280396c0 0x7f2c2803bb80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.608+0000 7f2c4a400700 1 --2- 172.21.15.57:0/180401460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c44108260 0x7f2c4407db20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.608+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 >> 172.21.15.57:0/180401460 conn(0x7f2c44103860 msgr2=0x7f2c44104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.608+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 shutdown_connections 2024-04-13T05:04:56.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:55.608+0000 7f2c4a400700 1 -- 172.21.15.57:0/180401460 wait complete. 2024-04-13T05:04:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:56 smithi057 ceph-mon[32331]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:04:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:56 smithi057 ceph-mon[32331]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:04:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:56 smithi057 ceph-mon[32331]: Saving service mgr spec with placement count:2 2024-04-13T05:04:56.924 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:56 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:04:58.037 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.924+0000 7f4a2be00700 1 Processor -- start 2024-04-13T05:04:58.038 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.924+0000 7f4a2be00700 1 -- start start 2024-04-13T05:04:58.038 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.925+0000 7f4a2be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:58.038 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.925+0000 7f4a2be00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2407c8a0 con 0x7f4a2407de20 2024-04-13T05:04:58.039 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.925+0000 7f4a29400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.926+0000 7f4a29400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38866/0 (socket says 172.21.15.57:38866) 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.926+0000 7f4a29400700 1 -- 172.21.15.57:0/3544979627 learned_addr learned my addr 172.21.15.57:0/3544979627 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.926+0000 7f4a29400700 1 -- 172.21.15.57:0/3544979627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2407ca20 con 0x7f4a2407de20 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.926+0000 7f4a29400700 1 --2- 172.21.15.57:0/3544979627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2407c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009e80 tx=0x7f4a1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e2e77d063949f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.927+0000 7f4a23e00700 1 -- 172.21.15.57:0/3544979627 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18017070 con 0x7f4a2407de20 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.927+0000 7f4a23e00700 1 -- 172.21.15.57:0/3544979627 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a1800b8c0 con 0x7f4a2407de20 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.927+0000 7f4a23e00700 1 -- 172.21.15.57:0/3544979627 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18004490 con 0x7f4a2407de20 2024-04-13T05:04:58.040 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.927+0000 7f4a2be00700 1 -- 172.21.15.57:0/3544979627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 msgr2=0x7f4a2407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.927+0000 7f4a2be00700 1 --2- 172.21.15.57:0/3544979627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2407c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009e80 tx=0x7f4a1800b330 comp rx=0 tx=0).stop 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.928+0000 7f4a2be00700 1 -- 172.21.15.57:0/3544979627 shutdown_connections 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.928+0000 7f4a2be00700 1 --2- 172.21.15.57:0/3544979627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2407c2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.928+0000 7f4a2be00700 1 -- 172.21.15.57:0/3544979627 >> 172.21.15.57:0/3544979627 conn(0x7f4a24103880 msgr2=0x7f4a24105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.928+0000 7f4a2be00700 1 -- 172.21.15.57:0/3544979627 shutdown_connections 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.928+0000 7f4a2be00700 1 -- 172.21.15.57:0/3544979627 wait complete. 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.929+0000 7f4a2be00700 1 Processor -- start 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.929+0000 7f4a2be00700 1 -- start start 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.930+0000 7f4a2be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.930+0000 7f4a2be00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2410e800 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.930+0000 7f4a29400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.930+0000 7f4a29400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38872/0 (socket says 172.21.15.57:38872) 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.930+0000 7f4a29400700 1 -- 172.21.15.57:0/1552529890 learned_addr learned my addr 172.21.15.57:0/1552529890 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.931+0000 7f4a29400700 1 -- 172.21.15.57:0/1552529890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2419e7b0 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.931+0000 7f4a29400700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2419e270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a18000f80 tx=0x7f4a18003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.931+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18020020 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.931+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a1800bba0 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.932+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2419ea40 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.932+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18017040 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.932+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a241a1730 con 0x7f4a2407de20 2024-04-13T05:04:58.044 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.933+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4a18023020 con 0x7f4a2407de20 2024-04-13T05:04:58.047 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.933+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a24064110 con 0x7f4a2407de20 2024-04-13T05:04:58.047 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.934+0000 7f4a22000700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f4a0c0396c0 0x7f4a0c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:04:58.047 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.934+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a1800f050 con 0x7f4a2407de20 2024-04-13T05:04:58.047 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.934+0000 7f4a28a00700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f4a0c0396c0 0x7f4a0c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.935+0000 7f4a28a00700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f4a0c0396c0 0x7f4a0c03bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a10006fd0 tx=0x7f4a10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:56.942+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a1802a020 con 0x7f4a2407de20 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.178+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4a241a1d00 con 0x7f4a2407de20 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.184+0000 7f4a22000700 1 -- 172.21.15.57:0/1552529890 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4a1802d350 con 0x7f4a2407de20 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.186+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f4a0c0396c0 msgr2=0x7f4a0c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.186+0000 7f4a2be00700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f4a0c0396c0 0x7f4a0c03bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a10006fd0 tx=0x7f4a10006e40 comp rx=0 tx=0).stop 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.186+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 msgr2=0x7f4a2419e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.186+0000 7f4a2be00700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2419e270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a18000f80 tx=0x7f4a18003d70 comp rx=0 tx=0).stop 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.187+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 shutdown_connections 2024-04-13T05:04:58.048 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.187+0000 7f4a2be00700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f4a0c0396c0 0x7f4a0c03bb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:58.049 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.187+0000 7f4a2be00700 1 --2- 172.21.15.57:0/1552529890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a2407de20 0x7f4a2419e270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:04:58.049 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.187+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 >> 172.21.15.57:0/1552529890 conn(0x7f4a24103880 msgr2=0x7f4a24104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:04:58.049 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.187+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 shutdown_connections 2024-04-13T05:04:58.049 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:57.187+0000 7f4a2be00700 1 -- 172.21.15.57:0/1552529890 wait complete. 2024-04-13T05:04:58.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:58 smithi057 ceph-mon[32331]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:04:58.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1552529890' entity='client.admin' 2024-04-13T05:05:00.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:59 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:05:00.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:59 smithi057 ceph-mon[32331]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:00.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:04:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3381563386' entity='client.admin' 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.124+0000 7f8b93a00700 1 Processor -- start 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.124+0000 7f8b93a00700 1 -- start start 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.125+0000 7f8b93a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.125+0000 7f8b93a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c07c7f0 con 0x7f8b8c07dd40 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.125+0000 7f8b91000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.125+0000 7f8b91000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38888/0 (socket says 172.21.15.57:38888) 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.125+0000 7f8b91000700 1 -- 172.21.15.57:0/2878052418 learned_addr learned my addr 172.21.15.57:0/2878052418 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:00.125 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.126+0000 7f8b91000700 1 -- 172.21.15.57:0/2878052418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c07c970 con 0x7f8b8c07dd40 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.126+0000 7f8b91000700 1 --2- 172.21.15.57:0/2878052418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c07c2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b800098a0 tx=0x7f8b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67229c8c861ed826 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.126+0000 7f8b8b400700 1 -- 172.21.15.57:0/2878052418 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80004160 con 0x7f8b8c07dd40 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.126+0000 7f8b8b400700 1 -- 172.21.15.57:0/2878052418 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b80004300 con 0x7f8b8c07dd40 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.126+0000 7f8b8b400700 1 -- 172.21.15.57:0/2878052418 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80011a50 con 0x7f8b8c07dd40 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 -- 172.21.15.57:0/2878052418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 msgr2=0x7f8b8c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 --2- 172.21.15.57:0/2878052418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c07c2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b800098a0 tx=0x7f8b80009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 -- 172.21.15.57:0/2878052418 shutdown_connections 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 --2- 172.21.15.57:0/2878052418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c07c2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 -- 172.21.15.57:0/2878052418 >> 172.21.15.57:0/2878052418 conn(0x7f8b8c103940 msgr2=0x7f8b8c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 -- 172.21.15.57:0/2878052418 shutdown_connections 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.127+0000 7f8b93a00700 1 -- 172.21.15.57:0/2878052418 wait complete. 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.128+0000 7f8b93a00700 1 Processor -- start 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.128+0000 7f8b93a00700 1 -- start start 2024-04-13T05:05:00.126 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.129+0000 7f8b93a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:00.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.129+0000 7f8b93a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c10e710 con 0x7f8b8c07dd40 2024-04-13T05:05:00.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.129+0000 7f8b91000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:00.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.129+0000 7f8b91000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c19e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38894/0 (socket says 172.21.15.57:38894) 2024-04-13T05:05:00.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.129+0000 7f8b91000700 1 -- 172.21.15.57:0/3381563386 learned_addr learned my addr 172.21.15.57:0/3381563386 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:00.127 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b91000700 1 -- 172.21.15.57:0/3381563386 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c19e770 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b91000700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c19e230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b80011a00 tx=0x7f8b80011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80003af0 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8001a8e0 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b80024840 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c19ea00 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.130+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c1a16f0 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.132+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c064110 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.132+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8b80022020 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.132+0000 7f8b89600700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f8b740396c0 0x7f8b7403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.132+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8b8004d990 con 0x7f8b8c07dd40 2024-04-13T05:05:00.130 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.133+0000 7f8b8be00700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f8b740396c0 0x7f8b7403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.133+0000 7f8b8be00700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f8b740396c0 0x7f8b7403bb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b78006fd0 tx=0x7f8b78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.138+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b80016260 con 0x7f8b8c07dd40 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.374+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f8b8c1a18c0 con 0x7f8b8c07dd40 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.380+0000 7f8b89600700 1 -- 172.21.15.57:0/3381563386 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f8b8002dde0 con 0x7f8b8c07dd40 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f8b740396c0 msgr2=0x7f8b7403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f8b740396c0 0x7f8b7403bb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b78006fd0 tx=0x7f8b78006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 msgr2=0x7f8b8c19e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c19e230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b80011a00 tx=0x7f8b80011ae0 comp rx=0 tx=0).stop 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 shutdown_connections 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f8b740396c0 0x7f8b7403bb80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 --2- 172.21.15.57:0/3381563386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b8c07dd40 0x7f8b8c19e230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 >> 172.21.15.57:0/3381563386 conn(0x7f8b8c103940 msgr2=0x7f8b8c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.383+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 shutdown_connections 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:04:59.384+0000 7f8b93a00700 1 -- 172.21.15.57:0/3381563386 wait complete. 2024-04-13T05:05:00.131 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling the dashboard module... 2024-04-13T05:05:02.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:02 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: ignoring --setuser ceph since I am not root 2024-04-13T05:05:02.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:02 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: ignoring --setgroup ceph since I am not root 2024-04-13T05:05:02.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:02 smithi057 ceph-mon[32331]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:02.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:02 smithi057 ceph-mon[32331]: from='mgr.14120 172.21.15.57:0/1133481434' entity='mgr.x' 2024-04-13T05:05:02.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3925277754' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-13T05:05:03.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:03 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3925277754' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-13T05:05:03.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:03 smithi057 ceph-mon[32331]: mgrmap e9: x(active, since 27s) 2024-04-13T05:05:03.243 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:02 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:02.905+0000 7fb20e037200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T05:05:03.243 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:03 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:03.034+0000 7fb20e037200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T05:05:03.814 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.772+0000 7f6fc1c00700 1 Processor -- start 2024-04-13T05:05:03.814 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.773+0000 7f6fc1c00700 1 -- start start 2024-04-13T05:05:03.814 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.773+0000 7f6fc1c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc07c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:03.814 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.773+0000 7f6fc1c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc07c7d0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.814 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.774+0000 7f6fbaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc07c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:03.814 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.774+0000 7f6fbaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc07c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38898/0 (socket says 172.21.15.57:38898) 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.774+0000 7f6fbaa00700 1 -- 172.21.15.57:0/519629347 learned_addr learned my addr 172.21.15.57:0/519629347 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.774+0000 7f6fbaa00700 1 -- 172.21.15.57:0/519629347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc07c950 con 0x7f6fbc07dde0 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.774+0000 7f6fbaa00700 1 --2- 172.21.15.57:0/519629347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc07c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0098a0 tx=0x7f6fac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815d2ec013e062a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.775+0000 7f6fb9600700 1 -- 172.21.15.57:0/519629347 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fac004160 con 0x7f6fbc07dde0 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.775+0000 7f6fb9600700 1 -- 172.21.15.57:0/519629347 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fac004300 con 0x7f6fbc07dde0 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.775+0000 7f6fb9600700 1 -- 172.21.15.57:0/519629347 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fac011a50 con 0x7f6fbc07dde0 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.775+0000 7f6fc1c00700 1 -- 172.21.15.57:0/519629347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 msgr2=0x7f6fbc07c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.775+0000 7f6fc1c00700 1 --2- 172.21.15.57:0/519629347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc07c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0098a0 tx=0x7f6fac009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.776+0000 7f6fc1c00700 1 -- 172.21.15.57:0/519629347 shutdown_connections 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.776+0000 7f6fc1c00700 1 --2- 172.21.15.57:0/519629347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc07c290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.776+0000 7f6fc1c00700 1 -- 172.21.15.57:0/519629347 >> 172.21.15.57:0/519629347 conn(0x7f6fbc103860 msgr2=0x7f6fbc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.776+0000 7f6fc1c00700 1 -- 172.21.15.57:0/519629347 shutdown_connections 2024-04-13T05:05:03.815 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.776+0000 7f6fc1c00700 1 -- 172.21.15.57:0/519629347 wait complete. 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.777+0000 7f6fc1c00700 1 Processor -- start 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.777+0000 7f6fc1c00700 1 -- start start 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.778+0000 7f6fc1c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.778+0000 7f6fc1c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc10e7c0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.778+0000 7f6fbaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.778+0000 7f6fbaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38906/0 (socket says 172.21.15.57:38906) 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.778+0000 7f6fbaa00700 1 -- 172.21.15.57:0/3925277754 learned_addr learned my addr 172.21.15.57:0/3925277754 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.778+0000 7f6fbaa00700 1 -- 172.21.15.57:0/3925277754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc19e7c0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.779+0000 7f6fbaa00700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc19e280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009130 tx=0x7f6fac004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.779+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fac003af0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.779+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fac01a8e0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.816 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.779+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc19ea50 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.779+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fac0248b0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.779+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc1a1740 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.780+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6fac022020 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.780+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc064110 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.781+0000 7f6fb3400700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f6fa0039670 0x7f6fa003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.781+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6fac04dc60 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.781+0000 7f6fba000700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f6fa0039670 0x7f6fa003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.782+0000 7f6fba000700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f6fa0039670 0x7f6fa003bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8006fd0 tx=0x7f6fa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:00.786+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fac01adb0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:01.092+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f6fbc1a1cf0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.013+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6fac04d5b0 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.018+0000 7f6fb3400700 1 -- 172.21.15.57:0/3925277754 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6fac01aa90 con 0x7f6fbc07dde0 2024-04-13T05:05:03.817 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.018+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f6fa0039670 msgr2=0x7f6fa003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.018+0000 7f6fc1c00700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f6fa0039670 0x7f6fa003bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8006fd0 tx=0x7f6fa8006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.018+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 msgr2=0x7f6fbc19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.018+0000 7f6fc1c00700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc19e280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009130 tx=0x7f6fac004620 comp rx=0 tx=0).stop 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.019+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 shutdown_connections 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.019+0000 7f6fc1c00700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f6fa0039670 0x7f6fa003bb30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.019+0000 7f6fc1c00700 1 --2- 172.21.15.57:0/3925277754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fbc07dde0 0x7f6fbc19e280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.019+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 >> 172.21.15.57:0/3925277754 conn(0x7f6fbc103860 msgr2=0x7f6fbc104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.019+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 shutdown_connections 2024-04-13T05:05:03.818 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:02.019+0000 7f6fc1c00700 1 -- 172.21.15.57:0/3925277754 wait complete. 2024-04-13T05:05:04.119 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:03 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:03.883+0000 7fb20e037200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T05:05:04.119 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:04.012+0000 7fb20e037200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T05:05:04.433 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:04.203+0000 7fb20e037200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T05:05:04.433 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:04.334+0000 7fb20e037200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T05:05:04.768 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:04.450+0000 7fb20e037200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T05:05:05.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3523684467' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-13T05:05:05.030 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:04.848+0000 7fb20e037200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T05:05:05.030 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:04.994+0000 7fb20e037200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T05:05:05.473 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:05 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:05.115+0000 7fb20e037200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T05:05:05.473 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:05 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:05.246+0000 7fb20e037200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T05:05:05.473 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:05 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:05.472+0000 7fb20e037200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T05:05:05.605 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:05:05.606 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-13T05:05:05.606 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-13T05:05:05.606 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-13T05:05:05.606 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.451+0000 7f1289c00700 1 Processor -- start 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.451+0000 7f1289c00700 1 -- start start 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.451+0000 7f1289c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f1284106f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.451+0000 7f1289c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f128407bdc0 con 0x7f1284106b50 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.452+0000 7f1282a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f1284106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.452+0000 7f1282a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f1284106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38932/0 (socket says 172.21.15.57:38932) 2024-04-13T05:05:05.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.452+0000 7f1282a00700 1 -- 172.21.15.57:0/609641258 learned_addr learned my addr 172.21.15.57:0/609641258 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.452+0000 7f1282a00700 1 -- 172.21.15.57:0/609641258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12841074b0 con 0x7f1284106b50 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.453+0000 7f1282a00700 1 --2- 172.21.15.57:0/609641258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f1284106f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12740098a0 tx=0x7f1274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0851564006d0cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.453+0000 7f1281600700 1 -- 172.21.15.57:0/609641258 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1274004160 con 0x7f1284106b50 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.453+0000 7f1281600700 1 -- 172.21.15.57:0/609641258 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1274004300 con 0x7f1284106b50 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.453+0000 7f1281600700 1 -- 172.21.15.57:0/609641258 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1274011a50 con 0x7f1284106b50 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.453+0000 7f1289c00700 1 -- 172.21.15.57:0/609641258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 msgr2=0x7f1284106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.454+0000 7f1289c00700 1 --2- 172.21.15.57:0/609641258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f1284106f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12740098a0 tx=0x7f1274009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:05.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.454+0000 7f1289c00700 1 -- 172.21.15.57:0/609641258 shutdown_connections 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.454+0000 7f1289c00700 1 --2- 172.21.15.57:0/609641258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f1284106f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.454+0000 7f1289c00700 1 -- 172.21.15.57:0/609641258 >> 172.21.15.57:0/609641258 conn(0x7f1284102730 msgr2=0x7f1284104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.454+0000 7f1289c00700 1 -- 172.21.15.57:0/609641258 shutdown_connections 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.454+0000 7f1289c00700 1 -- 172.21.15.57:0/609641258 wait complete. 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.455+0000 7f1289c00700 1 Processor -- start 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.456+0000 7f1289c00700 1 -- start start 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.456+0000 7f1289c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f128419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.456+0000 7f1289c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1284108520 con 0x7f1284106b50 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.456+0000 7f1282a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f128419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:05.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.456+0000 7f1282a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f128419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38946/0 (socket says 172.21.15.57:38946) 2024-04-13T05:05:05.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.456+0000 7f1282a00700 1 -- 172.21.15.57:0/3523684467 learned_addr learned my addr 172.21.15.57:0/3523684467 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:05.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f1282a00700 1 -- 172.21.15.57:0/3523684467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128419e7e0 con 0x7f1284106b50 2024-04-13T05:05:05.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f1282a00700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f128419e2a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1274011a00 tx=0x7f1274011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:05.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1274003af0 con 0x7f1284106b50 2024-04-13T05:05:05.613 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f127401a8e0 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1274024840 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f1289c00700 1 -- 172.21.15.57:0/3523684467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128419ea70 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.457+0000 7f1289c00700 1 -- 172.21.15.57:0/3523684467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12841a1760 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.458+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f1274022020 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.459+0000 7f127b400700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 0x7f126803bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.459+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1284064110 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.459+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f127404df00 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.459+0000 7f1282000700 1 -- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 msgr2=0x7f126803bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.459+0000 7f1282000700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 0x7f126803bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.465+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f127400b3e0 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.660+0000 7f1282000700 1 -- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 msgr2=0x7f126803bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.660+0000 7f1282000700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 0x7f126803bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.768+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1284062ef0 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.769+0000 7f127b400700 1 -- 172.21.15.57:0/3523684467 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f12740492d0 con 0x7f1284106b50 2024-04-13T05:05:05.614 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 msgr2=0x7f126803bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 0x7f126803bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 msgr2=0x7f128419e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f128419e2a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1274011a00 tx=0x7f1274011ae0 comp rx=0 tx=0).stop 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 shutdown_connections 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7f1268039720 0x7f126803bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 --2- 172.21.15.57:0/3523684467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1284106b50 0x7f128419e2a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 >> 172.21.15.57:0/3523684467 conn(0x7f1284102730 msgr2=0x7f1284194f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 shutdown_connections 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:04.772+0000 7f1278c00700 1 -- 172.21.15.57:0/3523684467 wait complete. 2024-04-13T05:05:05.615 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-04-13T05:05:05.616 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 9... 2024-04-13T05:05:07.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:06 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:06.846+0000 7fb20e037200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T05:05:07.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:07.025+0000 7fb20e037200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T05:05:07.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:07.460+0000 7fb20e037200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T05:05:07.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:07.584+0000 7fb20e037200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T05:05:08.109 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:07.853+0000 7fb20e037200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:05:08.505 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:08 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:08.109+0000 7fb20e037200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T05:05:08.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:08 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:08.505+0000 7fb20e037200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:05:09.153 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:08 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:08.832+0000 7fb20e037200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T05:05:09.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:09 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:09.153+0000 7fb20e037200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T05:05:10.529 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:10.236+0000 7fb20e037200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T05:05:11.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:10.967+0000 7fb20e037200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T05:05:11.279 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:11.103+0000 7fb20e037200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T05:05:11.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:05:11.404+0000 7fb20e037200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T05:05:11.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: Active manager daemon x restarted 2024-04-13T05:05:11.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: Activating manager daemon x 2024-04-13T05:05:11.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: osdmap e3: 0 total, 0 up, 0 in 2024-04-13T05:05:11.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: mgrmap e10: x(active, starting, since 0.0137041s) 2024-04-13T05:05:11.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:05:11.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-13T05:05:11.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-13T05:05:11.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-13T05:05:11.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-13T05:05:11.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:11 smithi057 ceph-mon[32331]: Manager daemon x is now available 2024-04-13T05:05:12.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:05:12.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-13T05:05:12.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-13T05:05:12.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:12 smithi057 ceph-mon[32331]: mgrmap e11: x(active, since 1.01108s) 2024-04-13T05:05:13.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-04-13T05:05:13.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-13T05:05:13.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-13T05:05:13.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-04-13T05:05:13.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.181+0000 7ff16a400700 1 Processor -- start 2024-04-13T05:05:13.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.181+0000 7ff16a400700 1 -- start start 2024-04-13T05:05:13.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.182+0000 7ff16a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff164106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:13.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.182+0000 7ff16a400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff16407be60 con 0x7ff1641068d0 2024-04-13T05:05:13.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.182+0000 7ff169000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff164106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:13.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.182+0000 7ff169000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff164106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56842/0 (socket says 172.21.15.57:56842) 2024-04-13T05:05:13.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.182+0000 7ff169000700 1 -- 172.21.15.57:0/3380457147 learned_addr learned my addr 172.21.15.57:0/3380457147 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:13.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.183+0000 7ff169000700 1 -- 172.21.15.57:0/3380457147 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff164107230 con 0x7ff1641068d0 2024-04-13T05:05:13.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.183+0000 7ff169000700 1 --2- 172.21.15.57:0/3380457147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff164106cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1580098a0 tx=0x7ff158009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c580618149048d5d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:13.241 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.183+0000 7ff163400700 1 -- 172.21.15.57:0/3380457147 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158004160 con 0x7ff1641068d0 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.183+0000 7ff163400700 1 -- 172.21.15.57:0/3380457147 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff158004300 con 0x7ff1641068d0 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.183+0000 7ff163400700 1 -- 172.21.15.57:0/3380457147 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158011a50 con 0x7ff1641068d0 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.184+0000 7ff16a400700 1 -- 172.21.15.57:0/3380457147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 msgr2=0x7ff164106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.184+0000 7ff16a400700 1 --2- 172.21.15.57:0/3380457147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff164106cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1580098a0 tx=0x7ff158009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.184+0000 7ff16a400700 1 -- 172.21.15.57:0/3380457147 shutdown_connections 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.184+0000 7ff16a400700 1 --2- 172.21.15.57:0/3380457147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff164106cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.184+0000 7ff16a400700 1 -- 172.21.15.57:0/3380457147 >> 172.21.15.57:0/3380457147 conn(0x7ff1641024f0 msgr2=0x7ff164104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:13.243 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.184+0000 7ff16a400700 1 -- 172.21.15.57:0/3380457147 shutdown_connections 2024-04-13T05:05:13.244 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.185+0000 7ff16a400700 1 -- 172.21.15.57:0/3380457147 wait complete. 2024-04-13T05:05:13.244 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.186+0000 7ff16a400700 1 Processor -- start 2024-04-13T05:05:13.244 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.186+0000 7ff16a400700 1 -- start start 2024-04-13T05:05:13.244 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.187+0000 7ff16a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff16419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.187+0000 7ff16a400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641082a0 con 0x7ff1641068d0 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.187+0000 7ff169000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff16419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.187+0000 7ff169000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff16419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56852/0 (socket says 172.21.15.57:56852) 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.187+0000 7ff169000700 1 -- 172.21.15.57:0/984882892 learned_addr learned my addr 172.21.15.57:0/984882892 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.187+0000 7ff169000700 1 -- 172.21.15.57:0/984882892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16419e610 con 0x7ff1641068d0 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.188+0000 7ff169000700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff16419e0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff158011a00 tx=0x7ff158011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.188+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158003af0 con 0x7ff1641068d0 2024-04-13T05:05:13.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.188+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff15801a8e0 con 0x7ff1641068d0 2024-04-13T05:05:13.249 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.188+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16419e8a0 con 0x7ff1641068d0 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.188+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158024840 con 0x7ff1641068d0 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.188+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641a1590 con 0x7ff1641068d0 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.189+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff158022020 con 0x7ff1641068d0 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.190+0000 7ff161600700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.190+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 --> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff14c03c2f0 con 0x7ff14c039720 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.190+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff15804db00 con 0x7ff1641068d0 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.190+0000 7ff163e00700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 msgr2=0x7ff14c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.190+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.391+0000 7ff163e00700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 msgr2=0x7ff14c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:13.250 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.391+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-13T05:05:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.792+0000 7ff163e00700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 msgr2=0x7ff14c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:06.792+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-13T05:05:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:07.593+0000 7ff163e00700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 msgr2=0x7ff14c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:07.593+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-13T05:05:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:09.195+0000 7ff163e00700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 msgr2=0x7ff14c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/3670387557 2024-04-13T05:05:13.254 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:09.195+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:11.417+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7ff15801aa90 con 0x7ff1641068d0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:11.417+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 msgr2=0x7ff14c03bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:11.417+0000 7ff161600700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.415+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ff15800bd10 con 0x7ff1641068d0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.415+0000 7ff161600700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff14c03cfc0 0x7ff14c03f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.415+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff158017ec0 con 0x7ff14c03cfc0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.415+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff14c03cfc0 0x7ff14c03f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.417+0000 7ff163e00700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff14c03cfc0 0x7ff14c03f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff150003a10 tx=0x7ff150009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.418+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff158017ec0 con 0x7ff14c03cfc0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.432+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff164064110 con 0x7ff14c03cfc0 2024-04-13T05:05:13.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff161600700 1 -- 172.21.15.57:0/984882892 <== mgr.14152 v2:172.21.15.57:6800/3239836246 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff164064110 con 0x7ff14c03cfc0 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff14c03cfc0 msgr2=0x7ff14c03f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff14c03cfc0 0x7ff14c03f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff150003a10 tx=0x7ff150009270 comp rx=0 tx=0).stop 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 msgr2=0x7ff16419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff16419e0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff158011a00 tx=0x7ff158011ae0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 shutdown_connections 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff14c03cfc0 0x7ff14c03f3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1641068d0 0x7ff16419e0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 --2- 172.21.15.57:0/984882892 >> [v2:172.21.15.57:6800/3670387557,v1:172.21.15.57:6801/3670387557] conn(0x7ff14c039720 0x7ff14c03bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 >> 172.21.15.57:0/984882892 conn(0x7ff1641024f0 msgr2=0x7ff164194cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.433+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 shutdown_connections 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:12.434+0000 7ff16a400700 1 -- 172.21.15.57:0/984882892 wait complete. 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 9 is available 2024-04-13T05:05:13.256 INFO:teuthology.orchestra.run.smithi057.stdout:Generating a dashboard self-signed certificate... 2024-04-13T05:05:13.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:13 smithi057 ceph-mon[32331]: [13/Apr/2024:05:05:12] ENGINE Bus STARTING 2024-04-13T05:05:13.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:13 smithi057 ceph-mon[32331]: [13/Apr/2024:05:05:12] ENGINE Serving on https://172.21.15.57:7150 2024-04-13T05:05:13.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:13 smithi057 ceph-mon[32331]: [13/Apr/2024:05:05:12] ENGINE Serving on http://172.21.15.57:8765 2024-04-13T05:05:13.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:13 smithi057 ceph-mon[32331]: [13/Apr/2024:05:05:12] ENGINE Bus STARTED 2024-04-13T05:05:14.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:14 smithi057 ceph-mon[32331]: mgrmap e12: x(active, since 2s) 2024-04-13T05:05:14.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:15.174 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-13T05:05:15.174 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.083+0000 7fbac2200700 1 Processor -- start 2024-04-13T05:05:15.174 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.084+0000 7fbac2200700 1 -- start start 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.084+0000 7fbac2200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc107350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.084+0000 7fbac2200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc107890 con 0x7fbabc106f10 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.085+0000 7fbac0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.085+0000 7fbac0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc107350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56916/0 (socket says 172.21.15.57:56916) 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.085+0000 7fbac0e00700 1 -- 172.21.15.57:0/3608714442 learned_addr learned my addr 172.21.15.57:0/3608714442 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.085+0000 7fbac0e00700 1 -- 172.21.15.57:0/3608714442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc1080f0 con 0x7fbabc106f10 2024-04-13T05:05:15.175 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.085+0000 7fbac0e00700 1 --2- 172.21.15.57:0/3608714442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc107350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbab00098a0 tx=0x7fbab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e057cd42476dd355 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:15.176 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.086+0000 7fbabb400700 1 -- 172.21.15.57:0/3608714442 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbab0004160 con 0x7fbabc106f10 2024-04-13T05:05:15.176 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.086+0000 7fbabb400700 1 -- 172.21.15.57:0/3608714442 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab0004300 con 0x7fbabc106f10 2024-04-13T05:05:15.176 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.086+0000 7fbabb400700 1 -- 172.21.15.57:0/3608714442 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbab0011a50 con 0x7fbabc106f10 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.086+0000 7fbac2200700 1 -- 172.21.15.57:0/3608714442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 msgr2=0x7fbabc107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.086+0000 7fbac2200700 1 --2- 172.21.15.57:0/3608714442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc107350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbab00098a0 tx=0x7fbab0009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.087+0000 7fbac2200700 1 -- 172.21.15.57:0/3608714442 shutdown_connections 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.087+0000 7fbac2200700 1 --2- 172.21.15.57:0/3608714442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc107350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.087+0000 7fbac2200700 1 -- 172.21.15.57:0/3608714442 >> 172.21.15.57:0/3608714442 conn(0x7fbabc102520 msgr2=0x7fbabc104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.087+0000 7fbac2200700 1 -- 172.21.15.57:0/3608714442 shutdown_connections 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.087+0000 7fbac2200700 1 -- 172.21.15.57:0/3608714442 wait complete. 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.088+0000 7fbac2200700 1 Processor -- start 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.088+0000 7fbac2200700 1 -- start start 2024-04-13T05:05:15.179 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.088+0000 7fbac2200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.088+0000 7fbac2200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc10b3b0 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.089+0000 7fbac0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.089+0000 7fbac0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56926/0 (socket says 172.21.15.57:56926) 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.089+0000 7fbac0e00700 1 -- 172.21.15.57:0/891198093 learned_addr learned my addr 172.21.15.57:0/891198093 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.089+0000 7fbac0e00700 1 -- 172.21.15.57:0/891198093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc19e670 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.089+0000 7fbac0e00700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc19e130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbab0011a20 tx=0x7fbab0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.090+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbab0003af0 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.090+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab001a8e0 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.090+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc19e900 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.090+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbab0024860 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.090+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc1a15f0 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.091+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fbab0022020 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.091+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc10b530 con 0x7fbabc106f10 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.091+0000 7fbab9600700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbaa40396c0 0x7fbaa403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:15.180 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.091+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbab004dbc0 con 0x7fbabc106f10 2024-04-13T05:05:15.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.092+0000 7fbabbe00700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbaa40396c0 0x7fbaa403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:15.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.093+0000 7fbabbe00700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbaa40396c0 0x7fbaa403bb80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:15.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.100+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbab000f2e0 con 0x7fbabc106f10 2024-04-13T05:05:15.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.349+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbabc02e2c0 con 0x7fbaa40396c0 2024-04-13T05:05:15.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.527+0000 7fbab9600700 1 -- 172.21.15.57:0/891198093 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbabc02e2c0 con 0x7fbaa40396c0 2024-04-13T05:05:15.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbaa40396c0 msgr2=0x7fbaa403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:15.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbaa40396c0 0x7fbaa403bb80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa8006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:15.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 msgr2=0x7fbabc19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:15.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc19e130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbab0011a20 tx=0x7fbab0011b00 comp rx=0 tx=0).stop 2024-04-13T05:05:15.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 shutdown_connections 2024-04-13T05:05:15.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbaa40396c0 0x7fbaa403bb80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:15.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 --2- 172.21.15.57:0/891198093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbabc106f10 0x7fbabc19e130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:15.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.529+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 >> 172.21.15.57:0/891198093 conn(0x7fbabc102520 msgr2=0x7fbabc1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:15.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.530+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 shutdown_connections 2024-04-13T05:05:15.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:14.530+0000 7fbac2200700 1 -- 172.21.15.57:0/891198093 wait complete. 2024-04-13T05:05:15.183 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial admin user... 2024-04-13T05:05:15.724 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:15 smithi057 ceph-mon[32331]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:05:15.724 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:15.724 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:16.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:16 smithi057 ceph-mon[32331]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:05:16.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:16.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:17.315 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$GEQthwi4lsuR24VtLjP2wegTRl7Tgp5OszWsb636n9k32j9GNLOs2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712984716, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.889+0000 7f2d7cc00700 1 Processor -- start 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.889+0000 7f2d7cc00700 1 -- start start 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.890+0000 7f2d7cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7810b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.890+0000 7f2d7cc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7807be60 con 0x7f2d78108cc0 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.890+0000 7f2d75e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7810b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.890+0000 7f2d75e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7810b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47874/0 (socket says 172.21.15.57:47874) 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.890+0000 7f2d75e00700 1 -- 172.21.15.57:0/2703355536 learned_addr learned my addr 172.21.15.57:0/2703355536 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.891+0000 7f2d75e00700 1 -- 172.21.15.57:0/2703355536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7810b5f0 con 0x7f2d78108cc0 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.891+0000 7f2d75e00700 1 --2- 172.21.15.57:0/2703355536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7810b0b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d680098a0 tx=0x7f2d68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7671c5c3f5183865 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.891+0000 7f2d74a00700 1 -- 172.21.15.57:0/2703355536 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d68004160 con 0x7f2d78108cc0 2024-04-13T05:05:17.316 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.891+0000 7f2d74a00700 1 -- 172.21.15.57:0/2703355536 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d68004300 con 0x7f2d78108cc0 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.891+0000 7f2d74a00700 1 -- 172.21.15.57:0/2703355536 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d68011a50 con 0x7f2d78108cc0 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2703355536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 msgr2=0x7f2d7810b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 --2- 172.21.15.57:0/2703355536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7810b0b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d680098a0 tx=0x7f2d68009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2703355536 shutdown_connections 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 --2- 172.21.15.57:0/2703355536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7810b0b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2703355536 >> 172.21.15.57:0/2703355536 conn(0x7f2d78102730 msgr2=0x7f2d78104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2703355536 shutdown_connections 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.892+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2703355536 wait complete. 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.893+0000 7f2d7cc00700 1 Processor -- start 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.893+0000 7f2d7cc00700 1 -- start start 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.894+0000 7f2d7cc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.894+0000 7f2d7cc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7810c600 con 0x7f2d78108cc0 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.894+0000 7f2d75e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:17.317 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.894+0000 7f2d75e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47876/0 (socket says 172.21.15.57:47876) 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.894+0000 7f2d75e00700 1 -- 172.21.15.57:0/2691601922 learned_addr learned my addr 172.21.15.57:0/2691601922 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.894+0000 7f2d75e00700 1 -- 172.21.15.57:0/2691601922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819e7c0 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.895+0000 7f2d75e00700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7819e280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011a00 tx=0x7f2d68011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.895+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d68003af0 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.895+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d6801a8e0 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.895+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d68024840 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.895+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819ea50 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.895+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a1740 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.896+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f2d68022020 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.896+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78064110 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.897+0000 7f2d6ea00700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d5c0396c0 0x7f2d5c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.897+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2d6804dc60 con 0x7f2d78108cc0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.897+0000 7f2d75400700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d5c0396c0 0x7f2d5c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:17.318 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.898+0000 7f2d75400700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d5c0396c0 0x7f2d5c03bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006fd0 tx=0x7f2d64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:15.902+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d6801adb0 con 0x7f2d78108cc0 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.139+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2d781a19d0 con 0x7f2d5c0396c0 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.400+0000 7f2d6ea00700 1 -- 172.21.15.57:0/2691601922 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2d781a19d0 con 0x7f2d5c0396c0 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d5c0396c0 msgr2=0x7f2d5c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d5c0396c0 0x7f2d5c03bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006fd0 tx=0x7f2d64006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 msgr2=0x7f2d7819e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7819e280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011a00 tx=0x7f2d68011ae0 comp rx=0 tx=0).stop 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 shutdown_connections 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d5c0396c0 0x7f2d5c03bb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 --2- 172.21.15.57:0/2691601922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d78108cc0 0x7f2d7819e280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 >> 172.21.15.57:0/2691601922 conn(0x7f2d78102730 msgr2=0x7f2d78103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.403+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 shutdown_connections 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:16.404+0000 7f2d7cc00700 1 -- 172.21.15.57:0/2691601922 wait complete. 2024-04-13T05:05:17.319 INFO:teuthology.orchestra.run.smithi057.stdout:Fetching dashboard port number... 2024-04-13T05:05:18.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:18 smithi057 ceph-mon[32331]: mgrmap e13: x(active, since 6s) 2024-04-13T05:05:18.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:18 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1696868355' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-13T05:05:19.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 8443 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.974+0000 7fdbaae00700 1 Processor -- start 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.974+0000 7fdbaae00700 1 -- start start 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.974+0000 7fdbaae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba41086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.974+0000 7fdbaae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4108c70 con 0x7fdba4108280 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.974+0000 7fdba3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba41086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.975+0000 7fdba3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba41086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47888/0 (socket says 172.21.15.57:47888) 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.975+0000 7fdba3e00700 1 -- 172.21.15.57:0/3918160029 learned_addr learned my addr 172.21.15.57:0/3918160029 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.975+0000 7fdba3e00700 1 -- 172.21.15.57:0/3918160029 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba4109430 con 0x7fdba4108280 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.975+0000 7fdba3e00700 1 --2- 172.21.15.57:0/3918160029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba41086a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb940098a0 tx=0x7fdb94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee78d7a2c586c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdba2a00700 1 -- 172.21.15.57:0/3918160029 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb94004160 con 0x7fdba4108280 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdba2a00700 1 -- 172.21.15.57:0/3918160029 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb94004300 con 0x7fdba4108280 2024-04-13T05:05:19.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdba2a00700 1 -- 172.21.15.57:0/3918160029 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb94011a50 con 0x7fdba4108280 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdbaae00700 1 -- 172.21.15.57:0/3918160029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 msgr2=0x7fdba41086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdbaae00700 1 --2- 172.21.15.57:0/3918160029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba41086a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb940098a0 tx=0x7fdb94009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdbaae00700 1 -- 172.21.15.57:0/3918160029 shutdown_connections 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdbaae00700 1 --2- 172.21.15.57:0/3918160029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba41086a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.976+0000 7fdbaae00700 1 -- 172.21.15.57:0/3918160029 >> 172.21.15.57:0/3918160029 conn(0x7fdba4103860 msgr2=0x7fdba4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.977+0000 7fdbaae00700 1 -- 172.21.15.57:0/3918160029 shutdown_connections 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.977+0000 7fdbaae00700 1 -- 172.21.15.57:0/3918160029 wait complete. 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdbaae00700 1 Processor -- start 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdbaae00700 1 -- start start 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdbaae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdbaae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdba410c7e0 con 0x7fdba4108280 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdba3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdba3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba419e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47896/0 (socket says 172.21.15.57:47896) 2024-04-13T05:05:19.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.978+0000 7fdba3e00700 1 -- 172.21.15.57:0/1696868355 learned_addr learned my addr 172.21.15.57:0/1696868355 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdba3e00700 1 -- 172.21.15.57:0/1696868355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba419e800 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdba3e00700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba419e2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb94011a20 tx=0x7fdb94011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb94003af0 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb9401a8e0 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb94024860 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba419ea90 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.979+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba41a1780 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.980+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba4064110 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.981+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdb94022020 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.981+0000 7fdba0c00700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb880397c0 0x7fdb8803bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.981+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb9404dbf0 con 0x7fdba4108280 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.981+0000 7fdba3400700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb880397c0 0x7fdb8803bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.982+0000 7fdba3400700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb880397c0 0x7fdb8803bc80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006fd0 tx=0x7fdb90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:19.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:17.987+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb9400f2e0 con 0x7fdba4108280 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.213+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdba41a1d30 con 0x7fdba4108280 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.214+0000 7fdba0c00700 1 -- 172.21.15.57:0/1696868355 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdb940094f0 con 0x7fdba4108280 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb880397c0 msgr2=0x7fdb8803bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb880397c0 0x7fdb8803bc80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006fd0 tx=0x7fdb90006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 msgr2=0x7fdba419e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba419e2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb94011a20 tx=0x7fdb94011b00 comp rx=0 tx=0).stop 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 shutdown_connections 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb880397c0 0x7fdb8803bc80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 --2- 172.21.15.57:0/1696868355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdba4108280 0x7fdba419e2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 >> 172.21.15.57:0/1696868355 conn(0x7fdba4103860 msgr2=0x7fdba407c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 shutdown_connections 2024-04-13T05:05:19.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:18.218+0000 7fdbaae00700 1 -- 172.21.15.57:0/1696868355 wait complete. 2024-04-13T05:05:19.476 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-13T05:05:19.476 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-04-13T05:05:19.505 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-13T05:05:19.505 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-04-13T05:05:19.505 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-04-13T05:05:19.505 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-13T05:05:19.510 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph Dashboard is now available at: 2024-04-13T05:05:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: URL: https://smithi057.front.sepia.ceph.com:8443/ 2024-04-13T05:05:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: User: admin 2024-04-13T05:05:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: Password: 5tu7udfe1h 2024-04-13T05:05:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:19.511 INFO:teuthology.orchestra.run.smithi057.stdout:Saving cluster configuration to /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config directory 2024-04-13T05:05:19.513 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling autotune for osd_memory_target 2024-04-13T05:05:21.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.482+0000 7fc052e00700 1 Processor -- start 2024-04-13T05:05:21.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.482+0000 7fc052e00700 1 -- start start 2024-04-13T05:05:21.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.483+0000 7fc052e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c07c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:21.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.483+0000 7fc052e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c07c7d0 con 0x7fc04c07dde0 2024-04-13T05:05:21.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.483+0000 7fc04be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c07c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:21.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.483+0000 7fc04be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c07c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47898/0 (socket says 172.21.15.57:47898) 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.483+0000 7fc04be00700 1 -- 172.21.15.57:0/1220926558 learned_addr learned my addr 172.21.15.57:0/1220926558 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.484+0000 7fc04be00700 1 -- 172.21.15.57:0/1220926558 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c07c950 con 0x7fc04c07dde0 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.484+0000 7fc04be00700 1 --2- 172.21.15.57:0/1220926558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c07c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098a0 tx=0x7fc03c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c76e208302ccb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.485+0000 7fc04b400700 1 -- 172.21.15.57:0/1220926558 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c004160 con 0x7fc04c07dde0 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.485+0000 7fc04b400700 1 -- 172.21.15.57:0/1220926558 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc03c004300 con 0x7fc04c07dde0 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.485+0000 7fc04b400700 1 -- 172.21.15.57:0/1220926558 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c011a50 con 0x7fc04c07dde0 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.485+0000 7fc052e00700 1 -- 172.21.15.57:0/1220926558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 msgr2=0x7fc04c07c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.485+0000 7fc052e00700 1 --2- 172.21.15.57:0/1220926558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c07c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098a0 tx=0x7fc03c009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.486+0000 7fc052e00700 1 -- 172.21.15.57:0/1220926558 shutdown_connections 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.486+0000 7fc052e00700 1 --2- 172.21.15.57:0/1220926558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c07c290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.486+0000 7fc052e00700 1 -- 172.21.15.57:0/1220926558 >> 172.21.15.57:0/1220926558 conn(0x7fc04c103860 msgr2=0x7fc04c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.486+0000 7fc052e00700 1 -- 172.21.15.57:0/1220926558 shutdown_connections 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.486+0000 7fc052e00700 1 -- 172.21.15.57:0/1220926558 wait complete. 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.487+0000 7fc052e00700 1 Processor -- start 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.487+0000 7fc052e00700 1 -- start start 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.487+0000 7fc052e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.487+0000 7fc052e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e7e0 con 0x7fc04c07dde0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.488+0000 7fc04be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.488+0000 7fc04be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47910/0 (socket says 172.21.15.57:47910) 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.488+0000 7fc04be00700 1 -- 172.21.15.57:0/3078664825 learned_addr learned my addr 172.21.15.57:0/3078664825 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.488+0000 7fc04be00700 1 -- 172.21.15.57:0/3078664825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c19e7e0 con 0x7fc04c07dde0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.489+0000 7fc04be00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c19e2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009130 tx=0x7fc03c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.489+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c003af0 con 0x7fc04c07dde0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.489+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc03c01a8e0 con 0x7fc04c07dde0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.489+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c0248b0 con 0x7fc04c07dde0 2024-04-13T05:05:21.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.489+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c19ea70 con 0x7fc04c07dde0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.489+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1a1760 con 0x7fc04c07dde0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.490+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc03c022020 con 0x7fc04c07dde0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.490+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c064110 con 0x7fc04c07dde0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.491+0000 7fc049600700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0300397c0 0x7fc03003bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.491+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc03c04dc60 con 0x7fc04c07dde0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.491+0000 7fc043e00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0300397c0 0x7fc03003bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.492+0000 7fc043e00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0300397c0 0x7fc03003bc80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc034006fd0 tx=0x7fc034006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.496+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc03c00b3e0 con 0x7fc04c07dde0 2024-04-13T05:05:21.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.727+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fc04c1a1930 con 0x7fc04c07dde0 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.728+0000 7fc049600700 1 -- 172.21.15.57:0/3078664825 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fc03c016260 con 0x7fc04c07dde0 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0300397c0 msgr2=0x7fc03003bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0300397c0 0x7fc03003bc80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc034006fd0 tx=0x7fc034006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 msgr2=0x7fc04c19e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c19e2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009130 tx=0x7fc03c004620 comp rx=0 tx=0).stop 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 shutdown_connections 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0300397c0 0x7fc03003bc80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 --2- 172.21.15.57:0/3078664825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc04c07dde0 0x7fc04c19e2a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 >> 172.21.15.57:0/3078664825 conn(0x7fc04c103860 msgr2=0x7fc04c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.732+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 shutdown_connections 2024-04-13T05:05:21.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:20.733+0000 7fc052e00700 1 -- 172.21.15.57:0/3078664825 wait complete. 2024-04-13T05:05:23.270 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.198+0000 7f20b1a00700 1 Processor -- start 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.199+0000 7f20b1a00700 1 -- start start 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.199+0000 7f20b1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac10a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.199+0000 7f20b1a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10ae60 con 0x7f20ac10a470 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.199+0000 7f20aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac10a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.200+0000 7f20aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac10a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47922/0 (socket says 172.21.15.57:47922) 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.200+0000 7f20aaa00700 1 -- 172.21.15.57:0/672184091 learned_addr learned my addr 172.21.15.57:0/672184091 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.200+0000 7f20aaa00700 1 -- 172.21.15.57:0/672184091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac10b610 con 0x7f20ac10a470 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.200+0000 7f20aaa00700 1 --2- 172.21.15.57:0/672184091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac10a890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f209c0098a0 tx=0x7f209c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01dcb7f27c1918d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:23.272 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20aa000700 1 -- 172.21.15.57:0/672184091 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f209c004160 con 0x7f20ac10a470 2024-04-13T05:05:23.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20aa000700 1 -- 172.21.15.57:0/672184091 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c004300 con 0x7f20ac10a470 2024-04-13T05:05:23.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20aa000700 1 -- 172.21.15.57:0/672184091 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f209c011a50 con 0x7f20ac10a470 2024-04-13T05:05:23.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20b1a00700 1 -- 172.21.15.57:0/672184091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 msgr2=0x7f20ac10a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:23.275 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20b1a00700 1 --2- 172.21.15.57:0/672184091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac10a890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f209c0098a0 tx=0x7f209c009bb0 comp rx=0 tx=0).stop 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20b1a00700 1 -- 172.21.15.57:0/672184091 shutdown_connections 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20b1a00700 1 --2- 172.21.15.57:0/672184091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac10a890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.201+0000 7f20b1a00700 1 -- 172.21.15.57:0/672184091 >> 172.21.15.57:0/672184091 conn(0x7f20ac07dbc0 msgr2=0x7f20ac108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.202+0000 7f20b1a00700 1 -- 172.21.15.57:0/672184091 shutdown_connections 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.202+0000 7f20b1a00700 1 -- 172.21.15.57:0/672184091 wait complete. 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.203+0000 7f20b1a00700 1 Processor -- start 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.203+0000 7f20b1a00700 1 -- start start 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.203+0000 7f20b1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.203+0000 7f20b1a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10e9c0 con 0x7f20ac10a470 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.203+0000 7f20aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:23.276 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.203+0000 7f20aaa00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac19e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47934/0 (socket says 172.21.15.57:47934) 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.204+0000 7f20aaa00700 1 -- 172.21.15.57:0/1930212460 learned_addr learned my addr 172.21.15.57:0/1930212460 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.204+0000 7f20aaa00700 1 -- 172.21.15.57:0/1930212460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac19e7f0 con 0x7f20ac10a470 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.204+0000 7f20aaa00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac19e2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f209c011a20 tx=0x7f209c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.204+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f209c003af0 con 0x7f20ac10a470 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.204+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f209c01a8e0 con 0x7f20ac10a470 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.204+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f209c024860 con 0x7f20ac10a470 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.205+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac19ea80 con 0x7f20ac10a470 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.205+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac1a1770 con 0x7f20ac10a470 2024-04-13T05:05:23.277 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.206+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f209c022020 con 0x7f20ac10a470 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.206+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ac064110 con 0x7f20ac10a470 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.206+0000 7f20a3400700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f20900397c0 0x7f209003bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.206+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f209c04dc00 con 0x7f20ac10a470 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.207+0000 7f20a3e00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f20900397c0 0x7f209003bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.207+0000 7f20a3e00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f20900397c0 0x7f209003bc80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2094006fd0 tx=0x7f2094006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.212+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f209c02c3a0 con 0x7f20ac10a470 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.563+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f20ac1a1940 con 0x7f20ac10a470 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.568+0000 7f20a3400700 1 -- 172.21.15.57:0/1930212460 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f209c01a570 con 0x7f20ac10a470 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f20900397c0 msgr2=0x7f209003bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f20900397c0 0x7f209003bc80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2094006fd0 tx=0x7f2094006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 msgr2=0x7f20ac19e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac19e2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f209c011a20 tx=0x7f209c011b00 comp rx=0 tx=0).stop 2024-04-13T05:05:23.280 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 shutdown_connections 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f20900397c0 0x7f209003bc80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 --2- 172.21.15.57:0/1930212460 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20ac10a470 0x7f20ac19e2b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.571+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 >> 172.21.15.57:0/1930212460 conn(0x7f20ac07dbc0 msgr2=0x7f20ac107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.572+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 shutdown_connections 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-04-13T05:05:22.572+0000 7f20b1a00700 1 -- 172.21.15.57:0/1930212460 wait complete. 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:Or, if you are only running a single cluster on this host: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: ceph telemetry on 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout:For more information see: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:23.282 INFO:teuthology.orchestra.run.smithi057.stdout:Bootstrap complete. 2024-04-13T05:05:23.401 INFO:tasks.cephadm:Fetching config... 2024-04-13T05:05:23.401 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:05:23.401 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-13T05:05:23.489 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-13T05:05:23.489 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:05:23.489 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-13T05:05:23.580 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-13T05:05:23.580 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:05:23.580 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-13T05:05:23.673 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-13T05:05:23.673 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:05:23.673 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-13T05:05:23.773 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-13T05:05:23.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-13T05:05:23.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:23 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1930212460' entity='client.admin' 2024-04-13T05:05:23.912 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:05:23.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-13T05:05:24.098 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:05:24.161 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-13T05:05:24.298 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbPM+fIwzTvACgiEmKmICXLRFESTO6LuEL1kQrwp7PF90GkhYnxQxQpoQ56PcXl+yArOEyrBf8aMK58vm4wbm+8p4GOozARiFGCZiNaOJGgCAEn6JAJ+wLFVnrGUyRa/ulw1vadYOp60Oy866hxpK6l34oZt03D3+0H24TlqhiH7v4nzqigzc0y8FFRv3NpRBe1oSOZu4Wh4widH+16bECU/yD161ruD4NSKU5dwTZqSPcizztEE5TuEeJe3v8K6NxsyFWNOvuPqdzu4hjNYm44xvBlgh6RbPcoOKgxbxLsBGqd+nPEVZSicpG92mid1melOgXScCID2AVTIcqGw64SFEaebiinK17VCsIXLce1UyyhY6IEw2EstPKfUfv7X7zzXxjF/clFc1j96lwhNyplo98STXyhCAQQTxZQW9SSYXgmmzCQ6oD5oe9krNjqVD8burYSRKY0KaMGFGYAWeVXIQHBx2MhA8NQgddCBQTJI9B1wBDC368dZUsK0oCmac= ceph-0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:05:24.344 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-13T05:05:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.538+0000 7f9bb6e00700 1 -- 172.21.15.57:0/1295812746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 msgr2=0x7f9bb0105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.538+0000 7f9bb6e00700 1 --2- 172.21.15.57:0/1295812746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0105390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009910 tx=0x7f9ba0009c20 comp rx=0 tx=0).stop 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.539+0000 7f9bb6e00700 1 -- 172.21.15.57:0/1295812746 shutdown_connections 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.539+0000 7f9bb6e00700 1 --2- 172.21.15.57:0/1295812746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0105390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.539+0000 7f9bb6e00700 1 -- 172.21.15.57:0/1295812746 >> 172.21.15.57:0/1295812746 conn(0x7f9bb00fc9b0 msgr2=0x7f9bb00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.539+0000 7f9bb6e00700 1 -- 172.21.15.57:0/1295812746 shutdown_connections 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.539+0000 7f9bb6e00700 1 -- 172.21.15.57:0/1295812746 wait complete. 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.540+0000 7f9bb6e00700 1 Processor -- start 2024-04-13T05:05:27.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.540+0000 7f9bb6e00700 1 -- start start 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.540+0000 7f9bb6e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.540+0000 7f9bb6e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb01083f0 con 0x7f9bb0102fa0 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.541+0000 7f9bafe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.541+0000 7f9bafe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51084/0 (socket says 172.21.15.57:51084) 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.541+0000 7f9bafe00700 1 -- 172.21.15.57:0/344727549 learned_addr learned my addr 172.21.15.57:0/344727549 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.541+0000 7f9bafe00700 1 -- 172.21.15.57:0/344727549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb019a220 con 0x7f9bb0102fa0 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.541+0000 7f9bafe00700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0199ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0005190 tx=0x7f9ba0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.542+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ba0027070 con 0x7f9bb0102fa0 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.542+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ba0022470 con 0x7f9bb0102fa0 2024-04-13T05:05:27.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.542+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ba000f5f0 con 0x7f9bb0102fa0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.542+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb019a510 con 0x7f9bb0102fa0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.542+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb019d190 con 0x7f9bb0102fa0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.543+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9ba002a070 con 0x7f9bb0102fa0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.543+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb0108b70 con 0x7f9bb0102fa0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.544+0000 7f9bacc00700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b94039700 0x7f9b9403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.544+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ba001e030 con 0x7f9bb0102fa0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.544+0000 7f9baf400700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b94039700 0x7f9b9403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.545+0000 7f9baf400700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b94039700 0x7f9b9403bbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006fd0 tx=0x7f9b9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:27.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.550+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ba0013070 con 0x7f9bb0102fa0 2024-04-13T05:05:27.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.789+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9bb002e1f0 con 0x7f9bb0102fa0 2024-04-13T05:05:27.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.793+0000 7f9bacc00700 1 -- 172.21.15.57:0/344727549 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9ba00310c0 con 0x7f9bb0102fa0 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b94039700 msgr2=0x7f9b9403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b94039700 0x7f9b9403bbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006fd0 tx=0x7f9b9c006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 msgr2=0x7f9bb0199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0199ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0005190 tx=0x7f9ba0004f80 comp rx=0 tx=0).stop 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 shutdown_connections 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b94039700 0x7f9b9403bbc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 --2- 172.21.15.57:0/344727549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9bb0102fa0 0x7f9bb0199ce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.796+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 >> 172.21.15.57:0/344727549 conn(0x7f9bb00fc9b0 msgr2=0x7f9bb00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.797+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 shutdown_connections 2024-04-13T05:05:27.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:27.797+0000 7f9bb6e00700 1 -- 172.21.15.57:0/344727549 wait complete. 2024-04-13T05:05:28.614 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-13T05:05:28.615 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-13T05:05:29.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:28 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/344727549' entity='client.admin' 2024-04-13T05:05:29.049 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:05:31.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:31.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:31.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:05:31.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:31.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:05:31.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:31.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 -- 172.21.15.57:0/986748855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 msgr2=0x7f93c8101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:31.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 --2- 172.21.15.57:0/986748855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8101830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009910 tx=0x7f93b8009c20 comp rx=0 tx=0).stop 2024-04-13T05:05:31.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 -- 172.21.15.57:0/986748855 shutdown_connections 2024-04-13T05:05:31.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 --2- 172.21.15.57:0/986748855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8101830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:31.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 -- 172.21.15.57:0/986748855 >> 172.21.15.57:0/986748855 conn(0x7f93c80fc9b0 msgr2=0x7f93c80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:31.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 -- 172.21.15.57:0/986748855 shutdown_connections 2024-04-13T05:05:31.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.701+0000 7f93cf600700 1 -- 172.21.15.57:0/986748855 wait complete. 2024-04-13T05:05:31.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.702+0000 7f93cf600700 1 Processor -- start 2024-04-13T05:05:31.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.703+0000 7f93cf600700 1 -- start start 2024-04-13T05:05:31.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.703+0000 7f93cf600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:31.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.703+0000 7f93cf600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8107350 con 0x7f93c8101410 2024-04-13T05:05:31.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.703+0000 7f93ccc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:31.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.703+0000 7f93ccc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51108/0 (socket says 172.21.15.57:51108) 2024-04-13T05:05:31.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.703+0000 7f93ccc00700 1 -- 172.21.15.57:0/91106514 learned_addr learned my addr 172.21.15.57:0/91106514 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:31.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.704+0000 7f93ccc00700 1 -- 172.21.15.57:0/91106514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c81980a0 con 0x7f93c8101410 2024-04-13T05:05:31.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.704+0000 7f93ccc00700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8197b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005950 tx=0x7f93b8004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:31.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.704+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b8026070 con 0x7f93c8101410 2024-04-13T05:05:31.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.705+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b800bc00 con 0x7f93c8101410 2024-04-13T05:05:31.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.705+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c8198330 con 0x7f93c8101410 2024-04-13T05:05:31.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.705+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b80039a0 con 0x7f93c8101410 2024-04-13T05:05:31.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.705+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c819b020 con 0x7f93c8101410 2024-04-13T05:05:31.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.706+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c8107ad0 con 0x7f93c8101410 2024-04-13T05:05:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.706+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f93b8029070 con 0x7f93c8101410 2024-04-13T05:05:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.707+0000 7f93c6000700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93ac039650 0x7f93ac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.707+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93b801e070 con 0x7f93c8101410 2024-04-13T05:05:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.707+0000 7f93bfe00700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93ac039650 0x7f93ac03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.708+0000 7f93bfe00700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93ac039650 0x7f93ac03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93b0006fd0 tx=0x7f93b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:31.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.713+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93b8013070 con 0x7f93c8101410 2024-04-13T05:05:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.952+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f93c802e0b0 con 0x7f93ac039650 2024-04-13T05:05:31.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.964+0000 7f93c6000700 1 -- 172.21.15.57:0/91106514 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f93c802e0b0 con 0x7f93ac039650 2024-04-13T05:05:31.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.966+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93ac039650 msgr2=0x7f93ac03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:31.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.966+0000 7f93cf600700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93ac039650 0x7f93ac03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93b0006fd0 tx=0x7f93b0006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:31.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.966+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 msgr2=0x7f93c8197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:31.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.966+0000 7f93cf600700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8197b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005950 tx=0x7f93b8004fa0 comp rx=0 tx=0).stop 2024-04-13T05:05:31.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.967+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 shutdown_connections 2024-04-13T05:05:31.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.967+0000 7f93cf600700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93ac039650 0x7f93ac03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:31.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.967+0000 7f93cf600700 1 --2- 172.21.15.57:0/91106514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93c8101410 0x7f93c8197b60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:31.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.967+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 >> 172.21.15.57:0/91106514 conn(0x7f93c80fc9b0 msgr2=0x7f93c806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:31.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.967+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 shutdown_connections 2024-04-13T05:05:31.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:31.967+0000 7f93cf600700 1 -- 172.21.15.57:0/91106514 wait complete. 2024-04-13T05:05:32.779 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi083 2024-04-13T05:05:32.779 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:05:32.779 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.conf 2024-04-13T05:05:32.838 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:05:32.838 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:05:32.929 INFO:tasks.cephadm:Adding host smithi083 to orchestrator... 2024-04-13T05:05:32.929 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch host add smithi083 2024-04-13T05:05:33.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:33.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:05:33.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:33.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:05:33.001 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:05:33.001 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:05:33.001 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:32 smithi057 ceph-mon[32331]: Updating smithi057:/etc/ceph/ceph.conf 2024-04-13T05:05:33.348 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:05:34.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:33 smithi057 ceph-mon[32331]: Updating smithi057:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:05:35.211 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:34 smithi057 ceph-mon[32331]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:05:35.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:34 smithi057 ceph-mon[32331]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:35.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:34 smithi057 ceph-mon[32331]: Updating smithi057:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:05:35.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:35.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:35.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 -- 172.21.15.57:0/1234463379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 msgr2=0x7f05d4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 --2- 172.21.15.57:0/1234463379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d4108920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009910 tx=0x7f05c4009c20 comp rx=0 tx=0).stop 2024-04-13T05:05:35.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 -- 172.21.15.57:0/1234463379 shutdown_connections 2024-04-13T05:05:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 --2- 172.21.15.57:0/1234463379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d4108920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 -- 172.21.15.57:0/1234463379 >> 172.21.15.57:0/1234463379 conn(0x7f05d4076f10 msgr2=0x7f05d4077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 -- 172.21.15.57:0/1234463379 shutdown_connections 2024-04-13T05:05:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.391+0000 7f05dae00700 1 -- 172.21.15.57:0/1234463379 wait complete. 2024-04-13T05:05:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.392+0000 7f05dae00700 1 Processor -- start 2024-04-13T05:05:35.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.393+0000 7f05dae00700 1 -- start start 2024-04-13T05:05:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.393+0000 7f05dae00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d41014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.393+0000 7f05dae00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f05d410c930 con 0x7f05d4108540 2024-04-13T05:05:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.393+0000 7f05d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d41014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.393+0000 7f05d3e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d41014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47030/0 (socket says 172.21.15.57:47030) 2024-04-13T05:05:35.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.393+0000 7f05d3e00700 1 -- 172.21.15.57:0/4210030998 learned_addr learned my addr 172.21.15.57:0/4210030998 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:35.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.394+0000 7f05d3e00700 1 -- 172.21.15.57:0/4210030998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d4103370 con 0x7f05d4108540 2024-04-13T05:05:35.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.394+0000 7f05d3e00700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d41014d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006010 tx=0x7f05c4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:35.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.394+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c4026070 con 0x7f05d4108540 2024-04-13T05:05:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.394+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05c400bc00 con 0x7f05d4108540 2024-04-13T05:05:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.394+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c4003930 con 0x7f05d4108540 2024-04-13T05:05:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.394+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d4101a10 con 0x7f05d4108540 2024-04-13T05:05:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.395+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d4101ed0 con 0x7f05d4108540 2024-04-13T05:05:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.396+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d4103c00 con 0x7f05d4108540 2024-04-13T05:05:35.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.396+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f05c4029070 con 0x7f05d4108540 2024-04-13T05:05:35.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.396+0000 7f05d0c00700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05b80396b0 0x7f05b803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:35.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.396+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f05c401e070 con 0x7f05d4108540 2024-04-13T05:05:35.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.397+0000 7f05d3400700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05b80396b0 0x7f05b803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:35.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.397+0000 7f05d3400700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05b80396b0 0x7f05b803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006fd0 tx=0x7f05c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:35.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.402+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05c4013070 con 0x7f05d4108540 2024-04-13T05:05:35.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:35.640+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}) -- 0x7f05d41029b0 con 0x7f05b80396b0 2024-04-13T05:05:37.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:36 smithi057 ceph-mon[32331]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:37.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:36 smithi057 ceph-mon[32331]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:05:38.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:37 smithi057 ceph-mon[32331]: Deploying cephadm binary to smithi083 2024-04-13T05:05:39.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:38 smithi057 ceph-mon[32331]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:41.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:41 smithi057 ceph-mon[32331]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:42.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.392+0000 7f05d0c00700 1 -- 172.21.15.57:0/4210030998 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f05d41029b0 con 0x7f05b80396b0 2024-04-13T05:05:42.392 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi083' with addr '172.21.15.83' 2024-04-13T05:05:42.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05b80396b0 msgr2=0x7f05b803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:42.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05b80396b0 0x7f05b803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006fd0 tx=0x7f05c0006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:42.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 msgr2=0x7f05d41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:42.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d41014d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006010 tx=0x7f05c4005780 comp rx=0 tx=0).stop 2024-04-13T05:05:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 shutdown_connections 2024-04-13T05:05:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05b80396b0 0x7f05b803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 --2- 172.21.15.57:0/4210030998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05d4108540 0x7f05d41014d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.396+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 >> 172.21.15.57:0/4210030998 conn(0x7f05d4076f10 msgr2=0x7f05d40fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.397+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 shutdown_connections 2024-04-13T05:05:42.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:42.397+0000 7f05dae00700 1 -- 172.21.15.57:0/4210030998 wait complete. 2024-04-13T05:05:43.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:43 smithi057 ceph-mon[32331]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:43.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:43.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:05:43.202 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch host ls --format=json 2024-04-13T05:05:43.679 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:05:44.125 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:44 smithi057 ceph-mon[32331]: Added host smithi083 2024-04-13T05:05:44.126 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:45.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:45 smithi057 ceph-mon[32331]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:45.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.589+0000 7f91c3800700 1 -- 172.21.15.57:0/441024217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 msgr2=0x7f91bc0752c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.589+0000 7f91c3800700 1 --2- 172.21.15.57:0/441024217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc0752c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009910 tx=0x7f91b0009c20 comp rx=0 tx=0).stop 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.589+0000 7f91c3800700 1 -- 172.21.15.57:0/441024217 shutdown_connections 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.589+0000 7f91c3800700 1 --2- 172.21.15.57:0/441024217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc0752c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.589+0000 7f91c3800700 1 -- 172.21.15.57:0/441024217 >> 172.21.15.57:0/441024217 conn(0x7f91bc0fdff0 msgr2=0x7f91bc100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.590+0000 7f91c3800700 1 -- 172.21.15.57:0/441024217 shutdown_connections 2024-04-13T05:05:45.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.590+0000 7f91c3800700 1 -- 172.21.15.57:0/441024217 wait complete. 2024-04-13T05:05:45.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.591+0000 7f91c3800700 1 Processor -- start 2024-04-13T05:05:45.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.591+0000 7f91c3800700 1 -- start start 2024-04-13T05:05:45.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.592+0000 7f91c3800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:45.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.592+0000 7f91c3800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc10ec20 con 0x7f91bc074ee0 2024-04-13T05:05:45.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.592+0000 7f91c0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:45.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.592+0000 7f91c0e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc102420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56886/0 (socket says 172.21.15.57:56886) 2024-04-13T05:05:45.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.592+0000 7f91c0e00700 1 -- 172.21.15.57:0/3777913502 learned_addr learned my addr 172.21.15.57:0/3777913502 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:45.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.592+0000 7f91c0e00700 1 -- 172.21.15.57:0/3777913502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc106380 con 0x7f91bc074ee0 2024-04-13T05:05:45.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.593+0000 7f91c0e00700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc102420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91b0006010 tx=0x7f91b0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:45.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.593+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91b0026070 con 0x7f91bc074ee0 2024-04-13T05:05:45.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.593+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91b000bc00 con 0x7f91bc074ee0 2024-04-13T05:05:45.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.593+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91b0003910 con 0x7f91bc074ee0 2024-04-13T05:05:45.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.593+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc102960 con 0x7f91bc074ee0 2024-04-13T05:05:45.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.593+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc102dc0 con 0x7f91bc074ee0 2024-04-13T05:05:45.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.594+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f91b0029070 con 0x7f91bc074ee0 2024-04-13T05:05:45.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.595+0000 7f91b9600700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91a40396b0 0x7f91a403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:45.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.595+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f91b001e070 con 0x7f91bc074ee0 2024-04-13T05:05:45.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.595+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc076590 con 0x7f91bc074ee0 2024-04-13T05:05:45.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.595+0000 7f91bbe00700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91a40396b0 0x7f91a403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:45.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.596+0000 7f91bbe00700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91a40396b0 0x7f91a403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:45.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.601+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91b0030020 con 0x7f91bc074ee0 2024-04-13T05:05:45.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.839+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91bc106550 con 0x7f91a40396b0 2024-04-13T05:05:45.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.840+0000 7f91b9600700 1 -- 172.21.15.57:0/3777913502 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f91bc106550 con 0x7f91a40396b0 2024-04-13T05:05:45.840 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:45.840 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}] 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.843+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91a40396b0 msgr2=0x7f91a403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.843+0000 7f91c3800700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91a40396b0 0x7f91a403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.843+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 msgr2=0x7f91bc102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.843+0000 7f91c3800700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc102420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91b0006010 tx=0x7f91b0005780 comp rx=0 tx=0).stop 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.844+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 shutdown_connections 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.844+0000 7f91c3800700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91a40396b0 0x7f91a403bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:45.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.844+0000 7f91c3800700 1 --2- 172.21.15.57:0/3777913502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91bc074ee0 0x7f91bc102420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:45.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.844+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 >> 172.21.15.57:0/3777913502 conn(0x7f91bc0fdff0 msgr2=0x7f91bc106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:45.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.844+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 shutdown_connections 2024-04-13T05:05:45.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:45.844+0000 7f91c3800700 1 -- 172.21.15.57:0/3777913502 wait complete. 2024-04-13T05:05:46.599 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-04-13T05:05:46.600 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:05:46.600 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-04-13T05:05:46.657 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:05:46.657 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:05:46.752 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-04-13T05:05:46.752 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch host add smithi189 2024-04-13T05:05:47.215 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:05:47.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:47 smithi057 ceph-mon[32331]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:47.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:47 smithi057 ceph-mon[32331]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:05:47.478 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:47 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.178+0000 7f6e5b000700 1 -- 172.21.15.57:0/3428516204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 msgr2=0x7f6e54108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.178+0000 7f6e5b000700 1 --2- 172.21.15.57:0/3428516204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54108970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400b0a0 tx=0x7f6e4400b3b0 comp rx=0 tx=0).stop 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.179+0000 7f6e5b000700 1 -- 172.21.15.57:0/3428516204 shutdown_connections 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.179+0000 7f6e5b000700 1 --2- 172.21.15.57:0/3428516204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54108970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.179+0000 7f6e5b000700 1 -- 172.21.15.57:0/3428516204 >> 172.21.15.57:0/3428516204 conn(0x7f6e540fdff0 msgr2=0x7f6e54100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.179+0000 7f6e5b000700 1 -- 172.21.15.57:0/3428516204 shutdown_connections 2024-04-13T05:05:49.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.179+0000 7f6e5b000700 1 -- 172.21.15.57:0/3428516204 wait complete. 2024-04-13T05:05:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.180+0000 7f6e5b000700 1 Processor -- start 2024-04-13T05:05:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.180+0000 7f6e5b000700 1 -- start start 2024-04-13T05:05:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.181+0000 7f6e5b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:49.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.181+0000 7f6e5b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5410ca00 con 0x7f6e54108590 2024-04-13T05:05:49.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.181+0000 7f6e53e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:49.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.181+0000 7f6e53e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54074db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56908/0 (socket says 172.21.15.57:56908) 2024-04-13T05:05:49.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.181+0000 7f6e53e00700 1 -- 172.21.15.57:0/3522372341 learned_addr learned my addr 172.21.15.57:0/3522372341 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:49.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.181+0000 7f6e53e00700 1 -- 172.21.15.57:0/3522372341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e54076ca0 con 0x7f6e54108590 2024-04-13T05:05:49.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.182+0000 7f6e53e00700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54074db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e440062a0 tx=0x7f6e44003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.185+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e44017040 con 0x7f6e54108590 2024-04-13T05:05:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.185+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e44022070 con 0x7f6e54108590 2024-04-13T05:05:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.185+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e540752f0 con 0x7f6e54108590 2024-04-13T05:05:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.185+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e4401d5e0 con 0x7f6e54108590 2024-04-13T05:05:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.186+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e540757b0 con 0x7f6e54108590 2024-04-13T05:05:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.187+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6e44004040 con 0x7f6e54108590 2024-04-13T05:05:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.187+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e54103c00 con 0x7f6e54108590 2024-04-13T05:05:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.187+0000 7f6e50c00700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6e380396b0 0x7f6e3803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:49.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.187+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6e4400f050 con 0x7f6e54108590 2024-04-13T05:05:49.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.188+0000 7f6e53400700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6e380396b0 0x7f6e3803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:49.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.188+0000 7f6e53400700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6e380396b0 0x7f6e3803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006fd0 tx=0x7f6e40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:49.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.193+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e4401d890 con 0x7f6e54108590 2024-04-13T05:05:49.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:49 smithi057 ceph-mon[32331]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:49.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:49.429+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7f6e54076350 con 0x7f6e380396b0 2024-04-13T05:05:51.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:51 smithi057 ceph-mon[32331]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:05:51.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:51 smithi057 ceph-mon[32331]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:52.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: Deploying cephadm binary to smithi189 2024-04-13T05:05:52.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:52.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:52.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:52.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:52.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:05:52.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:05:52.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:05:53.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:53 smithi057 ceph-mon[32331]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-13T05:05:53.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:53 smithi057 ceph-mon[32331]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:54.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:54 smithi057 ceph-mon[32331]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:05:55.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:55 smithi057 ceph-mon[32331]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:56.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.208+0000 7f6e50c00700 1 -- 172.21.15.57:0/3522372341 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6e54076350 con 0x7f6e380396b0 2024-04-13T05:05:56.208 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6e380396b0 msgr2=0x7f6e3803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6e380396b0 0x7f6e3803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006fd0 tx=0x7f6e40006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 msgr2=0x7f6e54074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54074db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e440062a0 tx=0x7f6e44003ce0 comp rx=0 tx=0).stop 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 shutdown_connections 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6e380396b0 0x7f6e3803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:56.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 --2- 172.21.15.57:0/3522372341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e54108590 0x7f6e54074db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:56.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 >> 172.21.15.57:0/3522372341 conn(0x7f6e540fdff0 msgr2=0x7f6e540ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:56.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 shutdown_connections 2024-04-13T05:05:56.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:56.212+0000 7f6e5b000700 1 -- 172.21.15.57:0/3522372341 wait complete. 2024-04-13T05:05:57.015 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch host ls --format=json 2024-04-13T05:05:57.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:57 smithi057 ceph-mon[32331]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:57.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:57 smithi057 ceph-mon[32331]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:05:57.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:57.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:57.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:57.466 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:05:58.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:58 smithi057 ceph-mon[32331]: Added host smithi189 2024-04-13T05:05:58.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:58 smithi057 ceph-mon[32331]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:05:58.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:58.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:58 smithi057 ceph-mon[32331]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:05:58.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 -- 172.21.15.57:0/3069845073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 msgr2=0x7fcdcc10ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 --2- 172.21.15.57:0/3069845073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc10ddb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000b0a0 tx=0x7fcdc000b3b0 comp rx=0 tx=0).stop 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 -- 172.21.15.57:0/3069845073 shutdown_connections 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 --2- 172.21.15.57:0/3069845073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc10ddb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 -- 172.21.15.57:0/3069845073 >> 172.21.15.57:0/3069845073 conn(0x7fcdcc078250 msgr2=0x7fcdcc07a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 -- 172.21.15.57:0/3069845073 shutdown_connections 2024-04-13T05:05:59.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.443+0000 7fcdd4000700 1 -- 172.21.15.57:0/3069845073 wait complete. 2024-04-13T05:05:59.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.444+0000 7fcdd4000700 1 Processor -- start 2024-04-13T05:05:59.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.445+0000 7fcdd4000700 1 -- start start 2024-04-13T05:05:59.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.445+0000 7fcdd4000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:59.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.445+0000 7fcdd4000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc112fd0 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.445+0000 7fcdd1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:59.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.446+0000 7fcdd1600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35890/0 (socket says 172.21.15.57:35890) 2024-04-13T05:05:59.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.446+0000 7fcdd1600700 1 -- 172.21.15.57:0/1969128865 learned_addr learned my addr 172.21.15.57:0/1969128865 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:05:59.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.446+0000 7fcdd1600700 1 -- 172.21.15.57:0/1969128865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc19e9a0 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.446+0000 7fcdd1600700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc19e460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0015040 tx=0x7fcdc0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:59.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.446+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc0017040 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.447+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdc0004040 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.447+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc19ec30 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.447+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc001d5a0 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.447+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1a2d30 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.448+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcdc0007a70 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.448+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc10a1f0 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.448+0000 7fcdca000700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcdb4039650 0x7fcdb403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:05:59.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.448+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcdc000f050 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.449+0000 7fcdd0c00700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcdb4039650 0x7fcdb403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:05:59.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.449+0000 7fcdd0c00700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcdb4039650 0x7fcdb403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8006fd0 tx=0x7fcdb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:05:59.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.454+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdc0007d60 con 0x7fcdcc10d9d0 2024-04-13T05:05:59.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:59.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:05:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:05:59.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.696+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdcc064110 con 0x7fcdb4039650 2024-04-13T05:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.698+0000 7fcdca000700 1 -- 172.21.15.57:0/1969128865 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fcdcc064110 con 0x7fcdb4039650 2024-04-13T05:05:59.697 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:05:59.697 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-04-13T05:05:59.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcdb4039650 msgr2=0x7fcdb403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:59.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcdb4039650 0x7fcdb403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8006fd0 tx=0x7fcdb8006e40 comp rx=0 tx=0).stop 2024-04-13T05:05:59.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 msgr2=0x7fcdcc19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:05:59.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc19e460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0015040 tx=0x7fcdc0009790 comp rx=0 tx=0).stop 2024-04-13T05:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 shutdown_connections 2024-04-13T05:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcdb4039650 0x7fcdb403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 --2- 172.21.15.57:0/1969128865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdcc10d9d0 0x7fcdcc19e460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 >> 172.21.15.57:0/1969128865 conn(0x7fcdcc078250 msgr2=0x7fcdcc110570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 shutdown_connections 2024-04-13T05:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:05:59.701+0000 7fcdd4000700 1 -- 172.21.15.57:0/1969128865 wait complete. 2024-04-13T05:06:00.455 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-13T05:06:00.455 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd crush tunables default 2024-04-13T05:06:00.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:00 smithi057 ceph-mon[32331]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:00.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:00 smithi057 ceph-mon[32331]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:06:00.910 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:06:02.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:02.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:02 smithi057 ceph-mon[32331]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.967+0000 7f8a8a000700 1 -- 172.21.15.57:0/161849231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 msgr2=0x7f8a84108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.967+0000 7f8a8a000700 1 --2- 172.21.15.57:0/161849231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a84108920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009910 tx=0x7f8a74009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.967+0000 7f8a8a000700 1 -- 172.21.15.57:0/161849231 shutdown_connections 2024-04-13T05:06:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.967+0000 7f8a8a000700 1 --2- 172.21.15.57:0/161849231 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a84108920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.968+0000 7f8a8a000700 1 -- 172.21.15.57:0/161849231 >> 172.21.15.57:0/161849231 conn(0x7f8a84076f10 msgr2=0x7f8a84077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.968+0000 7f8a8a000700 1 -- 172.21.15.57:0/161849231 shutdown_connections 2024-04-13T05:06:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.968+0000 7f8a8a000700 1 -- 172.21.15.57:0/161849231 wait complete. 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.969+0000 7f8a8a000700 1 Processor -- start 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.969+0000 7f8a8a000700 1 -- start start 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.969+0000 7f8a8a000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a8419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.969+0000 7f8a8a000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8410c930 con 0x7f8a84108540 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.970+0000 7f8a83400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a8419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.970+0000 7f8a83400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a8419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35908/0 (socket says 172.21.15.57:35908) 2024-04-13T05:06:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.970+0000 7f8a83400700 1 -- 172.21.15.57:0/2176083486 learned_addr learned my addr 172.21.15.57:0/2176083486 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.970+0000 7f8a83400700 1 -- 172.21.15.57:0/2176083486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8419a5a0 con 0x7f8a84108540 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.970+0000 7f8a83400700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a8419a060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a740056d0 tx=0x7f8a740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.971+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a74026070 con 0x7f8a84108540 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.971+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a7400bc00 con 0x7f8a84108540 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.971+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a740039b0 con 0x7f8a84108540 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.971+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8419a890 con 0x7f8a84108540 2024-04-13T05:06:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.971+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8419e930 con 0x7f8a84108540 2024-04-13T05:06:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.972+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84103c00 con 0x7f8a84108540 2024-04-13T05:06:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.972+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8a74029070 con 0x7f8a84108540 2024-04-13T05:06:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.973+0000 7f8a7be00700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a68039700 0x7f8a6803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.973+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a7401e070 con 0x7f8a84108540 2024-04-13T05:06:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.973+0000 7f8a82a00700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a68039700 0x7f8a6803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.974+0000 7f8a82a00700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a68039700 0x7f8a6803bbc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:02.978+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a74013070 con 0x7f8a84108540 2024-04-13T05:06:03.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.216+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8a84064110 con 0x7f8a84108540 2024-04-13T05:06:03.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.385+0000 7f8a7be00700 1 -- 172.21.15.57:0/2176083486 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8a7400e3e0 con 0x7f8a84108540 2024-04-13T05:06:03.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a68039700 msgr2=0x7f8a6803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:03.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a68039700 0x7f8a6803bbc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70006e40 comp rx=0 tx=0).stop 2024-04-13T05:06:03.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 msgr2=0x7f8a8419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a8419a060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a740056d0 tx=0x7f8a740057b0 comp rx=0 tx=0).stop 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 shutdown_connections 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a68039700 0x7f8a6803bbc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 --2- 172.21.15.57:0/2176083486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a84108540 0x7f8a8419a060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 >> 172.21.15.57:0/2176083486 conn(0x7f8a84076f10 msgr2=0x7f8a84104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.390+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 shutdown_connections 2024-04-13T05:06:03.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:03.391+0000 7f8a8a000700 1 -- 172.21.15.57:0/2176083486 wait complete. 2024-04-13T05:06:03.391 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2024-04-13T05:06:03.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:03 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2176083486' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-13T05:06:04.142 INFO:tasks.cephadm:Adding mon.a on smithi057 2024-04-13T05:06:04.142 INFO:tasks.cephadm:Adding mon.b on smithi083 2024-04-13T05:06:04.142 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-04-13T05:06:04.142 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch apply mon '3;smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c' 2024-04-13T05:06:04.572 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-13T05:06:04.697 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-13T05:06:04.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:04 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2176083486' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-13T05:06:04.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:04 smithi057 ceph-mon[32331]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:04.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:04 smithi057 ceph-mon[32331]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:06:06.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:06.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:07.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.331+0000 7f5f88200700 1 -- 172.21.15.189:0/853978810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 msgr2=0x7f5f80105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:07.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.331+0000 7f5f88200700 1 --2- 172.21.15.189:0/853978810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80105390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009910 tx=0x7f5f74009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:07.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.331+0000 7f5f88200700 1 -- 172.21.15.189:0/853978810 shutdown_connections 2024-04-13T05:06:07.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.331+0000 7f5f88200700 1 --2- 172.21.15.189:0/853978810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80105390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:07.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.331+0000 7f5f88200700 1 -- 172.21.15.189:0/853978810 >> 172.21.15.189:0/853978810 conn(0x7f5f800fc9b0 msgr2=0x7f5f800fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:07.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.331+0000 7f5f88200700 1 -- 172.21.15.189:0/853978810 shutdown_connections 2024-04-13T05:06:07.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.332+0000 7f5f88200700 1 -- 172.21.15.189:0/853978810 wait complete. 2024-04-13T05:06:07.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.333+0000 7f5f88200700 1 Processor -- start 2024-04-13T05:06:07.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.333+0000 7f5f88200700 1 -- start start 2024-04-13T05:06:07.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.333+0000 7f5f88200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:07.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.333+0000 7f5f88200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5f801083f0 con 0x7f5f80102fa0 2024-04-13T05:06:07.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.334+0000 7f5f85800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:07.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.334+0000 7f5f85800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.189:59912/0 (socket says 172.21.15.189:59912) 2024-04-13T05:06:07.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.334+0000 7f5f85800700 1 -- 172.21.15.189:0/2886538286 learned_addr learned my addr 172.21.15.189:0/2886538286 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:06:07.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.334+0000 7f5f85800700 1 -- 172.21.15.189:0/2886538286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8019a220 con 0x7f5f80102fa0 2024-04-13T05:06:07.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.335+0000 7f5f85800700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80199ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5f74005190 tx=0x7f5f74004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:07.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.335+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f74027070 con 0x7f5f80102fa0 2024-04-13T05:06:07.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.335+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8019a510 con 0x7f5f80102fa0 2024-04-13T05:06:07.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.336+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8019d190 con 0x7f5f80102fa0 2024-04-13T05:06:07.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.336+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f74004710 con 0x7f5f80102fa0 2024-04-13T05:06:07.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.336+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f74031060 con 0x7f5f80102fa0 2024-04-13T05:06:07.336 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.337+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f60005330 con 0x7f5f80102fa0 2024-04-13T05:06:07.336 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.337+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5f7402a070 con 0x7f5f80102fa0 2024-04-13T05:06:07.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.338+0000 7f5f7e000700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5f680396b0 0x7f5f6803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:07.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.338+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5f7401e030 con 0x7f5f80102fa0 2024-04-13T05:06:07.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.338+0000 7f5f84e00700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5f680396b0 0x7f5f6803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:07.339 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.340+0000 7f5f84e00700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5f680396b0 0x7f5f6803bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c006fd0 tx=0x7f5f6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:07.353 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.353+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f74013070 con 0x7f5f80102fa0 2024-04-13T05:06:07.596 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.597+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) -- 0x7f5f60000c80 con 0x7f5f680396b0 2024-04-13T05:06:07.616 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.617+0000 7f5f7e000700 1 -- 172.21.15.189:0/2886538286 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5f60000c80 con 0x7f5f680396b0 2024-04-13T05:06:07.616 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-04-13T05:06:07.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5f680396b0 msgr2=0x7f5f6803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:07.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5f680396b0 0x7f5f6803bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c006fd0 tx=0x7f5f6c006e40 comp rx=0 tx=0).stop 2024-04-13T05:06:07.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 msgr2=0x7f5f80199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80199ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5f74005190 tx=0x7f5f74004f80 comp rx=0 tx=0).stop 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 shutdown_connections 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5f680396b0 0x7f5f6803bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 --2- 172.21.15.189:0/2886538286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80102fa0 0x7f5f80199ce0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 >> 172.21.15.189:0/2886538286 conn(0x7f5f800fc9b0 msgr2=0x7f5f800fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.620+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 shutdown_connections 2024-04-13T05:06:07.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:07.621+0000 7f5f88200700 1 -- 172.21.15.189:0/2886538286 wait complete. 2024-04-13T05:06:07.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:07 smithi057 ceph-mon[32331]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-13T05:06:07.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:07 smithi057 ceph-mon[32331]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:08.320 DEBUG:teuthology.orchestra.run.smithi083:mon.b> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.b.service 2024-04-13T05:06:08.328 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.c.service 2024-04-13T05:06:08.332 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-13T05:06:08.332 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-13T05:06:08.767 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:09.012 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:08 smithi057 ceph-mon[32331]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:09.012 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:08 smithi057 ceph-mon[32331]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:09.012 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:08 smithi057 ceph-mon[32331]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c;count:3 2024-04-13T05:06:09.012 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:08 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:09.013 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:08 smithi057 ceph-mon[32331]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:10.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:10 smithi057 ceph-mon[32331]: Deploying daemon mon.c on smithi189 2024-04-13T05:06:10.732 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.732+0000 7fc5e4400700 1 -- 172.21.15.189:0/4094092134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 msgr2=0x7fc5dc10ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:10.732 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.732+0000 7fc5e4400700 1 --2- 172.21.15.189:0/4094092134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc10ddc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009910 tx=0x7fc5d0009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:10.732 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.732+0000 7fc5e4400700 1 -- 172.21.15.189:0/4094092134 shutdown_connections 2024-04-13T05:06:10.733 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.732+0000 7fc5e4400700 1 --2- 172.21.15.189:0/4094092134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc10ddc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:10.733 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.732+0000 7fc5e4400700 1 -- 172.21.15.189:0/4094092134 >> 172.21.15.189:0/4094092134 conn(0x7fc5dc078340 msgr2=0x7fc5dc07a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:10.733 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.733+0000 7fc5e4400700 1 -- 172.21.15.189:0/4094092134 shutdown_connections 2024-04-13T05:06:10.733 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.733+0000 7fc5e4400700 1 -- 172.21.15.189:0/4094092134 wait complete. 2024-04-13T05:06:10.733 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.734+0000 7fc5e4400700 1 Processor -- start 2024-04-13T05:06:10.733 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.734+0000 7fc5e4400700 1 -- start start 2024-04-13T05:06:10.734 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.734+0000 7fc5e4400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:10.734 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.734+0000 7fc5e4400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc112fe0 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.734 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.735+0000 7fc5e1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:10.734 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.735+0000 7fc5e1a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.189:59932/0 (socket says 172.21.15.189:59932) 2024-04-13T05:06:10.734 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.735+0000 7fc5e1a00700 1 -- 172.21.15.189:0/2885725841 learned_addr learned my addr 172.21.15.189:0/2885725841 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:06:10.735 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.736+0000 7fc5e1a00700 1 -- 172.21.15.189:0/2885725841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc19ea30 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.735 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.736+0000 7fc5e1a00700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc19e4f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0006010 tx=0x7fc5d0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:10.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.736+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5d0026070 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.736+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc19ed20 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.737+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1a2dc0 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.737+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5d000bba0 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.737+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5d0004830 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.737 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.738+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5bc005330 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.737 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.738+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc5d0029070 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.739+0000 7fc5da000700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc5c40397f0 0x7fc5c403bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:10.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.739+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc5d001e070 con 0x7fc5dc10d9e0 2024-04-13T05:06:10.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.739+0000 7fc5e1000700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc5c40397f0 0x7fc5c403bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:10.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.741+0000 7fc5e1000700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc5c40397f0 0x7fc5c403bcb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8006fd0 tx=0x7fc5c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:10.754 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:10.754+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5d0013070 con 0x7fc5dc10d9e0 2024-04-13T05:06:11.072 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.072+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc5bc0058f0 con 0x7fc5dc10d9e0 2024-04-13T05:06:11.073 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.074+0000 7fc5da000700 1 -- 172.21.15.189:0/2885725841 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fc5d000bd50 con 0x7fc5dc10d9e0 2024-04-13T05:06:11.074 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:06:11.074 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","modified":"2024-04-13T05:03:56.948897Z","created":"2024-04-13T05:03:56.948897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc5c40397f0 msgr2=0x7fc5c403bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc5c40397f0 0x7fc5c403bcb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8006fd0 tx=0x7fc5c8006e40 comp rx=0 tx=0).stop 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 msgr2=0x7fc5dc19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc19e4f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0006010 tx=0x7fc5d0005780 comp rx=0 tx=0).stop 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 shutdown_connections 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc5c40397f0 0x7fc5c403bcb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 --2- 172.21.15.189:0/2885725841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5dc10d9e0 0x7fc5dc19e4f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 >> 172.21.15.189:0/2885725841 conn(0x7fc5dc078340 msgr2=0x7fc5dc1105d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 shutdown_connections 2024-04-13T05:06:11.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:11.077+0000 7fc5e4400700 1 -- 172.21.15.189:0/2885725841 wait complete. 2024-04-13T05:06:11.078 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-04-13T05:06:11.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:11 smithi057 ceph-mon[32331]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:11.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:11 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/2885725841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:13.035 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-13T05:06:13.035 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-13T05:06:13.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:13 smithi057 ceph-mon[32331]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:13.702 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:06:15.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:15 smithi057 ceph-mon[32331]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:15.914 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:15 smithi189 podman[35485]: 2024-04-13 05:06:15.782023468 +0000 UTC m=+0.129053744 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:06:16.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:15 smithi189 podman[35485]: 2024-04-13 05:06:15.987263175 +0000 UTC m=+0.334293428 container create c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:06:16.307 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 -- 172.21.15.189:0/3870413887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 msgr2=0x7f26f810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:16.307 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 --2- 172.21.15.189:0/3870413887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f810aa60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009910 tx=0x7f26ec009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:16.307 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 -- 172.21.15.189:0/3870413887 shutdown_connections 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 --2- 172.21.15.189:0/3870413887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f810aa60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 -- 172.21.15.189:0/3870413887 >> 172.21.15.189:0/3870413887 conn(0x7f26f8100100 msgr2=0x7f26f8102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 -- 172.21.15.189:0/3870413887 shutdown_connections 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.307+0000 7f26fe000700 1 -- 172.21.15.189:0/3870413887 wait complete. 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.309+0000 7f26fe000700 1 Processor -- start 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.309+0000 7f26fe000700 1 -- start start 2024-04-13T05:06:16.308 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.309+0000 7f26fe000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:16.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.309+0000 7f26fe000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f26f810eae0 con 0x7f26f810a680 2024-04-13T05:06:16.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.310+0000 7f26fcc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:16.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.310+0000 7f26fcc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.189:59952/0 (socket says 172.21.15.189:59952) 2024-04-13T05:06:16.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.310+0000 7f26fcc00700 1 -- 172.21.15.189:0/1772809328 learned_addr learned my addr 172.21.15.189:0/1772809328 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:06:16.310 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.311+0000 7f26fcc00700 1 -- 172.21.15.189:0/1772809328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f819a4e0 con 0x7f26f810a680 2024-04-13T05:06:16.310 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.311+0000 7f26fcc00700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f8199fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0092e0 tx=0x7f26ec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:16.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.312+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26ec026070 con 0x7f26f810a680 2024-04-13T05:06:16.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.312+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f819a7d0 con 0x7f26f810a680 2024-04-13T05:06:16.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.312+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26ec00b6f0 con 0x7f26f810a680 2024-04-13T05:06:16.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.312+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26ec0037f0 con 0x7f26f810a680 2024-04-13T05:06:16.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.312+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f819e870 con 0x7f26f810a680 2024-04-13T05:06:16.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.313+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8105d30 con 0x7f26f810a680 2024-04-13T05:06:16.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.314+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f26ec029070 con 0x7f26f810a680 2024-04-13T05:06:16.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.314+0000 7f26f5600700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e0039650 0x7f26e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:16.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.314+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f26ec01e070 con 0x7f26f810a680 2024-04-13T05:06:16.314 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.315+0000 7f26f7e00700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e0039650 0x7f26e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:16.315 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.316+0000 7f26f7e00700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e0039650 0x7f26e003bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f26e4006fd0 tx=0x7f26e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:16.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.330+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26ec013070 con 0x7f26f810a680 2024-04-13T05:06:16.482 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.483+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f26ec027260 con 0x7f26f810a680 2024-04-13T05:06:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 podman[35485]: 2024-04-13 05:06:16.359197882 +0000 UTC m=+0.706228141 container init c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-13T05:06:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 podman[35485]: 2024-04-13 05:06:16.369458887 +0000 UTC m=+0.716489147 container start c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.schema-version=1.0) 2024-04-13T05:06:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: set uid:gid to 167:167 (ceph:ceph) 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev), process ceph-mon, pid 2 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pidfile_write: ignore empty --pid-file 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: load: jerasure load: lrc 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: RocksDB version: 7.9.2 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Git sha 0 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Compile date 2024-04-12 04:47:35 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: DB SUMMARY 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: DB Session ID: HAIHMRRA7LNJHADJKWXP 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: CURRENT file: CURRENT 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: IDENTITY file: IDENTITY 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-13T05:06:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.error_if_exists: 0 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.create_if_missing: 0 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.paranoid_checks: 1 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.env: 0x55976836dcc0 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.fs: PosixFileSystem 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.info_log: 0x55977f29dc40 2024-04-13T05:06:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_file_opening_threads: 16 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.statistics: (nil) 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.use_fsync: 0 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_log_file_size: 0 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.keep_log_file_num: 1000 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.recycle_log_file_num: 0 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.allow_fallocate: 1 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.allow_mmap_reads: 0 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.allow_mmap_writes: 0 2024-04-13T05:06:16.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.use_direct_reads: 0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.create_missing_column_families: 0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.db_log_dir: 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.wal_dir: 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.advise_random_on_open: 1 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.db_write_buffer_size: 0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.write_buffer_manager: 0x55977f26e1e0 2024-04-13T05:06:16.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.rate_limiter: (nil) 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.wal_recovery_mode: 2 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.enable_thread_tracking: 0 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.enable_pipelined_write: 0 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.unordered_write: 0 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.row_cache: None 2024-04-13T05:06:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.wal_filter: None 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.allow_ingest_behind: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.two_write_queues: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.manual_wal_flush: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.wal_compression: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.atomic_flush: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-13T05:06:16.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.log_readahead_size: 0 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.best_efforts_recovery: 0 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.allow_data_in_errors: 0 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.db_host_id: __hostname__ 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_background_jobs: 2 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_background_compactions: -1 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_subcompactions: 1 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-13T05:06:16.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_total_wal_size: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_open_files: -1 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bytes_per_sync: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_readahead_size: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_background_flushes: -1 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Compression algorithms supported: 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kZSTD supported: 0 2024-04-13T05:06:16.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kXpressCompression supported: 0 2024-04-13T05:06:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kLZ4HCCompression supported: 1 2024-04-13T05:06:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kZlibCompression supported: 1 2024-04-13T05:06:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kSnappyCompression supported: 1 2024-04-13T05:06:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kLZ4Compression supported: 1 2024-04-13T05:06:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: kBZip2Compression supported: 0 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.merge_operator: 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_filter: None 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_filter_factory: None 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.sst_partitioner_factory: None 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55977f29de00) 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks: 1 2024-04-13T05:06:16.591 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_top_level_index_and_filter: 1 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_type: 0 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_index_type: 0 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_shortening: 1 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: checksum: 4 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: no_block_cache: 0 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache: 0x55977f2a1090 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_name: BinnedLRUCache 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_options: 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: capacity : 536870912 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_shard_bits : 4 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: strict_capacity_limit : 0 2024-04-13T05:06:16.592 INFO:journalctl@ceph.mon.c.smithi189.stdout: high_pri_pool_ratio: 0.000 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_compressed: (nil) 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: persistent_cache: (nil) 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size: 4096 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size_deviation: 10 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_restart_interval: 16 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_block_restart_interval: 1 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: metadata_block_size: 4096 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: partition_filters: 0 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: use_delta_encoding: 1 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: filter_policy: bloomfilter 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: whole_key_filtering: 1 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: verify_compression: 0 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: read_amp_bytes_per_bit: 0 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: format_version: 5 2024-04-13T05:06:16.593 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_index_compression: 1 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_align: 0 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout: max_auto_readahead_size: 262144 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout: prepopulate_block_cache: 0 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout: initial_auto_readahead_size: 8192 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_file_reads_for_auto_readahead: 2 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.write_buffer_size: 33554432 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_write_buffer_number: 2 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression: NoCompression 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression: Disabled 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.prefix_extractor: nullptr 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.num_levels: 7 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-13T05:06:16.594 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.level: 32767 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.strategy: 0 2024-04-13T05:06:16.595 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.enabled: false 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.target_file_size_base: 67108864 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-13T05:06:16.596 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-13T05:06:16.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-13T05:06:16.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-13T05:06:16.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.arena_block_size: 1048576 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.disable_auto_compactions: 0 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-13T05:06:16.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.inplace_update_support: 0 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.bloom_locality: 0 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.max_successive_merges: 0 2024-04-13T05:06:16.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.paranoid_file_checks: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.force_consistency_checks: 1 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.report_bg_io_stats: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.ttl: 2592000 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.enable_blob_files: false 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.min_blob_size: 0 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.blob_file_size: 268435456 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-13T05:06:16.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.blob_file_starting_level: 0 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b51a57d3-2c3d-436d-a64a-24e54671e72b 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984776426812, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984776427828, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712984776, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b51a57d3-2c3d-436d-a64a-24e54671e72b", "db_session_id": "HAIHMRRA7LNJHADJKWXP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984776428067, "job": 1, "event": "recovery_finished"} 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55977f390000 2024-04-13T05:06:16.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: DB pointer 0x55977f37a000 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** DB Stats ** 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-13T05:06:16.602 INFO:journalctl@ceph.mon.c.smithi189.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-13T05:06:16.603 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache BinnedLRUCache@0x55977f2a1090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.8e-05 secs_since: 0 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: using public_addr v2:172.21.15.189:0/0 -> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: starting mon.c rank -1 at public addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] at bind addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(???) e0 preinit fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:16.604 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 bash[35485]: c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 systemd[1]: Started Ceph mon.c for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).mds e1 new map 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout: e1 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout: legacy client fscid: -1 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout: No filesystems configured 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-13T05:06:16.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-13T05:06:16.606 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-13T05:06:16.606 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:06:16.606 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:06:16.606 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:06:16.606 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Deploying cephadm binary to smithi083 2024-04-13T05:06:16.606 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Added host smithi083 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:16.607 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Deploying cephadm binary to smithi189 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Added host smithi189 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2176083486' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2176083486' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:16.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c;count:3 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: Deploying daemon mon.c on smithi189 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/2885725841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.611 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:16.612 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:16 smithi189 ceph-mon[35531]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-13T05:06:16.680 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:16.680+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f26f819b300 con 0x7f26f810a680 2024-04-13T05:06:21.516 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.516+0000 7f26f5600700 1 -- 172.21.15.189:0/1772809328 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f26ec030030 con 0x7f26f810a680 2024-04-13T05:06:21.516 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:06:21.516 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","modified":"2024-04-13T05:06:16.478295Z","created":"2024-04-13T05:03:56.948897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-13T05:06:21.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.520+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e0039650 msgr2=0x7f26e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:21.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.520+0000 7f26fe000700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e0039650 0x7f26e003bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f26e4006fd0 tx=0x7f26e4006e40 comp rx=0 tx=0).stop 2024-04-13T05:06:21.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.520+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 msgr2=0x7f26f8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:21.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.520+0000 7f26fe000700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f8199fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0092e0 tx=0x7f26ec005780 comp rx=0 tx=0).stop 2024-04-13T05:06:21.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.521+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 shutdown_connections 2024-04-13T05:06:21.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.521+0000 7f26fe000700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e0039650 0x7f26e003bb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:21.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.521+0000 7f26fe000700 1 --2- 172.21.15.189:0/1772809328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26f810a680 0x7f26f8199fa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:21.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.521+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 >> 172.21.15.189:0/1772809328 conn(0x7f26f8100100 msgr2=0x7f26f8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:21.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.521+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 shutdown_connections 2024-04-13T05:06:21.521 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:21.521+0000 7f26fe000700 1 -- 172.21.15.189:0/1772809328 wait complete. 2024-04-13T05:06:21.522 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-04-13T05:06:21.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: mon.a calling monitor election 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1772809328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: mon.c calling monitor election 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-13T05:06:21.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: monmap epoch 2 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: last_changed 2024-04-13T05:06:16.478295+0000 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: min_mon_release 19 (squid) 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: election_strategy: 1 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: fsmap 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: mgrmap e13: x(active, since 70s) 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: overall HEALTH_OK 2024-04-13T05:06:21.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:21.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:21.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:21.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:22.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: mon.a calling monitor election 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1772809328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: mon.c calling monitor election 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: monmap epoch 2 2024-04-13T05:06:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: last_changed 2024-04-13T05:06:16.478295+0000 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: min_mon_release 19 (squid) 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: election_strategy: 1 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: fsmap 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: mgrmap e13: x(active, since 70s) 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: overall HEALTH_OK 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:22.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:22 smithi189 ceph-mon[35531]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:22.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:22 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:22.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:22 smithi189 ceph-mon[35531]: Deploying daemon mon.b on smithi083 2024-04-13T05:06:22.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:22 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:23.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:22 smithi057 ceph-mon[32331]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:23.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:22 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:23.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:22 smithi057 ceph-mon[32331]: Deploying daemon mon.b on smithi083 2024-04-13T05:06:23.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:22 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:23.272 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-13T05:06:23.272 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-13T05:06:23.711 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:06:23.779 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:06:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:06:23.480+0000 7fb1c3200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-13T05:06:24.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:24 smithi057 ceph-mon[32331]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:24.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:24 smithi189 ceph-mon[35531]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.700+0000 7f7135e00700 1 -- 172.21.15.189:0/2852355600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7130108670 msgr2=0x7f7130108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.700+0000 7f7135e00700 1 --2- 172.21.15.189:0/2852355600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7130108670 0x7f7130108a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7120009910 tx=0x7f7120009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.701+0000 7f7135e00700 1 -- 172.21.15.189:0/2852355600 shutdown_connections 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.701+0000 7f7135e00700 1 --2- 172.21.15.189:0/2852355600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7130108670 0x7f7130108a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.701+0000 7f7135e00700 1 -- 172.21.15.189:0/2852355600 >> 172.21.15.189:0/2852355600 conn(0x7f71300fe0d0 msgr2=0x7f71301004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.701+0000 7f7135e00700 1 -- 172.21.15.189:0/2852355600 shutdown_connections 2024-04-13T05:06:25.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.701+0000 7f7135e00700 1 -- 172.21.15.189:0/2852355600 wait complete. 2024-04-13T05:06:25.701 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.702+0000 7f7135e00700 1 Processor -- start 2024-04-13T05:06:25.701 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.702+0000 7f7135e00700 1 -- start start 2024-04-13T05:06:25.701 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.702+0000 7f7135e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:25.701 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.702+0000 7f7135e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71300769f0 0x7f71300752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:25.701 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.702+0000 7f7135e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f713010cad0 con 0x7f71300769f0 2024-04-13T05:06:25.701 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.702+0000 7f7135e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f713010c950 con 0x7f7130108670 2024-04-13T05:06:25.702 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56132/0 (socket says 172.21.15.189:56132) 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 -- 172.21.15.189:0/1854502588 learned_addr learned my addr 172.21.15.189:0/1854502588 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712e000700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71300769f0 0x7f71300752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 -- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 msgr2=0x7f7130074db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 -- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 msgr2=0x7f7130074db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.703+0000 7f712ea00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.704+0000 7f712e000700 1 -- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 msgr2=0x7f7130074db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.704+0000 7f712e000700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.704+0000 7f712e000700 1 -- 172.21.15.189:0/1854502588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7130075830 con 0x7f71300769f0 2024-04-13T05:06:25.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.704+0000 7f712e000700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71300769f0 0x7f71300752f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b500 tx=0x7f711c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:25.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.705+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f711c011630 con 0x7f71300769f0 2024-04-13T05:06:25.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.705+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7130075b20 con 0x7f71300769f0 2024-04-13T05:06:25.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.705+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f711c011c80 con 0x7f71300769f0 2024-04-13T05:06:25.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.705+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f711c010d10 con 0x7f71300769f0 2024-04-13T05:06:25.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.705+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71301a8d90 con 0x7f71300769f0 2024-04-13T05:06:25.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.706+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130103d20 con 0x7f71300769f0 2024-04-13T05:06:25.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.707+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f711c018020 con 0x7f71300769f0 2024-04-13T05:06:25.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.707+0000 7f7134a00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71140393f0 0x7f711403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:25.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.707+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f711c04bbb0 con 0x7f71300769f0 2024-04-13T05:06:25.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.708+0000 7f712ea00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71140393f0 0x7f711403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:25.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.709+0000 7f712ea00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71140393f0 0x7f711403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7120009750 tx=0x7f712001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:25.724 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:25.726+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f711c01f070 con 0x7f71300769f0 2024-04-13T05:06:26.043 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.045+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f71301a90a0 con 0x7f71300769f0 2024-04-13T05:06:26.044 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.046+0000 7f7134a00700 1 -- 172.21.15.189:0/1854502588 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f711c018c10 con 0x7f71300769f0 2024-04-13T05:06:26.045 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:06:26.045 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","modified":"2024-04-13T05:06:16.478295Z","created":"2024-04-13T05:03:56.948897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-13T05:06:26.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71140393f0 msgr2=0x7f711403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:26.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71140393f0 0x7f711403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7120009750 tx=0x7f712001a040 comp rx=0 tx=0).stop 2024-04-13T05:06:26.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71300769f0 msgr2=0x7f71300752f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:26.047 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71300769f0 0x7f71300752f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b500 tx=0x7f711c00b8c0 comp rx=0 tx=0).stop 2024-04-13T05:06:26.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 shutdown_connections 2024-04-13T05:06:26.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7130108670 0x7f7130074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:26.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71140393f0 0x7f711403b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:26.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 --2- 172.21.15.189:0/1854502588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71300769f0 0x7f71300752f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:26.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.049+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 >> 172.21.15.189:0/1854502588 conn(0x7f71300fe0d0 msgr2=0x7f71300fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:26.048 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.050+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 shutdown_connections 2024-04-13T05:06:26.049 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:26.050+0000 7f7135e00700 1 -- 172.21.15.189:0/1854502588 wait complete. 2024-04-13T05:06:26.049 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-04-13T05:06:26.723 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:26 smithi189 ceph-mon[35531]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:26.723 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:26 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1854502588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:26.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:26 smithi083 podman[35115]: 2024-04-13 05:06:26.712463095 +0000 UTC m=+0.135177527 container create 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-13T05:06:26.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:26 smithi083 podman[35115]: 2024-04-13 05:06:26.982597795 +0000 UTC m=+0.405312237 container init 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:06:27.029 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:26 smithi057 ceph-mon[32331]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:26 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1854502588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:27.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:26 smithi083 podman[35115]: 2024-04-13 05:06:26.99322707 +0000 UTC m=+0.415941509 container start 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: set uid:gid to 167:167 (ceph:ceph) 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev), process ceph-mon, pid 2 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pidfile_write: ignore empty --pid-file 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: load: jerasure load: lrc 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: RocksDB version: 7.9.2 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Git sha 0 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Compile date 2024-04-12 04:47:35 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: DB SUMMARY 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: DB Session ID: 3XS22ZGAGVAQ7WSISQC9 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: CURRENT file: CURRENT 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: IDENTITY file: IDENTITY 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-13T05:06:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.error_if_exists: 0 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.create_if_missing: 0 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.paranoid_checks: 1 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.env: 0x55c8cc889cc0 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.fs: PosixFileSystem 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.info_log: 0x55c90cce3c40 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_file_opening_threads: 16 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.statistics: (nil) 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.use_fsync: 0 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_log_file_size: 0 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-13T05:06:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.keep_log_file_num: 1000 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.recycle_log_file_num: 0 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.allow_fallocate: 1 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.allow_mmap_reads: 0 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.allow_mmap_writes: 0 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.use_direct_reads: 0 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-13T05:06:27.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.create_missing_column_families: 0 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.db_log_dir: 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.wal_dir: 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.advise_random_on_open: 1 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.db_write_buffer_size: 0 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.write_buffer_manager: 0x55c90ccb41e0 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.rate_limiter: (nil) 2024-04-13T05:06:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.wal_recovery_mode: 2 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.enable_thread_tracking: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.enable_pipelined_write: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.unordered_write: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.row_cache: None 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.wal_filter: None 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.allow_ingest_behind: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.two_write_queues: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.manual_wal_flush: 0 2024-04-13T05:06:27.254 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.wal_compression: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.atomic_flush: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.log_readahead_size: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.best_efforts_recovery: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.allow_data_in_errors: 0 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.db_host_id: __hostname__ 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_background_jobs: 2 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_background_compactions: -1 2024-04-13T05:06:27.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_subcompactions: 1 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_total_wal_size: 0 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_open_files: -1 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bytes_per_sync: 0 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-13T05:06:27.256 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_readahead_size: 0 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_background_flushes: -1 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Compression algorithms supported: 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kZSTD supported: 0 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kXpressCompression supported: 0 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kLZ4HCCompression supported: 1 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kZlibCompression supported: 1 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kSnappyCompression supported: 1 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kLZ4Compression supported: 1 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: kBZip2Compression supported: 0 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-13T05:06:27.257 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.merge_operator: 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_filter: None 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_filter_factory: None 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.sst_partitioner_factory: None 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c90cce3e00) 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-13T05:06:27.259 INFO:journalctl@ceph.mon.b.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: index_type: 0 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: data_block_index_type: 0 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: index_shortening: 1 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: checksum: 4 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: no_block_cache: 0 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_cache: 0x55c90cce7090 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_cache_options: 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: capacity : 536870912 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: num_shard_bits : 4 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: strict_capacity_limit : 0 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_cache_compressed: (nil) 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: persistent_cache: (nil) 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_size: 4096 2024-04-13T05:06:27.260 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_size_deviation: 10 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_restart_interval: 16 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: index_block_restart_interval: 1 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: metadata_block_size: 4096 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: partition_filters: 0 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: use_delta_encoding: 1 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: filter_policy: bloomfilter 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: whole_key_filtering: 1 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: verify_compression: 0 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: format_version: 5 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: enable_index_compression: 1 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: block_align: 0 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: max_auto_readahead_size: 262144 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: prepopulate_block_cache: 0 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: initial_auto_readahead_size: 8192 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-04-13T05:06:27.261 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.write_buffer_size: 33554432 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_write_buffer_number: 2 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression: NoCompression 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression: Disabled 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.prefix_extractor: nullptr 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.num_levels: 7 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-13T05:06:27.262 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.level: 32767 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.strategy: 0 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.enabled: false 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-13T05:06:27.263 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.target_file_size_base: 67108864 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-13T05:06:27.264 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.arena_block_size: 1048576 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.disable_auto_compactions: 0 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-13T05:06:27.265 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.inplace_update_support: 0 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-13T05:06:27.266 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.bloom_locality: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.max_successive_merges: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.paranoid_file_checks: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.force_consistency_checks: 1 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.report_bg_io_stats: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.ttl: 2592000 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-13T05:06:27.267 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.enable_blob_files: false 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.min_blob_size: 0 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.blob_file_size: 268435456 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.blob_file_starting_level: 0 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-13T05:06:27.268 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7387334c-cb3f-45e1-b4c8-e9099f49d85f 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984787053820, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984787055915, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712984787, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7387334c-cb3f-45e1-b4c8-e9099f49d85f", "db_session_id": "3XS22ZGAGVAQ7WSISQC9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712984787056254, "job": 1, "event": "recovery_finished"} 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c90cdd6000 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: DB pointer 0x55c90cdc0000 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-13T05:06:27.269 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: ** DB Stats ** 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-04-13T05:06:27.270 INFO:journalctl@ceph.mon.b.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-04-13T05:06:27.271 INFO:journalctl@ceph.mon.b.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-13T05:06:27.272 INFO:journalctl@ceph.mon.b.smithi083.stdout: Interval compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout: Block cache BinnedLRUCache@0x55c90cce7090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: using public_addr v2:172.21.15.83:0/0 -> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2024-04-13T05:06:27.273 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: starting mon.b rank -1 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(???) e0 preinit fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 bash[35115]: 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 systemd[1]: Started Ceph mon.b for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).mds e1 new map 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout: e1 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout: legacy client fscid: -1 2024-04-13T05:06:27.274 INFO:journalctl@ceph.mon.b.smithi083.stdout: 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout: No filesystems configured 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-13T05:06:27.275 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Deploying cephadm binary to smithi083 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:27.276 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Added host smithi083 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.277 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Deploying cephadm binary to smithi189 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:06:27.278 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:27.279 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Added host smithi189 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:06:27.280 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.281 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.281 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2176083486' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-13T05:06:27.281 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2176083486' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.282 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c;count:3 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Deploying daemon mon.c on smithi189 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/2885725841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.283 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.a calling monitor election 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1772809328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.c calling monitor election 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-13T05:06:27.284 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: monmap epoch 2 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: last_changed 2024-04-13T05:06:16.478295+0000 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: min_mon_release 19 (squid) 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: election_strategy: 1 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: fsmap 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:27.285 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mgrmap e13: x(active, since 70s) 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: overall HEALTH_OK 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: Deploying daemon mon.b on smithi083 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1854502588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:27.286 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:27 smithi083 ceph-mon[35133]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-13T05:06:27.774 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-13T05:06:27.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-13T05:06:28.201 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: mon.a calling monitor election 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: mon.c calling monitor election 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: mon.b calling monitor election 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: monmap epoch 3 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: last_changed 2024-04-13T05:06:27.109610+0000 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:06:32.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: min_mon_release 19 (squid) 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: election_strategy: 1 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: fsmap 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: mgrmap e13: x(active, since 80s) 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: overall HEALTH_OK 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:32.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:32.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:06:32.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: mon.a calling monitor election 2024-04-13T05:06:32.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: mon.c calling monitor election 2024-04-13T05:06:32.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: mon.b calling monitor election 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: monmap epoch 3 2024-04-13T05:06:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: last_changed 2024-04-13T05:06:27.109610+0000 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: min_mon_release 19 (squid) 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: election_strategy: 1 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: fsmap 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: mgrmap e13: x(active, since 80s) 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: overall HEALTH_OK 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:32.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:32.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:06:32.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: mon.a calling monitor election 2024-04-13T05:06:32.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: mon.c calling monitor election 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: mon.b calling monitor election 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-13T05:06:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: monmap epoch 3 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: last_changed 2024-04-13T05:06:27.109610+0000 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: created 2024-04-13T05:03:56.948897+0000 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: min_mon_release 19 (squid) 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: election_strategy: 1 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: 2: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: fsmap 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: osdmap e4: 0 total, 0 up, 0 in 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: mgrmap e13: x(active, since 80s) 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: overall HEALTH_OK 2024-04-13T05:06:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:32.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 -- 172.21.15.189:0/2855176897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 msgr2=0x7fb6c4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:33.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 --2- 172.21.15.189:0/2855176897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6c4005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fa0 tx=0x7fb6d000ffc0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 -- 172.21.15.189:0/2855176897 shutdown_connections 2024-04-13T05:06:33.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 --2- 172.21.15.189:0/2855176897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6c4005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 -- 172.21.15.189:0/2855176897 >> 172.21.15.189:0/2855176897 conn(0x7fb6e00fcff0 msgr2=0x7fb6e00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:33.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 -- 172.21.15.189:0/2855176897 shutdown_connections 2024-04-13T05:06:33.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.227+0000 7fb6e5600700 1 -- 172.21.15.189:0/2855176897 wait complete. 2024-04-13T05:06:33.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.230+0000 7fb6e5600700 1 Processor -- start 2024-04-13T05:06:33.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.230+0000 7fb6e5600700 1 -- start start 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6e5600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6e019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6e5600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e019a840 0x7fb6e019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6e5600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e019f110 0x7fb6e019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6e5600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e010c6d0 con 0x7fb6e0101330 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6e5600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e010c550 con 0x7fb6e019a840 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6e5600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e010c850 con 0x7fb6e019f110 2024-04-13T05:06:33.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.231+0000 7fb6dea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6e019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:33.231 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6e019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.189:48418/0 (socket says 172.21.15.189:48418) 2024-04-13T05:06:33.231 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 -- 172.21.15.189:0/3538770167 learned_addr learned my addr 172.21.15.189:0/3538770167 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:06:33.231 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6df400700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e019f110 0x7fb6e019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:33.231 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 -- 172.21.15.189:0/3538770167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e019f110 msgr2=0x7fb6e019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:33.232 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e019f110 0x7fb6e019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.232 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 -- 172.21.15.189:0/3538770167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e019a840 msgr2=0x7fb6e019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:33.232 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e019a840 0x7fb6e019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.232 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.232+0000 7fb6dea00700 1 -- 172.21.15.189:0/3538770167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e01a6e80 con 0x7fb6e0101330 2024-04-13T05:06:33.232 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.233+0000 7fb6dea00700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6e019a300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00137f0 tx=0x7fb6d0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:33.232 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.233+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0029020 con 0x7fb6e0101330 2024-04-13T05:06:33.233 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.233+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e01a7050 con 0x7fb6e0101330 2024-04-13T05:06:33.233 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.233+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d00056c0 con 0x7fb6e0101330 2024-04-13T05:06:33.233 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.233+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00116d0 con 0x7fb6e0101330 2024-04-13T05:06:33.233 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.234+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e01a7590 con 0x7fb6e0101330 2024-04-13T05:06:33.233 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.235+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e0064110 con 0x7fb6e0101330 2024-04-13T05:06:33.233 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.235+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb6d001b630 con 0x7fb6e0101330 2024-04-13T05:06:33.234 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.236+0000 7fb6d7400700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6bc039ae0 0x7fb6bc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:33.234 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.236+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb6d004f040 con 0x7fb6e0101330 2024-04-13T05:06:33.235 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.236+0000 7fb6d7e00700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6bc039ae0 0x7fb6bc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:33.235 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.237+0000 7fb6d7e00700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6bc039ae0 0x7fb6bc03bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80098e0 tx=0x7fb6c8006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:33.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.253+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6d0033060 con 0x7fb6e0101330 2024-04-13T05:06:33.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:33 smithi083 ceph-mon[35133]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:33.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:33 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:33 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:33 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:33.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:33 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:33.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:33 smithi057 ceph-mon[32331]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:33.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:33 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:33 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:33 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:33.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:33 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:33.579 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:33 smithi189 ceph-mon[35531]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:33.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:33 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:33 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:33.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:33 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:33.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:33 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:06:33.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.624+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb6e002dfa0 con 0x7fb6e0101330 2024-04-13T05:06:33.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.625+0000 7fb6d7400700 1 -- 172.21.15.189:0/3538770167 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fb6d0022070 con 0x7fb6e0101330 2024-04-13T05:06:33.624 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-13T05:06:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","modified":"2024-04-13T05:06:27.109610Z","created":"2024-04-13T05:03:56.948897Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6bc039ae0 msgr2=0x7fb6bc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6bc039ae0 0x7fb6bc03bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80098e0 tx=0x7fb6c8006dd0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 msgr2=0x7fb6e019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6e019a300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00137f0 tx=0x7fb6d0005250 comp rx=0 tx=0).stop 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 shutdown_connections 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e019f110 0x7fb6e019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6bc039ae0 0x7fb6bc03bfa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb6e0101330 0x7fb6e019a300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 --2- 172.21.15.189:0/3538770167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e019a840 0x7fb6e019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 >> 172.21.15.189:0/3538770167 conn(0x7fb6e00fcff0 msgr2=0x7fb6e00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 shutdown_connections 2024-04-13T05:06:33.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:33.628+0000 7fb6e5600700 1 -- 172.21.15.189:0/3538770167 wait complete. 2024-04-13T05:06:33.628 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-04-13T05:06:34.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:34 smithi189 ceph-mon[35531]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:34.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:34 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/3538770167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:34.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:34 smithi083 ceph-mon[35133]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:34.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:34 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/3538770167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:34.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:34 smithi057 ceph-mon[32331]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:34.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:34 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/3538770167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-13T05:06:34.530 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:06:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:06:34.111+0000 7fb1c3200700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-13T05:06:34.735 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-13T05:06:34.735 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph config generate-minimal-conf 2024-04-13T05:06:35.190 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: Updating smithi057:/etc/ceph/ceph.conf 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-13T05:06:35.664 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:35 smithi057 ceph-mon[32331]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-13T05:06:35.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:35.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:35.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: Updating smithi057:/etc/ceph/ceph.conf 2024-04-13T05:06:35.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-13T05:06:35.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:35 smithi083 ceph-mon[35133]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-13T05:06:35.829 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: Updating smithi057:/etc/ceph/ceph.conf 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-13T05:06:35.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:35 smithi189 ceph-mon[35531]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-13T05:06:36.667 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:36 smithi057 ceph-mon[32331]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:36.667 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:36 smithi057 ceph-mon[32331]: Updating smithi057:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.667 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:36 smithi057 ceph-mon[32331]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.667 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:36 smithi057 ceph-mon[32331]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:36 smithi083 ceph-mon[35133]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:36.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:36 smithi083 ceph-mon[35133]: Updating smithi057:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:36 smithi083 ceph-mon[35133]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:36 smithi083 ceph-mon[35133]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:36 smithi189 ceph-mon[35531]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:36 smithi189 ceph-mon[35531]: Updating smithi057:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:36 smithi189 ceph-mon[35531]: Updating smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:36 smithi189 ceph-mon[35531]: Updating smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-13T05:06:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 -- 172.21.15.57:0/3658753581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd74074ee0 msgr2=0x7fdd740752c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 --2- 172.21.15.57:0/3658753581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd74074ee0 0x7fdd740752c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009910 tx=0x7fdd68009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 -- 172.21.15.57:0/3658753581 shutdown_connections 2024-04-13T05:06:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 --2- 172.21.15.57:0/3658753581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd74074ee0 0x7fdd740752c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 -- 172.21.15.57:0/3658753581 >> 172.21.15.57:0/3658753581 conn(0x7fdd740fdff0 msgr2=0x7fdd74100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:37.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 -- 172.21.15.57:0/3658753581 shutdown_connections 2024-04-13T05:06:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.476+0000 7fdd7c400700 1 -- 172.21.15.57:0/3658753581 wait complete. 2024-04-13T05:06:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.478+0000 7fdd7c400700 1 Processor -- start 2024-04-13T05:06:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.478+0000 7fdd7c400700 1 -- start start 2024-04-13T05:06:37.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.478+0000 7fdd7c400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd74074ee0 0x7fdd7419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:37.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.478+0000 7fdd7c400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd7419a930 0x7fdd7419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:37.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7c400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 0x7fdd7419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:37.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7c400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7410ec20 con 0x7fdd7419f200 2024-04-13T05:06:37.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7c400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7410eaa0 con 0x7fdd74074ee0 2024-04-13T05:06:37.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7c400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7410eda0 con 0x7fdd7419a930 2024-04-13T05:06:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 0x7fdd7419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7a400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 0x7fdd7419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48594/0 (socket says 172.21.15.57:48594) 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd7a400700 1 -- 172.21.15.57:0/2918202695 learned_addr learned my addr 172.21.15.57:0/2918202695 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd79a00700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd74074ee0 0x7fdd7419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.479+0000 7fdd79000700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd7419a930 0x7fdd7419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.480+0000 7fdd7a400700 1 -- 172.21.15.57:0/2918202695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd7419a930 msgr2=0x7fdd7419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.480+0000 7fdd7a400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd7419a930 0x7fdd7419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.480+0000 7fdd7a400700 1 -- 172.21.15.57:0/2918202695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd74074ee0 msgr2=0x7fdd7419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.480+0000 7fdd7a400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd74074ee0 0x7fdd7419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.480+0000 7fdd7a400700 1 -- 172.21.15.57:0/2918202695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd741a6f70 con 0x7fdd7419f200 2024-04-13T05:06:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.480+0000 7fdd7a400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 0x7fdd7419f6b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00ba50 tx=0x7fdd6c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.481+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c019070 con 0x7fdd7419f200 2024-04-13T05:06:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.481+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd6c00cde0 con 0x7fdd7419f200 2024-04-13T05:06:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.481+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd741a71a0 con 0x7fdd7419f200 2024-04-13T05:06:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.481+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c007350 con 0x7fdd7419f200 2024-04-13T05:06:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.481+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd741a76e0 con 0x7fdd7419f200 2024-04-13T05:06:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.482+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd74064110 con 0x7fdd7419f200 2024-04-13T05:06:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.483+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdd6c01c070 con 0x7fdd7419f200 2024-04-13T05:06:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.483+0000 7fdd72000700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd5c039ae0 0x7fdd5c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.484+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdd6c04bf60 con 0x7fdd7419f200 2024-04-13T05:06:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.484+0000 7fdd79a00700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd5c039ae0 0x7fdd5c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:37.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.485+0000 7fdd79a00700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd5c039ae0 0x7fdd5c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006010 tx=0x7fdd6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:37.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.493+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd6c014440 con 0x7fdd7419f200 2024-04-13T05:06:37.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.583 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:37.584 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:37.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.727+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdd741a7a60 con 0x7fdd7419f200 2024-04-13T05:06:37.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.727+0000 7fdd72000700 1 -- 172.21.15.57:0/2918202695 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fdd6c04d070 con 0x7fdd7419f200 2024-04-13T05:06:37.727 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:37.727 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2024-04-13T05:06:37.727 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:06:37.727 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-04-13T05:06:37.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.730+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd5c039ae0 msgr2=0x7fdd5c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:37.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.730+0000 7fdd7c400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd5c039ae0 0x7fdd5c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006010 tx=0x7fdd6801a040 comp rx=0 tx=0).stop 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 msgr2=0x7fdd7419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 0x7fdd7419f6b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00ba50 tx=0x7fdd6c00bd60 comp rx=0 tx=0).stop 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 shutdown_connections 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd5c039ae0 0x7fdd5c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd7419f200 0x7fdd7419f6b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd74074ee0 0x7fdd7419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 --2- 172.21.15.57:0/2918202695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd7419a930 0x7fdd7419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 >> 172.21.15.57:0/2918202695 conn(0x7fdd740fdff0 msgr2=0x7fdd74108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 shutdown_connections 2024-04-13T05:06:37.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:37.731+0000 7fdd7c400700 1 -- 172.21.15.57:0/2918202695 wait complete. 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:37.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:37.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:38.569 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-13T05:06:38.570 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:06:38.570 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2024-04-13T05:06:38.673 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:06:38.673 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:38.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:38 smithi057 ceph-mon[32331]: Reconfiguring mon.a (unknown last config time)... 2024-04-13T05:06:38.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:38 smithi057 ceph-mon[32331]: Reconfiguring daemon mon.a on smithi057 2024-04-13T05:06:38.779 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:38 smithi057 ceph-mon[32331]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:38.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:38 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2918202695' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:38.819 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:06:38.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-04-13T05:06:38.829 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:38 smithi189 ceph-mon[35531]: Reconfiguring mon.a (unknown last config time)... 2024-04-13T05:06:38.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:38 smithi189 ceph-mon[35531]: Reconfiguring daemon mon.a on smithi057 2024-04-13T05:06:38.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:38 smithi189 ceph-mon[35531]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:38.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:38 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2918202695' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:38.905 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:38 smithi083 ceph-mon[35133]: Reconfiguring mon.a (unknown last config time)... 2024-04-13T05:06:38.905 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:38 smithi083 ceph-mon[35133]: Reconfiguring daemon mon.a on smithi057 2024-04-13T05:06:38.905 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:38 smithi083 ceph-mon[35133]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:38.905 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:38 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2918202695' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:38.921 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:06:38.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:39.062 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:06:39.062 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-04-13T05:06:39.164 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:06:39.164 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-13T05:06:39.301 INFO:tasks.cephadm:Adding mgr.x on smithi057 2024-04-13T05:06:39.301 INFO:tasks.cephadm:Adding mgr.y on smithi083 2024-04-13T05:06:39.302 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-04-13T05:06:39.302 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch apply mgr '3;smithi057=x;smithi083=y;smithi189=z' 2024-04-13T05:06:39.769 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:06:41.216 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:41.216 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:41.216 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:41.216 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: Reconfiguring mon.b (monmap changed)... 2024-04-13T05:06:41.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:41.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:41.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:41.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:41 smithi189 ceph-mon[35531]: Reconfiguring daemon mon.b on smithi083 2024-04-13T05:06:41.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:41.229 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:41.230 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:41.230 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: Reconfiguring mon.b (monmap changed)... 2024-04-13T05:06:41.230 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:41.230 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:41.230 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:41.230 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:41 smithi083 ceph-mon[35133]: Reconfiguring daemon mon.b on smithi083 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: Reconfiguring mon.b (monmap changed)... 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:41.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:41 smithi057 ceph-mon[32331]: Reconfiguring daemon mon.b on smithi083 2024-04-13T05:06:41.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 -- 172.21.15.189:0/1080175083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c076ae0 msgr2=0x7f801c074f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:41.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 --2- 172.21.15.189:0/1080175083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c076ae0 0x7f801c074f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8010009910 tx=0x7f8010009c20 comp rx=0 tx=0).stop 2024-04-13T05:06:41.703 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 -- 172.21.15.189:0/1080175083 shutdown_connections 2024-04-13T05:06:41.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 --2- 172.21.15.189:0/1080175083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c076ae0 0x7f801c074f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 -- 172.21.15.189:0/1080175083 >> 172.21.15.189:0/1080175083 conn(0x7f801c0fdbb0 msgr2=0x7f801c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:41.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 -- 172.21.15.189:0/1080175083 shutdown_connections 2024-04-13T05:06:41.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.703+0000 7f8024600700 1 -- 172.21.15.189:0/1080175083 wait complete. 2024-04-13T05:06:41.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.705+0000 7f8024600700 1 Processor -- start 2024-04-13T05:06:41.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.705+0000 7f8024600700 1 -- start start 2024-04-13T05:06:41.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.705+0000 7f8024600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 0x7f801c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:41.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8024600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f801c19e8b0 0x7f801c1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:41.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8024600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c19ed30 0x7f801c1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:41.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8024600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f801c10ebf0 con 0x7f801c19ed30 2024-04-13T05:06:41.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8024600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1061c0 con 0x7f801c19e8b0 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8024600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f801c10ed70 con 0x7f801c076ae0 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 0x7f801c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 0x7f801c19e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:59928/0 (socket says 172.21.15.189:59928) 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 -- 172.21.15.189:0/2278927985 learned_addr learned my addr 172.21.15.189:0/2278927985 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 -- 172.21.15.189:0/2278927985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f801c19e8b0 msgr2=0x7f801c1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021200700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f801c19e8b0 0x7f801c1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8022600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c19ed30 0x7f801c1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f801c19e8b0 0x7f801c1a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 -- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c19ed30 msgr2=0x7f801c1a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c19ed30 0x7f801c1a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.706+0000 7f8021c00700 1 -- 172.21.15.189:0/2278927985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c1a6780 con 0x7f801c076ae0 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f8021200700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f801c19e8b0 0x7f801c1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f8021c00700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 0x7f801c19e370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8010005e00 tx=0x7f8010009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010027070 con 0x7f801c076ae0 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801000be10 con 0x7f801c076ae0 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801000f460 con 0x7f801c076ae0 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1a6a10 con 0x7f801c076ae0 2024-04-13T05:06:41.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.707+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1a6f20 con 0x7f801c076ae0 2024-04-13T05:06:41.709 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.709+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f801002a070 con 0x7f801c076ae0 2024-04-13T05:06:41.709 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.709+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c064110 con 0x7f801c076ae0 2024-04-13T05:06:41.709 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.709+0000 7f801a000700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8004039ae0 0x7f800403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:41.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.709+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f801004dad0 con 0x7f801c076ae0 2024-04-13T05:06:41.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.710+0000 7f8021200700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8004039ae0 0x7f800403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:41.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.711+0000 7f8021200700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8004039ae0 0x7f800403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f80080079b0 tx=0x7f8008008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:41.714 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.716+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8010013070 con 0x7f801c076ae0 2024-04-13T05:06:41.957 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.958+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi083=y;smithi189=z", "target": ["mon-mgr", ""]}) -- 0x7f801c062f80 con 0x7f8004039ae0 2024-04-13T05:06:41.971 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.973+0000 7f801a000700 1 -- 172.21.15.189:0/2278927985 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f801c062f80 con 0x7f8004039ae0 2024-04-13T05:06:41.971 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8004039ae0 msgr2=0x7f800403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8004039ae0 0x7f800403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f80080079b0 tx=0x7f8008008040 comp rx=0 tx=0).stop 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 msgr2=0x7f801c19e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 0x7f801c19e370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8010005e00 tx=0x7f8010009e60 comp rx=0 tx=0).stop 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 shutdown_connections 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c076ae0 0x7f801c19e370 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8004039ae0 0x7f800403bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f801c19ed30 0x7f801c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 --2- 172.21.15.189:0/2278927985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f801c19e8b0 0x7f801c1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:41.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 >> 172.21.15.189:0/2278927985 conn(0x7f801c0fdbb0 msgr2=0x7f801c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:41.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 shutdown_connections 2024-04-13T05:06:41.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:06:41.976+0000 7f8024600700 1 -- 172.21.15.189:0/2278927985 wait complete. 2024-04-13T05:06:42.686 DEBUG:teuthology.orchestra.run.smithi083:mgr.y> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.y.service 2024-04-13T05:06:42.691 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.z.service 2024-04-13T05:06:42.698 INFO:tasks.cephadm:Deploying OSDs... 2024-04-13T05:06:42.698 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:06:42.698 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2024-04-13T05:06:42.758 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-13T05:06:42.758 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-13 05:05:28.805004354 +0000 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-13 04:59:04.393490819 +0000 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-13 04:59:04.393490819 +0000 2024-04-13T05:06:42.813 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-04-13 04:59:04.393490819 +0000 2024-04-13T05:06:42.813 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-13T05:06:42.912 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-13T05:06:42.913 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-13T05:06:42.913 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000483333 s, 1.1 MB/s 2024-04-13T05:06:42.917 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-13T05:06:42.979 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-13 05:05:28.869003393 +0000 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-13 04:59:05.054480597 +0000 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-13 04:59:05.054480597 +0000 2024-04-13T05:06:43.030 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-04-13 04:59:05.054480597 +0000 2024-04-13T05:06:43.031 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-13T05:06:43.125 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-13T05:06:43.125 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-13T05:06:43.125 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.0004074 s, 1.3 MB/s 2024-04-13T05:06:43.129 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-13T05:06:43.193 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-13 05:05:28.929002493 +0000 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-13 04:59:05.688470792 +0000 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-13 04:59:05.688470792 +0000 2024-04-13T05:06:43.247 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-04-13 04:59:05.688470792 +0000 2024-04-13T05:06:43.247 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi083=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: Saving service mgr spec with placement smithi057=x;smithi083=y;smithi189=z;count:3 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: Reconfiguring mon.c (monmap changed)... 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:42 smithi083 ceph-mon[35133]: Reconfiguring daemon mon.c on smithi189 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi083=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: Saving service mgr spec with placement smithi057=x;smithi083=y;smithi189=z;count:3 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: Reconfiguring mon.c (monmap changed)... 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:43.265 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:43.266 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:43.266 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:42 smithi189 ceph-mon[35531]: Reconfiguring daemon mon.c on smithi189 2024-04-13T05:06:43.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:43.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi083=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:43.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: Saving service mgr spec with placement smithi057=x;smithi083=y;smithi189=z;count:3 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: Reconfiguring mon.c (monmap changed)... 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:42 smithi057 ceph-mon[32331]: Reconfiguring daemon mon.c on smithi189 2024-04-13T05:06:43.344 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-13T05:06:43.344 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-13T05:06:43.344 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000450828 s, 1.1 MB/s 2024-04-13T05:06:43.349 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-13T05:06:43.453 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2024-04-13T05:06:43.508 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-13 05:05:28.992001547 +0000 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-13 04:59:06.327460910 +0000 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-13 04:59:06.327460910 +0000 2024-04-13T05:06:43.509 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-04-13 04:59:06.327460910 +0000 2024-04-13T05:06:43.509 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-13T05:06:43.610 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-13T05:06:43.610 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-13T05:06:43.610 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000437131 s, 1.2 MB/s 2024-04-13T05:06:43.615 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-13T05:06:43.677 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-13T05:06:43.677 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-13T05:06:43.732 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-13T05:06:43.733 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-13T05:06:43.778 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-13T05:06:43.778 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-13 05:06:26.165562107 +0000 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-13 04:58:43.915393195 +0000 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-13 04:58:43.915393195 +0000 2024-04-13T05:06:43.779 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-13 04:58:43.915393195 +0000 2024-04-13T05:06:43.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-13T05:06:43.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-13T05:06:43.912 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-13T05:06:43.912 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000388754 s, 1.3 MB/s 2024-04-13T05:06:43.917 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-13T05:06:43.974 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-13T05:06:44.025 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-13T05:06:44.026 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:44.026 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-13T05:06:44.026 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:44.026 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:44.026 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-13 05:06:26.166562084 +0000 2024-04-13T05:06:44.026 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-13 04:58:44.536377888 +0000 2024-04-13T05:06:44.027 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-13 04:58:44.536377888 +0000 2024-04-13T05:06:44.027 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-13 04:58:44.536377888 +0000 2024-04-13T05:06:44.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-13T05:06:44.110 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-13T05:06:44.111 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-13T05:06:44.111 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000302586 s, 1.7 MB/s 2024-04-13T05:06:44.115 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-13T05:06:44.171 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-13 05:06:26.167562060 +0000 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-13 04:58:45.167362335 +0000 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-13 04:58:45.167362335 +0000 2024-04-13T05:06:44.223 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-13 04:58:45.167362335 +0000 2024-04-13T05:06:44.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-13T05:06:44.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-13T05:06:44.309 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-13T05:06:44.309 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000370059 s, 1.4 MB/s 2024-04-13T05:06:44.312 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-13T05:06:44.368 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-13 05:06:26.167562060 +0000 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-13 04:58:45.786347077 +0000 2024-04-13T05:06:44.419 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-13 04:58:45.786347077 +0000 2024-04-13T05:06:44.420 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-13 04:58:45.786347077 +0000 2024-04-13T05:06:44.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-13T05:06:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-13T05:06:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-13T05:06:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000344095 s, 1.5 MB/s 2024-04-13T05:06:44.503 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-13T05:06:44.557 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-13T05:06:44.557 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-04-13T05:06:44.611 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-13T05:06:44.611 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-04-13T05:06:44.658 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-13T05:06:44.658 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:44.658 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-13T05:06:44.659 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:44.659 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:44.659 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-13 05:06:15.259568715 +0000 2024-04-13T05:06:44.659 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-13 04:58:45.245100616 +0000 2024-04-13T05:06:44.659 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-13 04:58:45.245100616 +0000 2024-04-13T05:06:44.659 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-13 04:58:45.245100616 +0000 2024-04-13T05:06:44.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-13T05:06:44.784 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-13T05:06:44.784 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-13T05:06:44.784 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000358796 s, 1.4 MB/s 2024-04-13T05:06:44.788 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-13T05:06:44.835 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-13 05:06:15.260568695 +0000 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-13 04:58:45.838088643 +0000 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-13 04:58:45.838088643 +0000 2024-04-13T05:06:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-13 04:58:45.838088643 +0000 2024-04-13T05:06:44.877 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-13T05:06:44.987 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-13T05:06:44.987 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-13T05:06:44.987 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000300376 s, 1.7 MB/s 2024-04-13T05:06:44.990 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-13T05:06:45.030 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-13 05:06:15.260568695 +0000 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-13 04:58:46.434076610 +0000 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-13 04:58:46.434076610 +0000 2024-04-13T05:06:45.110 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-13 04:58:46.434076610 +0000 2024-04-13T05:06:45.111 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-13T05:06:45.230 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-13T05:06:45.230 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-13T05:06:45.230 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000341714 s, 1.5 MB/s 2024-04-13T05:06:45.234 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-13T05:06:45.280 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-13 05:06:15.261568676 +0000 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-13 04:58:47.045064275 +0000 2024-04-13T05:06:45.370 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-13 04:58:47.045064275 +0000 2024-04-13T05:06:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-13 04:58:47.045064275 +0000 2024-04-13T05:06:45.371 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:45.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:45.450 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-13T05:06:45.450 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-13T05:06:45.450 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000312101 s, 1.6 MB/s 2024-04-13T05:06:45.455 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:45.500 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2024-04-13T05:06:45.500 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-13T05:06:45.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:45.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:45.967 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:06:46.399 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:46 smithi189 ceph-mon[35531]: Deploying daemon mgr.z on smithi189 2024-04-13T05:06:46.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:46 smithi057 ceph-mon[32331]: Deploying daemon mgr.z on smithi189 2024-04-13T05:06:46.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:46 smithi083 ceph-mon[35133]: Deploying daemon mgr.z on smithi189 2024-04-13T05:06:47.185 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:47 smithi057 ceph-mon[32331]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:47.216 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:47 smithi189 ceph-mon[35531]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:47.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:47 smithi083 ceph-mon[35133]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:47.566 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:47 smithi189 systemd[1]: Starting Ceph mgr.z for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:06:47.830 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:47 smithi189 podman[37377]: 2024-04-13 05:06:47.472143638 +0000 UTC m=+0.034141399 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:06:47.830 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:47 smithi189 podman[37377]: 2024-04-13 05:06:47.593916081 +0000 UTC m=+0.155913894 container create 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-13T05:06:48.330 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:47 smithi189 podman[37377]: 2024-04-13 05:06:47.914045939 +0000 UTC m=+0.476043725 container init 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z, org.label-schema.build-date=20240409, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-13T05:06:48.331 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:47 smithi189 podman[37377]: 2024-04-13 05:06:47.923634562 +0000 UTC m=+0.485632382 container start 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-13T05:06:48.331 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:47 smithi189 ceph-mgr[37397]: -- 172.21.15.189:0/3858476002 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620645025a0 con 0x5620644d6800 2024-04-13T05:06:48.331 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:48 smithi189 bash[37377]: 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 2024-04-13T05:06:48.331 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:48 smithi189 systemd[1]: Started Ceph mgr.z for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:06:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-13T05:06:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-13T05:06:48.757 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-04-13T05:06:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-04-13T05:06:48.757 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0367551 s, 285 MB/s 2024-04-13T05:06:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-04-13T05:06:49.080 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:48.705+0000 7fe979781200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T05:06:49.080 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:48.827+0000 7fe979781200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-13T05:06:49.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:49.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:49.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:49.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-13T05:06:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:49.621 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2024-04-13T05:06:49.936 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:49.631+0000 7fe979781200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T05:06:49.936 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:49.754+0000 7fe979781200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T05:06:49.936 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:49.935+0000 7fe979781200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T05:06:50.126 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:06:50.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-mon[35531]: Deploying daemon mgr.y on smithi083 2024-04-13T05:06:50.330 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:50.060+0000 7fe979781200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T05:06:50.330 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:50.171+0000 7fe979781200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T05:06:50.388 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:50 smithi083 ceph-mon[35133]: Deploying daemon mgr.y on smithi083 2024-04-13T05:06:50.456 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:50 smithi057 ceph-mon[32331]: Deploying daemon mgr.y on smithi083 2024-04-13T05:06:50.766 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:50.511+0000 7fe979781200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T05:06:50.766 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:50.653+0000 7fe979781200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T05:06:51.080 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:50.765+0000 7fe979781200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T05:06:51.080 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:50.890+0000 7fe979781200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T05:06:51.456 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:51 smithi057 ceph-mon[32331]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:51.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:51 smithi083 ceph-mon[35133]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:51.580 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:51.110+0000 7fe979781200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T05:06:51.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:51 smithi189 ceph-mon[35531]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:51.801 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:51 smithi083 podman[36486]: 2024-04-13 05:06:51.753385444 +0000 UTC m=+0.359238732 container start 7e61f2ee308f84c8be8477f6348ef4128211db6f4e1e191965b21436c8a7d25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-13T05:06:52.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:51 smithi083 ceph-mgr[36505]: -- 172.21.15.83:0/2148763615 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55862c93c5a0 con 0x55862c911000 2024-04-13T05:06:52.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:51 smithi083 bash[36486]: 7e61f2ee308f84c8be8477f6348ef4128211db6f4e1e191965b21436c8a7d25a 2024-04-13T05:06:52.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:51 smithi083 systemd[1]: Started Ceph mgr.y for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 -- 172.21.15.57:0/2836577515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 msgr2=0x7fad8c0798f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 --2- 172.21.15.57:0/2836577515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c0798f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad8400b0a0 tx=0x7fad8400b3b0 comp rx=0 tx=0).stop 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 -- 172.21.15.57:0/2836577515 shutdown_connections 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 --2- 172.21.15.57:0/2836577515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c077460 0x7fad8c077880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 --2- 172.21.15.57:0/2836577515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c0798f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 --2- 172.21.15.57:0/2836577515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c0786b0 0x7fad8c078b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 -- 172.21.15.57:0/2836577515 >> 172.21.15.57:0/2836577515 conn(0x7fad8c1001f0 msgr2=0x7fad8c102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 -- 172.21.15.57:0/2836577515 shutdown_connections 2024-04-13T05:06:52.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.293+0000 7fad91a00700 1 -- 172.21.15.57:0/2836577515 wait complete. 2024-04-13T05:06:52.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.294+0000 7fad91a00700 1 Processor -- start 2024-04-13T05:06:52.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.294+0000 7fad91a00700 1 -- start start 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c077460 0x7fad8c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c0786b0 0x7fad8c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c1a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c114b00 con 0x7fad8c079070 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c114980 con 0x7fad8c077460 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c114c80 con 0x7fad8c0786b0 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c1a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad83e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c0786b0 0x7fad8c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad83e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c0786b0 0x7fad8c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:36174/0 (socket says 172.21.15.57:36174) 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad83e00700 1 -- 172.21.15.57:0/3953738917 learned_addr learned my addr 172.21.15.57:0/3953738917 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:06:52.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad8be00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c077460 0x7fad8c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:52.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91000700 1 -- 172.21.15.57:0/3953738917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c0786b0 msgr2=0x7fad8c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.295+0000 7fad91000700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c0786b0 0x7fad8c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad91000700 1 -- 172.21.15.57:0/3953738917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c077460 msgr2=0x7fad8c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad91000700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c077460 0x7fad8c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad91000700 1 -- 172.21.15.57:0/3953738917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c1a6d30 con 0x7fad8c079070 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad8be00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c077460 0x7fad8c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad91000700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c1a6690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad84015000 tx=0x7fad84004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84021070 con 0x7fad8c079070 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad84007b80 con 0x7fad8c079070 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840052a0 con 0x7fad8c079070 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c1a6fc0 con 0x7fad8c079070 2024-04-13T05:06:52.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.296+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c1a7420 con 0x7fad8c079070 2024-04-13T05:06:52.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.297+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c0697a0 con 0x7fad8c079070 2024-04-13T05:06:52.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.298+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fad84024070 con 0x7fad8c079070 2024-04-13T05:06:52.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.298+0000 7fad89600700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fad70039890 0x7fad7003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:06:52.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.298+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad8400f020 con 0x7fad8c079070 2024-04-13T05:06:52.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.299+0000 7fad8be00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fad70039890 0x7fad7003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:06:52.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.299+0000 7fad8be00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fad70039890 0x7fad7003bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009cb0 tx=0x7fad7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:06:52.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.304+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad8402b020 con 0x7fad8c079070 2024-04-13T05:06:52.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:52.572+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fad8c062ef0 con 0x7fad70039890 2024-04-13T05:06:52.830 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:52.441+0000 7fe979781200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T05:06:52.830 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:52.620+0000 7fe979781200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T05:06:52.955 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:52.547+0000 7f1d78bf1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-13T05:06:52.955 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:52.679+0000 7f1d78bf1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-13T05:06:53.248 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:53.248 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:06:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:53.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:53.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:53.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:06:53.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:06:53.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:53.060+0000 7fe979781200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:53.187+0000 7fe979781200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:06:53.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:06:53.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:06:53.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:53.713 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:53.456+0000 7fe979781200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:06:53.713 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:53.712+0000 7fe979781200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T05:06:53.846 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:53.535+0000 7f1d78bf1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-13T05:06:53.847 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:53.664+0000 7f1d78bf1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-13T05:06:54.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:53 smithi189 ceph-mon[35531]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:54.248 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:53 smithi083 ceph-mon[35133]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:54.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:53.846+0000 7f1d78bf1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-13T05:06:54.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:53.972+0000 7f1d78bf1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-13T05:06:54.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:54.093+0000 7f1d78bf1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-13T05:06:54.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:53 smithi057 ceph-mon[32331]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:06:54.433 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:54.110+0000 7fe979781200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:06:54.433 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:54.433+0000 7fe979781200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T05:06:54.708 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:54.441+0000 7f1d78bf1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-13T05:06:54.708 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:54.589+0000 7f1d78bf1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-13T05:06:54.998 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:54.708+0000 7f1d78bf1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-13T05:06:54.998 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:54.838+0000 7f1d78bf1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-13T05:06:55.080 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:54.746+0000 7fe979781200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T05:06:55.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:55 smithi057 ceph-mon[32331]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:55.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.498 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:55.064+0000 7f1d78bf1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-13T05:06:55.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:55 smithi083 ceph-mon[35133]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:55.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:55 smithi189 ceph-mon[35531]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:55.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:55.994 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:55.800+0000 7fe979781200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T05:06:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:56.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:56.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:56.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:56.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:56.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:56.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:56.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:56.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:06:56.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:56.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-13T05:06:56.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-13T05:06:56.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:06:56.580 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:56.528+0000 7fe979781200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T05:06:56.748 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:56.404+0000 7f1d78bf1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-13T05:06:56.748 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:56.579+0000 7f1d78bf1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-13T05:06:56.954 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:56.661+0000 7fe979781200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T05:06:57.206 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:06:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z[37393]: 2024-04-13T05:06:56.954+0000 7fe979781200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T05:06:57.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:57.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: Reconfiguring mgr.x (unknown last config time)... 2024-04-13T05:06:57.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: Reconfiguring daemon mgr.x on smithi057 2024-04-13T05:06:57.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3319447518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]: dispatch 2024-04-13T05:06:57.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:57.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]: dispatch 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]': finished 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: osdmap e5: 1 total, 0 up, 1 in 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: Standby manager daemon z started 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-13T05:06:57.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:57 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-13T05:06:57.248 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:56.995+0000 7f1d78bf1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:57.115+0000 7f1d78bf1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: Reconfiguring mgr.x (unknown last config time)... 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: Reconfiguring daemon mgr.x on smithi057 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3319447518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]: dispatch 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]: dispatch 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]': finished 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: osdmap e5: 1 total, 0 up, 1 in 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: Standby manager daemon z started 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-13T05:06:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-13T05:06:57.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:06:57.523+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fad8401ac70 con 0x7fad8c079070 2024-04-13T05:06:57.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:57.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: Reconfiguring mgr.x (unknown last config time)... 2024-04-13T05:06:57.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: Reconfiguring daemon mgr.x on smithi057 2024-04-13T05:06:57.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3319447518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]: dispatch 2024-04-13T05:06:57.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:06:57.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]: dispatch 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f852b23-d1b5-4c86-8946-0a9213323ecd"}]': finished 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: osdmap e5: 1 total, 0 up, 1 in 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: Standby manager daemon z started 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-13T05:06:57.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:57 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.189:0/299320011' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-13T05:06:57.748 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:57.375+0000 7f1d78bf1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:06:57.748 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:57.624+0000 7f1d78bf1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-13T05:06:58.334 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:58.003+0000 7f1d78bf1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-13T05:06:58.334 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/840611321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:06:58.335 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:58 smithi083 ceph-mon[35133]: mgrmap e14: x(active, since 106s), standbys: z 2024-04-13T05:06:58.335 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-13T05:06:58.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/840611321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:06:58.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:58 smithi057 ceph-mon[32331]: mgrmap e14: x(active, since 106s), standbys: z 2024-04-13T05:06:58.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-13T05:06:58.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/840611321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:06:58.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:58 smithi189 ceph-mon[35531]: mgrmap e14: x(active, since 106s), standbys: z 2024-04-13T05:06:58.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-13T05:06:58.644 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:58.334+0000 7f1d78bf1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-13T05:06:58.998 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:58.644+0000 7f1d78bf1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-13T05:06:59.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:06:59 smithi083 ceph-mon[35133]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:59.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:06:59 smithi057 ceph-mon[32331]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:06:59 smithi189 ceph-mon[35531]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:06:59.998 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:06:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:06:59.683+0000 7f1d78bf1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-13T05:07:00.748 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:07:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:07:00.379+0000 7f1d78bf1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-13T05:07:00.748 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:07:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:07:00.507+0000 7f1d78bf1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-13T05:07:01.140 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:07:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y[36501]: 2024-04-13T05:07:00.787+0000 7f1d78bf1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-13T05:07:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:01.167+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fad840102f0 con 0x7fad8c079070 2024-04-13T05:07:01.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:01 smithi057 ceph-mon[32331]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:01.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:01 smithi057 ceph-mon[32331]: Standby manager daemon y started 2024-04-13T05:07:01.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:01 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-13T05:07:01.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:01 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-13T05:07:01.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:01 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-13T05:07:01.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:01 smithi057 ceph-mon[32331]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-13T05:07:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:01 smithi083 ceph-mon[35133]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:01 smithi083 ceph-mon[35133]: Standby manager daemon y started 2024-04-13T05:07:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:01 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-13T05:07:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:01 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-13T05:07:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:01 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-13T05:07:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:01 smithi083 ceph-mon[35133]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-13T05:07:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:01 smithi189 ceph-mon[35531]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:01 smithi189 ceph-mon[35531]: Standby manager daemon y started 2024-04-13T05:07:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:01 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-13T05:07:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:01 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-13T05:07:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:01 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-13T05:07:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:01 smithi189 ceph-mon[35531]: from='mgr.? 172.21.15.83:0/117177071' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-13T05:07:02.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:02 smithi057 ceph-mon[32331]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-04-13T05:07:02.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-13T05:07:02.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:02 smithi083 ceph-mon[35133]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-04-13T05:07:02.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-13T05:07:02.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:02 smithi189 ceph-mon[35531]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-04-13T05:07:02.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-13T05:07:03.274 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:03 smithi057 ceph-mon[32331]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:03.274 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:03.274 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:03.274 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:03.436 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:03 smithi083 ceph-mon[35133]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:03.437 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:03.437 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:03.437 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:03.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:03 smithi189 ceph-mon[35531]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:03.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:03.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:03.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:05.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:05 smithi083 ceph-mon[35133]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:05.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:05.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:05.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:05.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:05 smithi057 ceph-mon[32331]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:05.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:05.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:05.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:05.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:05 smithi189 ceph-mon[35531]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:05.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:05.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:05.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:06.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-13T05:07:06.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:06.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-13T05:07:06.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:06.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-13T05:07:06.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:07.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:07 smithi057 ceph-mon[32331]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:07.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:07 smithi057 ceph-mon[32331]: Deploying daemon osd.0 on smithi057 2024-04-13T05:07:07.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:07 smithi083 ceph-mon[35133]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:07.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:07 smithi083 ceph-mon[35133]: Deploying daemon osd.0 on smithi057 2024-04-13T05:07:07.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:07 smithi189 ceph-mon[35531]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:07.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:07 smithi189 ceph-mon[35531]: Deploying daemon osd.0 on smithi057 2024-04-13T05:07:09.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:09 smithi057 ceph-mon[32331]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:09.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:09 smithi083 ceph-mon[35133]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:09.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:09 smithi189 ceph-mon[35531]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:10.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:10 smithi083 ceph-mon[35133]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:10.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:10 smithi057 ceph-mon[32331]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:10.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:10 smithi189 ceph-mon[35531]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:12.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:12 smithi057 ceph-mon[32331]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:12.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:12 smithi189 ceph-mon[35531]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:12.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:12 smithi083 ceph-mon[35133]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:14.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:14 smithi083 ceph-mon[35133]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:14.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:14.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:14.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:14.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:14 smithi057 ceph-mon[32331]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:14.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:14.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:14.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:14.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:14 smithi189 ceph-mon[35531]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:14.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:14.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:14.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:16.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:16 smithi057 ceph-mon[32331]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:16.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:16 smithi189 ceph-mon[35531]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:16.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:16 smithi083 ceph-mon[35133]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:18 smithi057 ceph-mon[32331]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-13T05:07:19.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:19.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:19.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:19.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:18 smithi189 ceph-mon[35531]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:19.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:18 smithi083 ceph-mon[35133]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-13T05:07:19.333 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2024-04-13T05:07:19.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.332+0000 7fad89600700 1 -- 172.21.15.57:0/3953738917 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fad8c062ef0 con 0x7fad70039890 2024-04-13T05:07:19.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fad70039890 msgr2=0x7fad7003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:19.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fad70039890 0x7fad7003bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009cb0 tx=0x7fad7c009260 comp rx=0 tx=0).stop 2024-04-13T05:07:19.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 msgr2=0x7fad8c1a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:19.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c1a6690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad84015000 tx=0x7fad84004cf0 comp rx=0 tx=0).stop 2024-04-13T05:07:19.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 shutdown_connections 2024-04-13T05:07:19.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fad70039890 0x7fad7003bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad8c079070 0x7fad8c1a6690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8c077460 0x7fad8c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 --2- 172.21.15.57:0/3953738917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad8c0786b0 0x7fad8c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.336+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 >> 172.21.15.57:0/3953738917 conn(0x7fad8c1001f0 msgr2=0x7fad8c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:07:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.337+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 shutdown_connections 2024-04-13T05:07:19.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:07:19.337+0000 7fad91a00700 1 -- 172.21.15.57:0/3953738917 wait complete. 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: osdmap e6: 1 total, 0 up, 1 in 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:20.152 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:20.234 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.0.service 2024-04-13T05:07:20.239 INFO:tasks.cephadm:Deploying osd.1 on smithi083 with /dev/vg_nvme/lv_4... 2024-04-13T05:07:20.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: osdmap e6: 1 total, 0 up, 1 in 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:20.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:20.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-13T05:07:20.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: osdmap e6: 1 total, 0 up, 1 in 2024-04-13T05:07:20.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:07:20.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:20.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:20.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:20.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:07:21.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-mon[32331]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-mon[32331]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:07:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-mon[32331]: osdmap e7: 1 total, 0 up, 1 in 2024-04-13T05:07:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-mon[32331]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' 2024-04-13T05:07:21.031 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0[47287]: 2024-04-13T05:07:20.685+0000 7efd27400700 -1 osd.0 0 waiting for initial osdmap 2024-04-13T05:07:21.031 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:07:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0[47287]: 2024-04-13T05:07:20.707+0000 7efd1d200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:07:21.106 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:20 smithi083 ceph-mon[35133]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:21.106 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:20 smithi083 ceph-mon[35133]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:07:21.106 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:20 smithi083 ceph-mon[35133]: osdmap e7: 1 total, 0 up, 1 in 2024-04-13T05:07:21.107 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.107 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.107 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:20 smithi083 ceph-mon[35133]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' 2024-04-13T05:07:21.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:20 smithi189 ceph-mon[35531]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:21.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:20 smithi189 ceph-mon[35531]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:07:21.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:20 smithi189 ceph-mon[35531]: osdmap e7: 1 total, 0 up, 1 in 2024-04-13T05:07:21.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:20 smithi189 ceph-mon[35531]: from='osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452]' entity='osd.0' 2024-04-13T05:07:21.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:21 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:07:21.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:21 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:07:21.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:21 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:21.983 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:21 smithi083 ceph-mon[35133]: osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] boot 2024-04-13T05:07:21.984 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:21 smithi083 ceph-mon[35133]: osdmap e8: 1 total, 1 up, 1 in 2024-04-13T05:07:21.984 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:21 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:22.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:21 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:07:22.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:21 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:07:22.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:22.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:21 smithi057 ceph-mon[32331]: osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] boot 2024-04-13T05:07:22.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:21 smithi057 ceph-mon[32331]: osdmap e8: 1 total, 1 up, 1 in 2024-04-13T05:07:22.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:22.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:21 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:07:22.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:21 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:07:22.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:22.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:21 smithi189 ceph-mon[35531]: osd.0 [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] boot 2024-04-13T05:07:22.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:21 smithi189 ceph-mon[35531]: osdmap e8: 1 total, 1 up, 1 in 2024-04-13T05:07:22.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-13T05:07:23.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:22 smithi083 ceph-mon[35133]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:23.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:22 smithi057 ceph-mon[32331]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:23.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:22 smithi189 ceph-mon[35531]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-13T05:07:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-13T05:07:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-13T05:07:23.498 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-13T05:07:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-13T05:07:23.498 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0332616 s, 315 MB/s 2024-04-13T05:07:23.498 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-13T05:07:23.973 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:23 smithi057 ceph-mon[32331]: osdmap e9: 1 total, 1 up, 1 in 2024-04-13T05:07:24.164 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:23 smithi083 ceph-mon[35133]: osdmap e9: 1 total, 1 up, 1 in 2024-04-13T05:07:24.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-04-13T05:07:24.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:23 smithi189 ceph-mon[35531]: osdmap e9: 1 total, 1 up, 1 in 2024-04-13T05:07:24.730 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:07:25.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:24 smithi057 ceph-mon[32331]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:25.171 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:24 smithi083 ceph-mon[35133]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:25.232 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:24 smithi189 ceph-mon[35531]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:26.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi057 2024-04-13T05:07:26.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:07:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi057 to 17094M 2024-04-13T05:07:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:26 smithi083 ceph-mon[35133]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:26.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi057 2024-04-13T05:07:26.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi057 to 17094M 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:26 smithi057 ceph-mon[32331]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:26.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi057 2024-04-13T05:07:26.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi057 to 17094M 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:26.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:26 smithi189 ceph-mon[35531]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.704+0000 7f8719600700 1 -- 172.21.15.83:0/1872530936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714104d50 msgr2=0x7f8714105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:26.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.704+0000 7f8719600700 1 --2- 172.21.15.83:0/1872530936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714104d50 0x7f8714105170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f87040099c0 tx=0x7f8704009cd0 comp rx=0 tx=0).stop 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 -- 172.21.15.83:0/1872530936 shutdown_connections 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 --2- 172.21.15.83:0/1872530936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87141068f0 0x7f871410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 --2- 172.21.15.83:0/1872530936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714104d50 0x7f8714105170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 --2- 172.21.15.83:0/1872530936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8714105f50 0x7f87141063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 -- 172.21.15.83:0/1872530936 >> 172.21.15.83:0/1872530936 conn(0x7f8714100310 msgr2=0x7f8714102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 -- 172.21.15.83:0/1872530936 shutdown_connections 2024-04-13T05:07:26.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.705+0000 7f8719600700 1 -- 172.21.15.83:0/1872530936 wait complete. 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 Processor -- start 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 -- start start 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 0x7f871419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714105f50 0x7f871419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87141068f0 0x7f87141a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8714110810 con 0x7f87141068f0 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8714110690 con 0x7f8714105f50 2024-04-13T05:07:26.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.706+0000 7f8719600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8714110990 con 0x7f8714104d50 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.707+0000 7f8712a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 0x7f871419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.707+0000 7f8712a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 0x7f871419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41412/0 (socket says 172.21.15.83:41412) 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.707+0000 7f8712a00700 1 -- 172.21.15.83:0/1991168075 learned_addr learned my addr 172.21.15.83:0/1991168075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.707+0000 7f8713400700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87141068f0 0x7f87141a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.707+0000 7f870be00700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714105f50 0x7f871419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.708+0000 7f8712a00700 1 -- 172.21.15.83:0/1991168075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714105f50 msgr2=0x7f871419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.708+0000 7f8712a00700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714105f50 0x7f871419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.708+0000 7f8712a00700 1 -- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87141068f0 msgr2=0x7f87141a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.708+0000 7f8712a00700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87141068f0 0x7f87141a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:26.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.708+0000 7f8712a00700 1 -- 172.21.15.83:0/1991168075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87141a2af0 con 0x7f8714104d50 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.708+0000 7f8712a00700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 0x7f871419ae90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f87040047b0 tx=0x7f8704009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.709+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704027070 con 0x7f8714104d50 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.709+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87141a2d80 con 0x7f8714104d50 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.709+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8704004050 con 0x7f8714104d50 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.709+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87141a3180 con 0x7f8714104d50 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.709+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704022cb0 con 0x7f8714104d50 2024-04-13T05:07:26.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.710+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871402dfa0 con 0x7f8714104d50 2024-04-13T05:07:26.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.714+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f870402a070 con 0x7f8714104d50 2024-04-13T05:07:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.715+0000 7f870b400700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86f80a3cb0 0x7f86f80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:26.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.716+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f87040cf9d0 con 0x7f8714104d50 2024-04-13T05:07:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.716+0000 7f870be00700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86f80a3cb0 0x7f86f80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:26.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.717+0000 7f870be00700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86f80a3cb0 0x7f86f80a6170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:07:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.726+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8704031080 con 0x7f8714104d50 2024-04-13T05:07:26.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:26.963+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8714062ef0 con 0x7f86f80a3cb0 2024-04-13T05:07:27.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:27 smithi189 ceph-mon[35531]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:07:27.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:07:27.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:07:27.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:27.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:27 smithi083 ceph-mon[35133]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:07:27.748 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:07:27.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:07:27.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:27.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:27 smithi057 ceph-mon[32331]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:07:27.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:07:27.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:07:27.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:28.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:28 smithi083 ceph-mon[35133]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:28.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:28 smithi189 ceph-mon[35531]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:28.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:28 smithi057 ceph-mon[32331]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:30.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:30 smithi057 ceph-mon[32331]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:30.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:30 smithi189 ceph-mon[35531]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:30.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:30 smithi083 ceph-mon[35133]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:31.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:31 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/2749055785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]: dispatch 2024-04-13T05:07:31.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:31 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]: dispatch 2024-04-13T05:07:31.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:31 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]': finished 2024-04-13T05:07:31.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:31 smithi057 ceph-mon[32331]: osdmap e10: 2 total, 1 up, 2 in 2024-04-13T05:07:31.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:31.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:31 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/2749055785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]: dispatch 2024-04-13T05:07:31.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:31 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]: dispatch 2024-04-13T05:07:31.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:31 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]': finished 2024-04-13T05:07:31.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:31 smithi189 ceph-mon[35531]: osdmap e10: 2 total, 1 up, 2 in 2024-04-13T05:07:31.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:31.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:31 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/2749055785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]: dispatch 2024-04-13T05:07:31.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:31 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]: dispatch 2024-04-13T05:07:31.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:31 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3eb9f311-c136-431f-a658-5084aa4c2fc6"}]': finished 2024-04-13T05:07:31.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:31 smithi083 ceph-mon[35133]: osdmap e10: 2 total, 1 up, 2 in 2024-04-13T05:07:31.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:32.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:32 smithi057 ceph-mon[32331]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:32.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:32 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/3453684343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:07:32.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:32 smithi189 ceph-mon[35531]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:32.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:32 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/3453684343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:07:32.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:32 smithi083 ceph-mon[35133]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:32.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:32 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/3453684343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:07:34.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:34 smithi057 ceph-mon[32331]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:34.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:34 smithi189 ceph-mon[35531]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:34.837 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:34 smithi083 ceph-mon[35133]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:36.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:36 smithi083 ceph-mon[35133]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:36 smithi189 ceph-mon[35531]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:37.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:36 smithi057 ceph-mon[32331]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:38.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:38 smithi083 ceph-mon[35133]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:38.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:38 smithi189 ceph-mon[35531]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:39.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:38 smithi057 ceph-mon[32331]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:40.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:40 smithi189 ceph-mon[35531]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:40.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:40 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-13T05:07:40.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:40 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:40.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:40 smithi189 ceph-mon[35531]: Deploying daemon osd.1 on smithi083 2024-04-13T05:07:40.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:40 smithi083 ceph-mon[35133]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:40.843 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:40 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-13T05:07:40.844 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:40 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:40.844 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:40 smithi083 ceph-mon[35133]: Deploying daemon osd.1 on smithi083 2024-04-13T05:07:41.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:40 smithi057 ceph-mon[32331]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:41.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:40 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-13T05:07:41.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:40 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:41.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:40 smithi057 ceph-mon[32331]: Deploying daemon osd.1 on smithi083 2024-04-13T05:07:42.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:42 smithi083 ceph-mon[35133]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:42.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:42 smithi189 ceph-mon[35531]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:43.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:42 smithi057 ceph-mon[32331]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:44.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:44 smithi189 ceph-mon[35531]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:44.899 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:44 smithi083 ceph-mon[35133]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:45.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:44 smithi057 ceph-mon[32331]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:46.696 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:46 smithi083 ceph-mon[35133]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:46.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:46 smithi189 ceph-mon[35531]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:47.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:46 smithi057 ceph-mon[32331]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:48.712 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:48 smithi083 ceph-mon[35133]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:48.712 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:48.712 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:48.712 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:48.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:48 smithi057 ceph-mon[32331]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:48.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:48.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:48.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:48.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:48 smithi189 ceph-mon[35531]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:48.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:48.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:48.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:50.593 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:50 smithi083 ceph-mon[35133]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:50.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:50 smithi057 ceph-mon[32331]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:50.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:50 smithi189 ceph-mon[35531]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-13T05:07:51.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:51 smithi083 ceph-mon[35133]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-13T05:07:51.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:51.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:51.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:51.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:51.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:51.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-13T05:07:51.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:51 smithi189 ceph-mon[35531]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-13T05:07:52.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:52.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:52.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:52.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:52.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:52.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-13T05:07:52.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:51 smithi057 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-13T05:07:52.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.225+0000 7f870b400700 1 -- 172.21.15.83:0/1991168075 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8714062ef0 con 0x7f86f80a3cb0 2024-04-13T05:07:52.225 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 1 on host 'smithi083' 2024-04-13T05:07:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.228+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86f80a3cb0 msgr2=0x7f86f80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.228+0000 7f8719600700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86f80a3cb0 0x7f86f80a6170 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc00b410 comp rx=0 tx=0).stop 2024-04-13T05:07:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 msgr2=0x7f871419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 0x7f871419ae90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f87040047b0 tx=0x7f8704009580 comp rx=0 tx=0).stop 2024-04-13T05:07:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 shutdown_connections 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8714104d50 0x7f871419ae90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86f80a3cb0 0x7f86f80a6170 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87141068f0 0x7f87141a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 --2- 172.21.15.83:0/1991168075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8714105f50 0x7f871419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 >> 172.21.15.83:0/1991168075 conn(0x7f8714100310 msgr2=0x7f8714109370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 shutdown_connections 2024-04-13T05:07:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:07:52.229+0000 7f8719600700 1 -- 172.21.15.83:0/1991168075 wait complete. 2024-04-13T05:07:52.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:52.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-13T05:07:52.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:07:52.969 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: osdmap e11: 2 total, 1 up, 2 in 2024-04-13T05:07:52.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:52.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:07:52.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:52.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:52.970 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:53.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:53.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: osdmap e11: 2 total, 1 up, 2 in 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:53.037 DEBUG:teuthology.orchestra.run.smithi083:osd.1> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.1.service 2024-04-13T05:07:53.043 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-04-13T05:07:53.043 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-13T05:07:53.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:53.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-13T05:07:53.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:07:53.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: osdmap e11: 2 total, 1 up, 2 in 2024-04-13T05:07:53.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:07:53.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:53.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:53.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:07:53.478 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:07:53.633 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1[39681]: 2024-04-13T05:07:53.488+0000 7fa243e00700 -1 osd.1 0 waiting for initial osdmap 2024-04-13T05:07:53.634 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1[39681]: 2024-04-13T05:07:53.516+0000 7fa239c00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:07:53.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:53 smithi057 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:07:53.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:53 smithi057 ceph-mon[32331]: osdmap e12: 2 total, 1 up, 2 in 2024-04-13T05:07:53.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:53 smithi057 ceph-mon[32331]: from='osd.1 ' entity='osd.1' 2024-04-13T05:07:53.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:53 smithi189 ceph-mon[35531]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:07:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:53 smithi189 ceph-mon[35531]: osdmap e12: 2 total, 1 up, 2 in 2024-04-13T05:07:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:53 smithi189 ceph-mon[35531]: from='osd.1 ' entity='osd.1' 2024-04-13T05:07:53.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-mon[35133]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:07:53.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-mon[35133]: osdmap e12: 2 total, 1 up, 2 in 2024-04-13T05:07:53.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:53.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:53 smithi083 ceph-mon[35133]: from='osd.1 ' entity='osd.1' 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] boot 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: osdmap e13: 2 total, 2 up, 2 in 2024-04-13T05:07:54.742 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] boot 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: osdmap e13: 2 total, 2 up, 2 in 2024-04-13T05:07:54.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:55.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:07:55.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:07:55.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-13T05:07:55.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:55.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: osd.1 [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] boot 2024-04-13T05:07:55.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: osdmap e13: 2 total, 2 up, 2 in 2024-04-13T05:07:55.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-13T05:07:56.271 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-13T05:07:56.271 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-13T05:07:56.271 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-13T05:07:56.271 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-13T05:07:56.271 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0364559 s, 288 MB/s 2024-04-13T05:07:56.271 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-13T05:07:56.998 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:56 smithi083 ceph-mon[35133]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:07:56.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:56 smithi083 ceph-mon[35133]: osdmap e14: 2 total, 2 up, 2 in 2024-04-13T05:07:57.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:56 smithi057 ceph-mon[32331]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:07:57.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:56 smithi057 ceph-mon[32331]: osdmap e14: 2 total, 2 up, 2 in 2024-04-13T05:07:57.032 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:56 smithi189 ceph-mon[35531]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:07:57.032 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:56 smithi189 ceph-mon[35531]: osdmap e14: 2 total, 2 up, 2 in 2024-04-13T05:07:57.099 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-04-13T05:07:57.572 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi083 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-13T05:07:59.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.257 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:59.257 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:59.257 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:07:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi083 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-13T05:07:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:59.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:59.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:07:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:07:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi083 2024-04-13T05:07:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:07:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-13T05:07:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:07:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:07:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:07:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:07:59.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.705+0000 7f890a800700 1 -- 172.21.15.189:0/172197639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410acf0 msgr2=0x7f89041115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:59.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.705+0000 7f890a800700 1 --2- 172.21.15.189:0/172197639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410acf0 0x7f89041115a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00b0a0 tx=0x7f88fc00b3b0 comp rx=0 tx=0).stop 2024-04-13T05:07:59.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 -- 172.21.15.189:0/172197639 shutdown_connections 2024-04-13T05:07:59.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 --2- 172.21.15.189:0/172197639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410acf0 0x7f89041115a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:59.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 --2- 172.21.15.189:0/172197639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8904109150 0x7f8904109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:59.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 --2- 172.21.15.189:0/172197639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890410a350 0x7f890410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:59.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 -- 172.21.15.189:0/172197639 >> 172.21.15.189:0/172197639 conn(0x7f8904078120 msgr2=0x7f890407a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:07:59.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 -- 172.21.15.189:0/172197639 shutdown_connections 2024-04-13T05:07:59.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.706+0000 7f890a800700 1 -- 172.21.15.189:0/172197639 wait complete. 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.707+0000 7f890a800700 1 Processor -- start 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.707+0000 7f890a800700 1 -- start start 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f890a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8904109150 0x7f890419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f890a800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 0x7f890419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f890a800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890410acf0 0x7f89041a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f890a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8904114c40 con 0x7f8904109150 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f890a800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8904114ac0 con 0x7f890410acf0 2024-04-13T05:07:59.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f890a800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8904114dc0 con 0x7f890410a350 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f8902a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 0x7f890419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f8902a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 0x7f890419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:58044/0 (socket says 172.21.15.189:58044) 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f8902a00700 1 -- 172.21.15.189:0/1497593082 learned_addr learned my addr 172.21.15.189:0/1497593082 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.708+0000 7f8903400700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8904109150 0x7f890419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8903e00700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890410acf0 0x7f89041a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8902a00700 1 -- 172.21.15.189:0/1497593082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890410acf0 msgr2=0x7f89041a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8902a00700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890410acf0 0x7f89041a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8902a00700 1 -- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8904109150 msgr2=0x7f890419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:07:59.710 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8902a00700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8904109150 0x7f890419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8902a00700 1 -- 172.21.15.189:0/1497593082 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89041a6df0 con 0x7f890410a350 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f8902a00700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 0x7f890419f6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88f000d9b0 tx=0x7f88f000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0017070 con 0x7f890410a350 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88f0004830 con 0x7f890410a350 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89041a70e0 con 0x7f890410a350 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.709+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f00053a0 con 0x7f890410a350 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.710+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041a7620 con 0x7f890410a350 2024-04-13T05:07:59.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.711+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89040697a0 con 0x7f890410a350 2024-04-13T05:07:59.713 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.714+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f88f00049e0 con 0x7f890410a350 2024-04-13T05:07:59.715 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.715+0000 7f88fbe00700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88e80a3c30 0x7f88e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:07:59.715 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.715+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f88f00ceae0 con 0x7f890410a350 2024-04-13T05:07:59.715 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.716+0000 7f8903400700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88e80a3c30 0x7f88e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:07:59.716 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.717+0000 7f8903400700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88e80a3c30 0x7f88e80a60f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f4005fd0 tx=0x7f88f4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:07:59.721 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.721+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f0020020 con 0x7f890410a350 2024-04-13T05:07:59.955 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:07:59.955+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8904062fb0 con 0x7f88e80a3c30 2024-04-13T05:08:00.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:00 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:08:00.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:00 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:08:00.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:00 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:00.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:00 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:08:00.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:00 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:08:00.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:00 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:00.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:00 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:08:00.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:00 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:08:00.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:00 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:01.220 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:01 smithi189 ceph-mon[35531]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:01.220 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:01 smithi189 ceph-mon[35531]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:08:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:01 smithi083 ceph-mon[35133]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:01 smithi083 ceph-mon[35133]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:08:01.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:01 smithi057 ceph-mon[32331]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:01.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:01 smithi057 ceph-mon[32331]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:08:03.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:03 smithi083 ceph-mon[35133]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:03.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:03 smithi057 ceph-mon[32331]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:03.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:03 smithi189 ceph-mon[35531]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:04.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:04 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1336800749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]: dispatch 2024-04-13T05:08:04.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:04 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]: dispatch 2024-04-13T05:08:04.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:04 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]': finished 2024-04-13T05:08:04.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:04 smithi189 ceph-mon[35531]: osdmap e15: 3 total, 2 up, 3 in 2024-04-13T05:08:04.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:04 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:04.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:04 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1336800749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]: dispatch 2024-04-13T05:08:04.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:04 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]: dispatch 2024-04-13T05:08:04.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:04 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]': finished 2024-04-13T05:08:04.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:04 smithi083 ceph-mon[35133]: osdmap e15: 3 total, 2 up, 3 in 2024-04-13T05:08:04.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:04 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:04.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:04 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1336800749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]: dispatch 2024-04-13T05:08:04.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:04 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]: dispatch 2024-04-13T05:08:04.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:04 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83f5d37c-64d6-4a58-81c2-c0f1b7a492ac"}]': finished 2024-04-13T05:08:04.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:04 smithi057 ceph-mon[32331]: osdmap e15: 3 total, 2 up, 3 in 2024-04-13T05:08:04.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:04 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:05.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:05 smithi057 ceph-mon[32331]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:05.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:05 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1766238845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:08:05.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:05 smithi083 ceph-mon[35133]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:05.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:05 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1766238845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:08:05.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:05 smithi189 ceph-mon[35531]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:05.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:05 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1766238845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:08:07.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:07 smithi083 ceph-mon[35133]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:07.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:07 smithi057 ceph-mon[32331]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:07.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:07 smithi189 ceph-mon[35531]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:09.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:09 smithi083 ceph-mon[35133]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:09.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:09 smithi057 ceph-mon[32331]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:09.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:09 smithi189 ceph-mon[35531]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:11.328 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:11 smithi189 ceph-mon[35531]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:11.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:11 smithi083 ceph-mon[35133]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:11.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:11 smithi057 ceph-mon[32331]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:13.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:13 smithi083 ceph-mon[35133]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:13.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-13T05:08:13.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:13.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:13 smithi057 ceph-mon[32331]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:13.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-13T05:08:13.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:13.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:13 smithi189 ceph-mon[35531]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:13.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-13T05:08:13.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:14.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:14 smithi189 ceph-mon[35531]: Deploying daemon osd.2 on smithi189 2024-04-13T05:08:14.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:14 smithi083 ceph-mon[35133]: Deploying daemon osd.2 on smithi189 2024-04-13T05:08:14.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:14 smithi057 ceph-mon[32331]: Deploying daemon osd.2 on smithi189 2024-04-13T05:08:15.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:15 smithi189 ceph-mon[35531]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:15.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:15 smithi083 ceph-mon[35133]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:15 smithi057 ceph-mon[32331]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:17.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:17 smithi083 ceph-mon[35133]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:17.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:17 smithi057 ceph-mon[32331]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:17.554 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:17 smithi189 ceph-mon[35531]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:18.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:18 smithi189 ceph-mon[35531]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:18.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:18 smithi083 ceph-mon[35133]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:18.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:18 smithi057 ceph-mon[32331]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:20.682 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:20 smithi189 ceph-mon[35531]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:20.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:20 smithi057 ceph-mon[32331]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:20.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:20 smithi083 ceph-mon[35133]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:22.220 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:22.220 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:22.220 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:21 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:22.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:21 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:22.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:21 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:22.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:21 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:22.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:21 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:23.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:22 smithi189 ceph-mon[35531]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:23.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:22 smithi057 ceph-mon[32331]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:23.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:22 smithi083 ceph-mon[35133]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:25.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:24 smithi083 ceph-mon[35133]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:25.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:24 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:25.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:24 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:25.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:24 smithi057 ceph-mon[32331]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:25.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:24 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:25.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:24 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:25.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:24 smithi189 ceph-mon[35531]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:25.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:24 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:25.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:24 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:25.778 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.777+0000 7f88fbe00700 1 -- 172.21.15.189:0/1497593082 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8904062fb0 con 0x7f88e80a3c30 2024-04-13T05:08:25.778 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88e80a3c30 msgr2=0x7f88e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88e80a3c30 0x7f88e80a60f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f4005fd0 tx=0x7f88f4005e50 comp rx=0 tx=0).stop 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 msgr2=0x7f890419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 0x7f890419f6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88f000d9b0 tx=0x7f88f000dd70 comp rx=0 tx=0).stop 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 shutdown_connections 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88e80a3c30 0x7f88e80a60f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890410acf0 0x7f89041a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:25.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8904109150 0x7f890419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:25.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 --2- 172.21.15.189:0/1497593082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f890410a350 0x7f890419f6d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:25.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.781+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 >> 172.21.15.189:0/1497593082 conn(0x7f8904078120 msgr2=0x7f890410d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:08:25.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.782+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 shutdown_connections 2024-04-13T05:08:25.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:08:25.782+0000 7f890a800700 1 -- 172.21.15.189:0/1497593082 wait complete. 2024-04-13T05:08:26.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-13T05:08:26.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-13T05:08:26.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:26.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:08:26.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:25 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:26.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-13T05:08:26.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-13T05:08:26.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:26.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:08:26.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:25 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:26.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-13T05:08:26.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-13T05:08:26.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:26.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:08:26.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:26.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:25 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:26.628 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.2.service 2024-04-13T05:08:26.633 INFO:tasks.cephadm:Deploying osd.3 on smithi057 with /dev/vg_nvme/lv_3... 2024-04-13T05:08:26.633 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-13T05:08:27.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:27.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-13T05:08:27.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: osdmap e16: 3 total, 2 up, 3 in 2024-04-13T05:08:27.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: from='osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:08:27.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:27.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:08:27.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:08:27.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: osdmap e17: 3 total, 2 up, 3 in 2024-04-13T05:08:27.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:27.101 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: osdmap e16: 3 total, 2 up, 3 in 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: from='osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: osdmap e17: 3 total, 2 up, 3 in 2024-04-13T05:08:27.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:27.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:27.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: osdmap e16: 3 total, 2 up, 3 in 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: from='osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: osdmap e17: 3 total, 2 up, 3 in 2024-04-13T05:08:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:28.080 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:27 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:08:28.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:27 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:08:28.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:28.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:27 smithi189 ceph-mon[35531]: from='osd.2 ' entity='osd.2' 2024-04-13T05:08:28.081 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2[40679]: 2024-04-13T05:08:27.771+0000 7f93ea600700 -1 osd.2 0 waiting for initial osdmap 2024-04-13T05:08:28.081 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2[40679]: 2024-04-13T05:08:27.793+0000 7f93e0400700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:08:28.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:27 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:08:28.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:27 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:08:28.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:28.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:27 smithi083 ceph-mon[35133]: from='osd.2 ' entity='osd.2' 2024-04-13T05:08:28.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:27 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:08:28.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:27 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:08:28.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:28.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:27 smithi057 ceph-mon[32331]: from='osd.2 ' entity='osd.2' 2024-04-13T05:08:29.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:28 smithi057 ceph-mon[32331]: pgmap v105: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:29.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:29.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:28 smithi057 ceph-mon[32331]: osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] boot 2024-04-13T05:08:29.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:28 smithi057 ceph-mon[32331]: osdmap e18: 3 total, 3 up, 3 in 2024-04-13T05:08:29.028 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:29.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:28 smithi083 ceph-mon[35133]: pgmap v105: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:29.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:29.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:28 smithi083 ceph-mon[35133]: osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] boot 2024-04-13T05:08:29.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:28 smithi083 ceph-mon[35133]: osdmap e18: 3 total, 3 up, 3 in 2024-04-13T05:08:29.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:29.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:28 smithi189 ceph-mon[35531]: pgmap v105: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-13T05:08:29.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:29.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:28 smithi189 ceph-mon[35531]: osd.2 [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] boot 2024-04-13T05:08:29.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:28 smithi189 ceph-mon[35531]: osdmap e18: 3 total, 3 up, 3 in 2024-04-13T05:08:29.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-13T05:08:30.219 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:29 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:08:30.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:29 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:08:30.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:29 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:08:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-13T05:08:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-13T05:08:30.422 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-04-13T05:08:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-04-13T05:08:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.033386 s, 314 MB/s 2024-04-13T05:08:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-04-13T05:08:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:30 smithi083 ceph-mon[35133]: pgmap v107: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-13T05:08:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:30 smithi083 ceph-mon[35133]: osdmap e19: 3 total, 3 up, 3 in 2024-04-13T05:08:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:08:31.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:30 smithi057 ceph-mon[32331]: pgmap v107: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:31.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-13T05:08:31.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:30 smithi057 ceph-mon[32331]: osdmap e19: 3 total, 3 up, 3 in 2024-04-13T05:08:31.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:08:31.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:30 smithi189 ceph-mon[35531]: pgmap v107: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:31.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-13T05:08:31.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:30 smithi189 ceph-mon[35531]: osdmap e19: 3 total, 3 up, 3 in 2024-04-13T05:08:31.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:08:31.361 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2024-04-13T05:08:31.749 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43918]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43918]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43921]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43921]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43921]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:31.750 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43921]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:31.779 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51840]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-13T05:08:31.779 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51840]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:31.780 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51840]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:31.780 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51840]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:31.780 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51851]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-13T05:08:31.780 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51851]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:31.780 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51851]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:31.780 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51851]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:31.816 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44402]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44402]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44402]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:31.878 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44405]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:31.879 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44405]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51901]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51901]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51917]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:32.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:31 smithi057 sudo[51917]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:31 smithi083 sudo[43924]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 sudo[43924]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 sudo[43927]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 sudo[43927]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 sudo[43927]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 sudo[43927]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: osdmap e20: 3 total, 3 up, 3 in 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi189 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi189 to 17089M 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44408]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44408]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44411]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44411]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:31 smithi189 sudo[44411]: pam_unix(sudo:session): session closed for user root 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-13T05:08:32.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: osdmap e20: 3 total, 3 up, 3 in 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi189 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi189 to 17089M 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:32.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: osdmap e20: 3 total, 3 up, 3 in 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi189 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi189 to 17089M 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:08:32.390 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-13T05:08:32.391 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-13T05:08:33.204 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:33 smithi057 ceph-mon[32331]: pgmap v110: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:33.204 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:33 smithi057 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:33.204 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:33 smithi057 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:33.204 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:33 smithi057 ceph-mon[32331]: osdmap e21: 3 total, 3 up, 3 in 2024-04-13T05:08:33.204 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:33 smithi057 ceph-mon[32331]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-13T05:08:33.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:33 smithi189 ceph-mon[35531]: pgmap v110: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:33.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:33 smithi189 ceph-mon[35531]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:33.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:33 smithi189 ceph-mon[35531]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:33.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:33 smithi189 ceph-mon[35531]: osdmap e21: 3 total, 3 up, 3 in 2024-04-13T05:08:33.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:33 smithi189 ceph-mon[35531]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-13T05:08:33.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:33 smithi083 ceph-mon[35133]: pgmap v110: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:33.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:33 smithi083 ceph-mon[35133]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-13T05:08:33.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:33 smithi083 ceph-mon[35133]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-13T05:08:33.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:33 smithi083 ceph-mon[35133]: osdmap e21: 3 total, 3 up, 3 in 2024-04-13T05:08:33.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:33 smithi083 ceph-mon[35133]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-13T05:08:34.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 -- 172.21.15.57:0/1793939792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af4105ec0 msgr2=0x7f4af41082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 --2- 172.21.15.57:0/1793939792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af4105ec0 0x7f4af41082b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009990 tx=0x7f4ae0009ca0 comp rx=0 tx=0).stop 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 -- 172.21.15.57:0/1793939792 shutdown_connections 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 --2- 172.21.15.57:0/1793939792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af41087f0 0x7f4af410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 --2- 172.21.15.57:0/1793939792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af4103590 0x7f4af4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 --2- 172.21.15.57:0/1793939792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af4105ec0 0x7f4af41082b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 -- 172.21.15.57:0/1793939792 >> 172.21.15.57:0/1793939792 conn(0x7f4af40fcfa0 msgr2=0x7f4af40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 -- 172.21.15.57:0/1793939792 shutdown_connections 2024-04-13T05:08:34.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.006+0000 7f4afbe00700 1 -- 172.21.15.57:0/1793939792 wait complete. 2024-04-13T05:08:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.007+0000 7f4afbe00700 1 Processor -- start 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.007+0000 7f4afbe00700 1 -- start start 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.007+0000 7f4afbe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af4103590 0x7f4af419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.007+0000 7f4afbe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af4105ec0 0x7f4af419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4afbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 0x7f4af41a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4afbe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410e5b0 con 0x7f4af41087f0 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4afbe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410e430 con 0x7f4af4105ec0 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4afbe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410e730 con 0x7f4af4103590 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 0x7f4af41a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 0x7f4af41a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35946/0 (socket says 172.21.15.57:35946) 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 -- 172.21.15.57:0/46042588 learned_addr learned my addr 172.21.15.57:0/46042588 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:08:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9400700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af4103590 0x7f4af419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af8a00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af4105ec0 0x7f4af419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 -- 172.21.15.57:0/46042588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af4103590 msgr2=0x7f4af419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af4103590 0x7f4af419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 -- 172.21.15.57:0/46042588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af4105ec0 msgr2=0x7f4af419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af4105ec0 0x7f4af419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.008+0000 7f4af9e00700 1 -- 172.21.15.57:0/46042588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af41a2b00 con 0x7f4af41087f0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.009+0000 7f4af9e00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 0x7f4af41a2460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4aec007ee0 tx=0x7f4aec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.009+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec004db0 con 0x7f4af41087f0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.009+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aec00dbd0 con 0x7f4af41087f0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.009+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af41a2df0 con 0x7f4af41087f0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.009+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec015400 con 0x7f4af41087f0 2024-04-13T05:08:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.009+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41a32a0 con 0x7f4af41087f0 2024-04-13T05:08:34.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.011+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af40697a0 con 0x7f4af41087f0 2024-04-13T05:08:34.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.013+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4aec005000 con 0x7f4af41087f0 2024-04-13T05:08:34.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.014+0000 7f4af2000700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4adc0a3c40 0x7f4adc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:08:34.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.014+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f4aec0ceb80 con 0x7f4af41087f0 2024-04-13T05:08:34.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.014+0000 7f4af9400700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4adc0a3c40 0x7f4adc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:08:34.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.015+0000 7f4af9400700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4adc0a3c40 0x7f4adc0a6100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005950 tx=0x7f4ae80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:08:34.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.019+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4aec010070 con 0x7f4af41087f0 2024-04-13T05:08:34.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:08:34.262+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4af4062ef0 con 0x7f4adc0a3c40 2024-04-13T05:08:35.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:35 smithi189 ceph-mon[35531]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:35.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:08:35.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:08:35.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:35.462 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:35 smithi057 ceph-mon[32331]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:35.462 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:08:35.462 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:08:35.462 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:35.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:35 smithi083 ceph-mon[35133]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:35.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:08:35.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:08:35.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:36.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:36 smithi057 ceph-mon[32331]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:08:36.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:36 smithi083 ceph-mon[35133]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:08:36.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:36 smithi189 ceph-mon[35531]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:08:37.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:37 smithi057 ceph-mon[32331]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:37.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:37 smithi083 ceph-mon[35133]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:37.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:37 smithi189 ceph-mon[35531]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:39.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:39 smithi189 ceph-mon[35531]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:39.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/748416446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e4840c9-26d6-4827-bca1-2fe9391d51ef"}]: dispatch 2024-04-13T05:08:39.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/748416446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e4840c9-26d6-4827-bca1-2fe9391d51ef"}]': finished 2024-04-13T05:08:39.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:39 smithi189 ceph-mon[35531]: osdmap e22: 4 total, 3 up, 4 in 2024-04-13T05:08:39.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:08:39.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:39 smithi083 ceph-mon[35133]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:39.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/748416446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e4840c9-26d6-4827-bca1-2fe9391d51ef"}]: dispatch 2024-04-13T05:08:39.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/748416446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e4840c9-26d6-4827-bca1-2fe9391d51ef"}]': finished 2024-04-13T05:08:39.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:39 smithi083 ceph-mon[35133]: osdmap e22: 4 total, 3 up, 4 in 2024-04-13T05:08:39.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:08:39.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:39 smithi057 ceph-mon[32331]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:39.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/748416446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e4840c9-26d6-4827-bca1-2fe9391d51ef"}]: dispatch 2024-04-13T05:08:39.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/748416446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e4840c9-26d6-4827-bca1-2fe9391d51ef"}]': finished 2024-04-13T05:08:39.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:39 smithi057 ceph-mon[32331]: osdmap e22: 4 total, 3 up, 4 in 2024-04-13T05:08:39.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:08:40.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:40 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3696673715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:08:40.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:40 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3696673715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:08:40.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:40 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3696673715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:08:41.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:41 smithi083 ceph-mon[35133]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:41.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:41 smithi057 ceph-mon[32331]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:41.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:41 smithi189 ceph-mon[35531]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:43 smithi057 ceph-mon[32331]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:43.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:43 smithi083 ceph-mon[35133]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:43.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:43 smithi189 ceph-mon[35531]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:45.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:45 smithi057 ceph-mon[32331]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:45.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:45 smithi083 ceph-mon[35133]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:45.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:45 smithi189 ceph-mon[35531]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:47.399 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:47 smithi057 ceph-mon[32331]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:47.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:47 smithi083 ceph-mon[35133]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:47.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:47 smithi189 ceph-mon[35531]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:49.473 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:49 smithi057 ceph-mon[32331]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:49.473 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-13T05:08:49.474 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:49 smithi083 ceph-mon[35133]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-13T05:08:49.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:49 smithi189 ceph-mon[35531]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-13T05:08:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:08:50.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:50 smithi057 ceph-mon[32331]: Deploying daemon osd.3 on smithi057 2024-04-13T05:08:50.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:50 smithi083 ceph-mon[35133]: Deploying daemon osd.3 on smithi057 2024-04-13T05:08:50.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:50 smithi189 ceph-mon[35531]: Deploying daemon osd.3 on smithi057 2024-04-13T05:08:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:51 smithi057 ceph-mon[32331]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:51.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:51 smithi083 ceph-mon[35133]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:51.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:51 smithi189 ceph-mon[35531]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:53.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:53 smithi057 ceph-mon[32331]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:53.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:53 smithi083 ceph-mon[35133]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:53.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:53 smithi189 ceph-mon[35531]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:55.229 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:55 smithi057 ceph-mon[32331]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:55.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:55 smithi083 ceph-mon[35133]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:55.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:55 smithi189 ceph-mon[35531]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:57.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:57 smithi189 ceph-mon[35531]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:57.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:57.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:57.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:57.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:57 smithi083 ceph-mon[35133]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:57.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:57.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:57.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:57.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:57 smithi057 ceph-mon[32331]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:57.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:57.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:08:57.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:08:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:08:59 smithi083 ceph-mon[35133]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:59.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:08:59 smithi057 ceph-mon[32331]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:08:59.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:08:59 smithi189 ceph-mon[35531]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:01 smithi083 ceph-mon[35133]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:01 smithi083 ceph-mon[35133]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-13T05:09:01.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:01 smithi057 ceph-mon[32331]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:01.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:01.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:01.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:01 smithi057 ceph-mon[32331]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-13T05:09:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:01 smithi189 ceph-mon[35531]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:01.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:01 smithi189 ceph-mon[35531]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-13T05:09:02.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.064+0000 7f4af2000700 1 -- 172.21.15.57:0/46042588 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4af4062ef0 con 0x7f4adc0a3c40 2024-04-13T05:09:02.065 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 3 on host 'smithi057' 2024-04-13T05:09:02.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4adc0a3c40 msgr2=0x7f4adc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:02.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4adc0a3c40 0x7f4adc0a6100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005950 tx=0x7f4ae80058e0 comp rx=0 tx=0).stop 2024-04-13T05:09:02.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 msgr2=0x7f4af41a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:02.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 0x7f4af41a2460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4aec007ee0 tx=0x7f4aec00d3b0 comp rx=0 tx=0).stop 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 shutdown_connections 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af4103590 0x7f4af419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4adc0a3c40 0x7f4adc0a6100 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4af41087f0 0x7f4af41a2460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 --2- 172.21.15.57:0/46042588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af4105ec0 0x7f4af419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.068+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 >> 172.21.15.57:0/46042588 conn(0x7f4af40fcfa0 msgr2=0x7f4af4102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.069+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 shutdown_connections 2024-04-13T05:09:02.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:09:02.069+0000 7f4afbe00700 1 -- 172.21.15.57:0/46042588 wait complete. 2024-04-13T05:09:02.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:02.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-13T05:09:02.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: osdmap e23: 4 total, 3 up, 4 in 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:02.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:02.918 DEBUG:teuthology.orchestra.run.smithi057:osd.3> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.3.service 2024-04-13T05:09:02.926 INFO:tasks.cephadm:Deploying osd.4 on smithi083 with /dev/vg_nvme/lv_3... 2024-04-13T05:09:02.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: osdmap e23: 4 total, 3 up, 4 in 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:03.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:03.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: osdmap e23: 4 total, 3 up, 4 in 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:03.367 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: osdmap e24: 4 total, 3 up, 4 in 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] boot 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: osdmap e25: 4 total, 4 up, 4 in 2024-04-13T05:09:03.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:09:03.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: osdmap e24: 4 total, 3 up, 4 in 2024-04-13T05:09:03.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:03.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] boot 2024-04-13T05:09:03.808 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: osdmap e25: 4 total, 4 up, 4 in 2024-04-13T05:09:03.809 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: from='osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: osdmap e24: 4 total, 3 up, 4 in 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: osd.3 [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] boot 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: osdmap e25: 4 total, 4 up, 4 in 2024-04-13T05:09:04.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-13T05:09:05.069 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:04 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:09:05.070 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:04 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:09:05.070 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:04 smithi083 ceph-mon[35133]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:05.070 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:04 smithi083 ceph-mon[35133]: osdmap e26: 4 total, 4 up, 4 in 2024-04-13T05:09:05.197 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:04 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:09:05.197 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:04 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:09:05.198 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:04 smithi057 ceph-mon[32331]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:05.198 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:04 smithi057 ceph-mon[32331]: osdmap e26: 4 total, 4 up, 4 in 2024-04-13T05:09:05.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:04 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:09:05.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:04 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:09:05.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:04 smithi189 ceph-mon[35531]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-13T05:09:05.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:04 smithi189 ceph-mon[35531]: osdmap e26: 4 total, 4 up, 4 in 2024-04-13T05:09:06.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:06 smithi083 ceph-mon[35133]: osdmap e27: 4 total, 4 up, 4 in 2024-04-13T05:09:06.267 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-13T05:09:06.267 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-13T05:09:06.267 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-13T05:09:06.267 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-13T05:09:06.268 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349998 s, 300 MB/s 2024-04-13T05:09:06.268 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-13T05:09:06.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:06 smithi057 ceph-mon[32331]: osdmap e27: 4 total, 4 up, 4 in 2024-04-13T05:09:06.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:06 smithi189 ceph-mon[35531]: osdmap e27: 4 total, 4 up, 4 in 2024-04-13T05:09:07.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-04-13T05:09:07.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:07 smithi083 ceph-mon[35133]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:07.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:07 smithi083 ceph-mon[35133]: osdmap e28: 4 total, 4 up, 4 in 2024-04-13T05:09:07.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:07 smithi083 ceph-mon[35133]: osdmap e29: 4 total, 4 up, 4 in 2024-04-13T05:09:07.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:07 smithi057 ceph-mon[32331]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:07.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:07 smithi057 ceph-mon[32331]: osdmap e28: 4 total, 4 up, 4 in 2024-04-13T05:09:07.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:07 smithi057 ceph-mon[32331]: osdmap e29: 4 total, 4 up, 4 in 2024-04-13T05:09:07.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:07 smithi189 ceph-mon[35531]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:07.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:07 smithi189 ceph-mon[35531]: osdmap e28: 4 total, 4 up, 4 in 2024-04-13T05:09:07.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:07 smithi189 ceph-mon[35531]: osdmap e29: 4 total, 4 up, 4 in 2024-04-13T05:09:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:09.329 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 -- 172.21.15.83:0/151504104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9301046b0 msgr2=0x7fb93010af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 --2- 172.21.15.83:0/151504104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9301046b0 0x7fb93010af60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb92800b0d0 tx=0x7fb92800b3e0 comp rx=0 tx=0).stop 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 -- 172.21.15.83:0/151504104 shutdown_connections 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 --2- 172.21.15.83:0/151504104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9301046b0 0x7fb93010af60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 --2- 172.21.15.83:0/151504104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb930102b10 0x7fb930102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 --2- 172.21.15.83:0/151504104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb930103d10 0x7fb930104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 -- 172.21.15.83:0/151504104 >> 172.21.15.83:0/151504104 conn(0x7fb9300fe0b0 msgr2=0x7fb9301004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 -- 172.21.15.83:0/151504104 shutdown_connections 2024-04-13T05:09:09.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.499+0000 7fb936000700 1 -- 172.21.15.83:0/151504104 wait complete. 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.500+0000 7fb936000700 1 Processor -- start 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.500+0000 7fb936000700 1 -- start start 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.500+0000 7fb936000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb930102b10 0x7fb93019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.500+0000 7fb936000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 0x7fb93019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.500+0000 7fb936000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9301046b0 0x7fb9301a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.500+0000 7fb936000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010e5e0 con 0x7fb9301046b0 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb936000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010e460 con 0x7fb930103d10 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb936000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010e760 con 0x7fb930102b10 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 0x7fb93019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 0x7fb93019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32986/0 (socket says 172.21.15.83:32986) 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 -- 172.21.15.83:0/3829302419 learned_addr learned my addr 172.21.15.83:0/3829302419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-13T05:09:09.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92f400700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb930102b10 0x7fb93019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92fe00700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9301046b0 0x7fb9301a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 -- 172.21.15.83:0/3829302419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb930102b10 msgr2=0x7fb93019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb930102b10 0x7fb93019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 -- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9301046b0 msgr2=0x7fb9301a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9301046b0 0x7fb9301a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:09.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.501+0000 7fb92ea00700 1 -- 172.21.15.83:0/3829302419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9301a2ac0 con 0x7fb930103d10 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.502+0000 7fb92ea00700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 0x7fb93019b3a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c920 tx=0x7fb91c00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.502+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c007b00 con 0x7fb930103d10 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.502+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb91c004ce0 con 0x7fb930103d10 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.502+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9301a2db0 con 0x7fb930103d10 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.502+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c005600 con 0x7fb930103d10 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.502+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9301a3270 con 0x7fb930103d10 2024-04-13T05:09:09.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.503+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9300697a0 con 0x7fb930103d10 2024-04-13T05:09:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.506+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb91c004750 con 0x7fb930103d10 2024-04-13T05:09:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.507+0000 7fb927e00700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9140a3d40 0x7fb9140a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.508+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fb91c0cf1a0 con 0x7fb930103d10 2024-04-13T05:09:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.508+0000 7fb92f400700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9140a3d40 0x7fb9140a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.509+0000 7fb92f400700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9140a3d40 0x7fb9140a6200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb920005fd0 tx=0x7fb920005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:09:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.511+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb91c014030 con 0x7fb930103d10 2024-04-13T05:09:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:09.748+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb930076400 con 0x7fb9140a3d40 2024-04-13T05:09:10.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:10 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi057 2024-04-13T05:09:10.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:10 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi057 to 8547M 2024-04-13T05:09:10.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:09:10.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:09:10.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:10.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:10 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi057 2024-04-13T05:09:10.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:10 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi057 to 8547M 2024-04-13T05:09:10.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:09:10.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:09:10.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:10.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:10 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi057 2024-04-13T05:09:10.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:10 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi057 to 8547M 2024-04-13T05:09:10.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:09:10.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:09:10.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:11.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:11 smithi189 ceph-mon[35531]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:11.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:11 smithi189 ceph-mon[35531]: from='client.34217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:09:11.432 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:11 smithi083 ceph-mon[35133]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:11.432 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:11 smithi083 ceph-mon[35133]: from='client.34217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:09:11.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:11 smithi057 ceph-mon[32331]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:11.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:11 smithi057 ceph-mon[32331]: from='client.34217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:09:13.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:13 smithi189 ceph-mon[35531]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:09:13.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:13 smithi083 ceph-mon[35133]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:09:13.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:13 smithi057 ceph-mon[32331]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:09:14.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:14 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/1964744047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]: dispatch 2024-04-13T05:09:14.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:14 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]: dispatch 2024-04-13T05:09:14.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:14 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]': finished 2024-04-13T05:09:14.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:14 smithi189 ceph-mon[35531]: osdmap e30: 5 total, 4 up, 5 in 2024-04-13T05:09:14.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:14.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:14 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/1964744047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]: dispatch 2024-04-13T05:09:14.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:14 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]: dispatch 2024-04-13T05:09:14.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:14 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]': finished 2024-04-13T05:09:14.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:14 smithi083 ceph-mon[35133]: osdmap e30: 5 total, 4 up, 5 in 2024-04-13T05:09:14.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:14.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:14 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/1964744047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]: dispatch 2024-04-13T05:09:14.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:14 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]: dispatch 2024-04-13T05:09:14.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:14 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03b6258a-179f-48c4-acce-0e2929065e44"}]': finished 2024-04-13T05:09:14.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:14 smithi057 ceph-mon[32331]: osdmap e30: 5 total, 4 up, 5 in 2024-04-13T05:09:14.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:15.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:15 smithi189 ceph-mon[35531]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-13T05:09:15.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:15 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/3931043981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:09:15.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:15 smithi083 ceph-mon[35133]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-13T05:09:15.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:15 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/3931043981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:09:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:15 smithi057 ceph-mon[32331]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-13T05:09:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:15 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/3931043981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:09:17.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:17 smithi083 ceph-mon[35133]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-13T05:09:17.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:17 smithi189 ceph-mon[35531]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-13T05:09:17.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:17 smithi057 ceph-mon[32331]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-13T05:09:19.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:19 smithi189 ceph-mon[35531]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:09:19.441 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:19 smithi083 ceph-mon[35133]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:09:19.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:19 smithi057 ceph-mon[32331]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:09:21.304 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:21 smithi083 ceph-mon[35133]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:09:21.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:21 smithi057 ceph-mon[32331]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:09:21.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:21 smithi189 ceph-mon[35531]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:09:23.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:23 smithi083 ceph-mon[35133]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:23.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-13T05:09:23.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:23.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:23 smithi057 ceph-mon[32331]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-13T05:09:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:23.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:23 smithi189 ceph-mon[35531]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:23.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-13T05:09:23.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:24.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:24 smithi083 ceph-mon[35133]: Deploying daemon osd.4 on smithi083 2024-04-13T05:09:24.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:24 smithi057 ceph-mon[32331]: Deploying daemon osd.4 on smithi083 2024-04-13T05:09:24.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:24 smithi189 ceph-mon[35531]: Deploying daemon osd.4 on smithi083 2024-04-13T05:09:25.363 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:25 smithi083 ceph-mon[35133]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:25.482 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:25 smithi189 ceph-mon[35531]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:25.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:25 smithi057 ceph-mon[32331]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:27.361 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:27 smithi083 ceph-mon[35133]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:27.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:27 smithi057 ceph-mon[32331]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:27.580 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:27 smithi189 ceph-mon[35531]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:29.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:29 smithi083 ceph-mon[35133]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:29.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:29 smithi057 ceph-mon[32331]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:29.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:29 smithi189 ceph-mon[35531]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:31.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:31 smithi083 ceph-mon[35133]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:31.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:31.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:31.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:31.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:31 smithi057 ceph-mon[32331]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:31.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:31.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:31.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:31 smithi189 ceph-mon[35531]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:33.260 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:33 smithi083 ceph-mon[35133]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:33.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:33 smithi057 ceph-mon[32331]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:33.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:33 smithi189 ceph-mon[35531]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:35.188 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:35 smithi083 ceph-mon[35133]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:35.188 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:35.188 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:35.188 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:35 smithi083 ceph-mon[35133]: from='osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-13T05:09:35.188 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:35 smithi083 ceph-mon[35133]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-13T05:09:35.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:35 smithi057 ceph-mon[32331]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:35.442 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:35.442 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:35.442 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:35 smithi057 ceph-mon[32331]: from='osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-13T05:09:35.442 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:35 smithi057 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-13T05:09:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:35 smithi189 ceph-mon[35531]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:35 smithi189 ceph-mon[35531]: from='osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-13T05:09:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:35 smithi189 ceph-mon[35531]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-13T05:09:35.777 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 4 on host 'smithi083' 2024-04-13T05:09:35.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.776+0000 7fb927e00700 1 -- 172.21.15.83:0/3829302419 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb930076400 con 0x7fb9140a3d40 2024-04-13T05:09:35.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9140a3d40 msgr2=0x7fb9140a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9140a3d40 0x7fb9140a6200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb920005fd0 tx=0x7fb920005dd0 comp rx=0 tx=0).stop 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 msgr2=0x7fb93019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 0x7fb93019b3a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c920 tx=0x7fb91c00cce0 comp rx=0 tx=0).stop 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 shutdown_connections 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb930102b10 0x7fb93019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9140a3d40 0x7fb9140a6200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9301046b0 0x7fb9301a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:35.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 --2- 172.21.15.83:0/3829302419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb930103d10 0x7fb93019b3a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:35.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 >> 172.21.15.83:0/3829302419 conn(0x7fb9300fe0b0 msgr2=0x7fb930074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:09:35.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 shutdown_connections 2024-04-13T05:09:35.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:09:35.780+0000 7fb936000700 1 -- 172.21.15.83:0/3829302419 wait complete. 2024-04-13T05:09:36.530 DEBUG:teuthology.orchestra.run.smithi083:osd.4> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.4.service 2024-04-13T05:09:36.534 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-04-13T05:09:36.535 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-13T05:09:36.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: from='osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: osdmap e31: 5 total, 4 up, 5 in 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:36.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:36 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: from='osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: osdmap e31: 5 total, 4 up, 5 in 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:36.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:36 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-13T05:09:36.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: from='osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:09:36.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: osdmap e31: 5 total, 4 up, 5 in 2024-04-13T05:09:36.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:36.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:09:36.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:36.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:36.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:36 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:36.980 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:09:37.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:37 smithi083 ceph-mon[35133]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:09:37.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:37 smithi083 ceph-mon[35133]: osdmap e32: 5 total, 4 up, 5 in 2024-04-13T05:09:37.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:37.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:37 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:37.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:37 smithi057 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:09:37.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:37 smithi057 ceph-mon[32331]: osdmap e32: 5 total, 4 up, 5 in 2024-04-13T05:09:37.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:37.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:37 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:37 smithi189 ceph-mon[35531]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:09:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:37 smithi189 ceph-mon[35531]: osdmap e32: 5 total, 4 up, 5 in 2024-04-13T05:09:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:37.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:37 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:09:38.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:09:38.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] boot 2024-04-13T05:09:38.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: osdmap e33: 5 total, 5 up, 5 in 2024-04-13T05:09:38.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:38 smithi083 ceph-mon[35133]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] boot 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: osdmap e33: 5 total, 5 up, 5 in 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.753 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:38 smithi189 ceph-mon[35531]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: osd.4 [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] boot 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: osdmap e33: 5 total, 5 up, 5 in 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-13T05:09:38.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:38 smithi057 ceph-mon[32331]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-13T05:09:39.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:39 smithi057 ceph-mon[32331]: osdmap e34: 5 total, 5 up, 5 in 2024-04-13T05:09:39.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:39 smithi189 ceph-mon[35531]: osdmap e34: 5 total, 5 up, 5 in 2024-04-13T05:09:39.969 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-13T05:09:39.970 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-13T05:09:39.970 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-13T05:09:39.970 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-13T05:09:39.970 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0333598 s, 314 MB/s 2024-04-13T05:09:39.970 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-13T05:09:39.999 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:39 smithi083 ceph-mon[35133]: osdmap e34: 5 total, 5 up, 5 in 2024-04-13T05:09:40.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:40 smithi189 ceph-mon[35531]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:40.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:40 smithi083 ceph-mon[35133]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:40.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:40 smithi057 ceph-mon[32331]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:40.828 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-04-13T05:09:41.270 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi083 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:42.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:42.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:42.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:42.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi083 2024-04-13T05:09:42.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:42.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi083 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:42.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:09:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:09:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:09:43.140 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:42 smithi189 ceph-mon[35531]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:42 smithi083 ceph-mon[35133]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:43.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:42 smithi057 ceph-mon[32331]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 -- 172.21.15.189:0/977534397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 msgr2=0x7fb468109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 --2- 172.21.15.189:0/977534397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb468109570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009990 tx=0x7fb45c009ca0 comp rx=0 tx=0).stop 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 -- 172.21.15.189:0/977534397 shutdown_connections 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 --2- 172.21.15.189:0/977534397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb468109570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 --2- 172.21.15.189:0/977534397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 0x7fb4681115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 --2- 172.21.15.189:0/977534397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46810a350 0x7fb46810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 -- 172.21.15.189:0/977534397 >> 172.21.15.189:0/977534397 conn(0x7fb468078120 msgr2=0x7fb46807a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 -- 172.21.15.189:0/977534397 shutdown_connections 2024-04-13T05:09:43.309 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.308+0000 7fb46fc00700 1 -- 172.21.15.189:0/977534397 wait complete. 2024-04-13T05:09:43.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.309+0000 7fb46fc00700 1 Processor -- start 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.309+0000 7fb46fc00700 1 -- start start 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.309+0000 7fb46fc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb46819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.309+0000 7fb46fc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46810a350 0x7fb46819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46fc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 0x7fb4681a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46fc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb468114c30 con 0x7fb46810acf0 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46fc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb468114ab0 con 0x7fb46810a350 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46fc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb468114db0 con 0x7fb468109150 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb46819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb46819f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:41854/0 (socket says 172.21.15.189:41854) 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 -- 172.21.15.189:0/2580125639 learned_addr learned my addr 172.21.15.189:0/2580125639 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb467e00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46810a350 0x7fb46819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:43.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46dc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 0x7fb4681a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 -- 172.21.15.189:0/2580125639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46810a350 msgr2=0x7fb46819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46810a350 0x7fb46819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 -- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 msgr2=0x7fb4681a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 0x7fb4681a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46d200700 1 -- 172.21.15.189:0/2580125639 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681a6f30 con 0x7fb468109150 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.310+0000 7fb46dc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 0x7fb4681a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.311+0000 7fb46d200700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb46819f2d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0037f0 tx=0x7fb45c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.311+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003c90 con 0x7fb468109150 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.311+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45c004830 con 0x7fb468109150 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.311+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681a71c0 con 0x7fb468109150 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.311+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0054e0 con 0x7fb468109150 2024-04-13T05:09:43.313 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.311+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681a7610 con 0x7fb468109150 2024-04-13T05:09:43.314 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.312+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4680697a0 con 0x7fb468109150 2024-04-13T05:09:43.315 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.314+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb45c0049e0 con 0x7fb468109150 2024-04-13T05:09:43.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.316+0000 7fb465600700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4500a3cc0 0x7fb4500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:09:43.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.316+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fb45c0d08b0 con 0x7fb468109150 2024-04-13T05:09:43.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.317+0000 7fb467e00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4500a3cc0 0x7fb4500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:09:43.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.318+0000 7fb467e00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4500a3cc0 0x7fb4500a6180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb454005950 tx=0x7fb4540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:09:43.320 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.319+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb45c013070 con 0x7fb468109150 2024-04-13T05:09:43.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:09:43.554+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb468062ef0 con 0x7fb4500a3cc0 2024-04-13T05:09:44.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:09:44.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:09:44.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:44.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:09:44.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:09:44.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:44.330 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:09:44.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:09:44.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:45.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:44 smithi083 ceph-mon[35133]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:45.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:44 smithi083 ceph-mon[35133]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:09:45.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:44 smithi057 ceph-mon[32331]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:45.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:44 smithi057 ceph-mon[32331]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:09:45.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:44 smithi189 ceph-mon[35531]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:45.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:44 smithi189 ceph-mon[35531]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:09:47.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:46 smithi083 ceph-mon[35133]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:47.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:46 smithi057 ceph-mon[32331]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:46 smithi189 ceph-mon[35531]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:48.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:47 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1638746964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]: dispatch 2024-04-13T05:09:48.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:47 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]: dispatch 2024-04-13T05:09:48.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:47 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]': finished 2024-04-13T05:09:48.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:47 smithi083 ceph-mon[35133]: osdmap e35: 6 total, 5 up, 6 in 2024-04-13T05:09:48.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:47 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:09:48.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:47 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1638746964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]: dispatch 2024-04-13T05:09:48.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:47 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]: dispatch 2024-04-13T05:09:48.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:47 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]': finished 2024-04-13T05:09:48.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:47 smithi057 ceph-mon[32331]: osdmap e35: 6 total, 5 up, 6 in 2024-04-13T05:09:48.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:47 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:09:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:47 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1638746964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]: dispatch 2024-04-13T05:09:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:47 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]: dispatch 2024-04-13T05:09:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:47 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad5b3787-2395-43e1-9595-98db29fc5e47"}]': finished 2024-04-13T05:09:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:47 smithi189 ceph-mon[35531]: osdmap e35: 6 total, 5 up, 6 in 2024-04-13T05:09:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:47 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:09:49.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:48 smithi083 ceph-mon[35133]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:49.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:48 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/2483385947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:09:49.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:48 smithi057 ceph-mon[32331]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:49.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:48 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/2483385947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:09:49.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:48 smithi189 ceph-mon[35531]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:49.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:48 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/2483385947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:09:51.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:50 smithi083 ceph-mon[35133]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:51.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:50 smithi057 ceph-mon[32331]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:51.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:50 smithi189 ceph-mon[35531]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:53.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:52 smithi083 ceph-mon[35133]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:53.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:52 smithi057 ceph-mon[32331]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:53.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:52 smithi189 ceph-mon[35531]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:55.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:54 smithi189 ceph-mon[35531]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:55.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:54 smithi083 ceph-mon[35133]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:55.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:54 smithi057 ceph-mon[32331]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:57.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:56 smithi083 ceph-mon[35133]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:57.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:56 smithi057 ceph-mon[32331]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:57.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:56 smithi189 ceph-mon[35531]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:58.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-13T05:09:58.225 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:58.225 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:58 smithi189 ceph-mon[35531]: Deploying daemon osd.5 on smithi189 2024-04-13T05:09:58.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-13T05:09:58.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:58.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:57 smithi057 ceph-mon[32331]: Deploying daemon osd.5 on smithi189 2024-04-13T05:09:58.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-13T05:09:58.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:09:58.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:58 smithi083 ceph-mon[35133]: Deploying daemon osd.5 on smithi189 2024-04-13T05:09:59.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:09:59 smithi057 ceph-mon[32331]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:59.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:09:59 smithi189 ceph-mon[35531]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:09:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:09:59 smithi083 ceph-mon[35133]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:00.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:00 smithi057 ceph-mon[32331]: overall HEALTH_OK 2024-04-13T05:10:00.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:00 smithi189 ceph-mon[35531]: overall HEALTH_OK 2024-04-13T05:10:00.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:00 smithi083 ceph-mon[35133]: overall HEALTH_OK 2024-04-13T05:10:01.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:01 smithi057 ceph-mon[32331]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:01.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:01 smithi189 ceph-mon[35531]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:01.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:01 smithi083 ceph-mon[35133]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:03.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:03 smithi057 ceph-mon[32331]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:03.302 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:03 smithi083 ceph-mon[35133]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:03.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:03 smithi189 ceph-mon[35531]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:05.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:05 smithi083 ceph-mon[35133]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:05.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:05 smithi189 ceph-mon[35531]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:05 smithi057 ceph-mon[32331]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:06.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:06.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:06.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:06.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:06.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:06.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:06.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:06.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:06.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:07.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:07 smithi189 ceph-mon[35531]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:07.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:07 smithi083 ceph-mon[35133]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:07.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:07 smithi057 ceph-mon[32331]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:09 smithi189 ceph-mon[35531]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:09.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:09 smithi083 ceph-mon[35133]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:09.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:09.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:09.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:09 smithi057 ceph-mon[32331]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:09.969 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.968+0000 7fb465600700 1 -- 172.21.15.189:0/2580125639 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb468062ef0 con 0x7fb4500a3cc0 2024-04-13T05:10:09.970 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-04-13T05:10:09.972 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.971+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4500a3cc0 msgr2=0x7fb4500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:09.972 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.971+0000 7fb46fc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4500a3cc0 0x7fb4500a6180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb454005950 tx=0x7fb4540058e0 comp rx=0 tx=0).stop 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.971+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 msgr2=0x7fb46819f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.971+0000 7fb46fc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb46819f2d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0037f0 tx=0x7fb45c0047b0 comp rx=0 tx=0).stop 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 shutdown_connections 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb468109150 0x7fb46819f2d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4500a3cc0 0x7fb4500a6180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb46810acf0 0x7fb4681a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 --2- 172.21.15.189:0/2580125639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb46810a350 0x7fb46819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 >> 172.21.15.189:0/2580125639 conn(0x7fb468078120 msgr2=0x7fb46810d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 shutdown_connections 2024-04-13T05:10:09.973 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:10:09.972+0000 7fb46fc00700 1 -- 172.21.15.189:0/2580125639 wait complete. 2024-04-13T05:10:10.346 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:10 smithi189 ceph-mon[35531]: from='osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-13T05:10:10.346 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:10 smithi189 ceph-mon[35531]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-13T05:10:10.346 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:10.346 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:10.346 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:10.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:10 smithi083 ceph-mon[35133]: from='osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-13T05:10:10.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:10 smithi083 ceph-mon[35133]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-13T05:10:10.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:10.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:10.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:10 smithi057 ceph-mon[32331]: from='osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-13T05:10:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:10 smithi057 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-13T05:10:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:10.766 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.5.service 2024-04-13T05:10:10.772 INFO:tasks.cephadm:Deploying osd.6 on smithi057 with /dev/vg_nvme/lv_2... 2024-04-13T05:10:10.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-13T05:10:11.247 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:10:11.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:11 smithi057 ceph-mon[32331]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:11.280 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:11 smithi057 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-13T05:10:11.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:11 smithi057 ceph-mon[32331]: from='osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:10:11.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:11 smithi057 ceph-mon[32331]: osdmap e36: 6 total, 5 up, 6 in 2024-04-13T05:10:11.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:11.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:11 smithi057 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:10:11.313 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-mon[35531]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:11.313 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-mon[35531]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-13T05:10:11.313 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-mon[35531]: from='osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:10:11.313 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-mon[35531]: osdmap e36: 6 total, 5 up, 6 in 2024-04-13T05:10:11.313 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:11.314 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-mon[35531]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:10:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:11 smithi083 ceph-mon[35133]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:11 smithi083 ceph-mon[35133]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-13T05:10:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:11 smithi083 ceph-mon[35133]: from='osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:10:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:11 smithi083 ceph-mon[35133]: osdmap e36: 6 total, 5 up, 6 in 2024-04-13T05:10:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:11 smithi083 ceph-mon[35133]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:10:12.241 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:10:11.959+0000 7f9a9b800700 -1 osd.5 0 waiting for initial osdmap 2024-04-13T05:10:12.241 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:10:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:10:11.983+0000 7f9a91600700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: osdmap e37: 6 total, 5 up, 6 in 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:12.242 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:12 smithi189 ceph-mon[35531]: from='osd.5 ' entity='osd.5' 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: osdmap e37: 6 total, 5 up, 6 in 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:12 smithi057 ceph-mon[32331]: from='osd.5 ' entity='osd.5' 2024-04-13T05:10:12.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:10:12.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:10:12.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:10:12.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: osdmap e37: 6 total, 5 up, 6 in 2024-04-13T05:10:12.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:12.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:12.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:12 smithi083 ceph-mon[35133]: from='osd.5 ' entity='osd.5' 2024-04-13T05:10:13.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:13 smithi057 ceph-mon[32331]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:13.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:13.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:13 smithi057 ceph-mon[32331]: osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] boot 2024-04-13T05:10:13.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:13 smithi057 ceph-mon[32331]: osdmap e38: 6 total, 6 up, 6 in 2024-04-13T05:10:13.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:13.316 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:13 smithi057 ceph-mon[32331]: osdmap e39: 6 total, 6 up, 6 in 2024-04-13T05:10:13.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:13 smithi083 ceph-mon[35133]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:13 smithi083 ceph-mon[35133]: osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] boot 2024-04-13T05:10:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:13 smithi083 ceph-mon[35133]: osdmap e38: 6 total, 6 up, 6 in 2024-04-13T05:10:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:13 smithi083 ceph-mon[35133]: osdmap e39: 6 total, 6 up, 6 in 2024-04-13T05:10:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:13 smithi189 ceph-mon[35531]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-13T05:10:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:13 smithi189 ceph-mon[35531]: osd.5 [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] boot 2024-04-13T05:10:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:13 smithi189 ceph-mon[35531]: osdmap e38: 6 total, 6 up, 6 in 2024-04-13T05:10:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-13T05:10:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:13 smithi189 ceph-mon[35531]: osdmap e39: 6 total, 6 up, 6 in 2024-04-13T05:10:14.404 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-13T05:10:14.404 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-13T05:10:14.404 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-04-13T05:10:14.404 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-04-13T05:10:14.404 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0331177 s, 317 MB/s 2024-04-13T05:10:14.404 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-04-13T05:10:15.224 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:15 smithi189 ceph-mon[35531]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:15.224 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:15 smithi189 ceph-mon[35531]: osdmap e40: 6 total, 6 up, 6 in 2024-04-13T05:10:15.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:15 smithi083 ceph-mon[35133]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:15.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:15 smithi083 ceph-mon[35133]: osdmap e40: 6 total, 6 up, 6 in 2024-04-13T05:10:15.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:15 smithi057 ceph-mon[32331]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:15.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:15 smithi057 ceph-mon[32331]: osdmap e40: 6 total, 6 up, 6 in 2024-04-13T05:10:15.379 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2024-04-13T05:10:15.816 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:10:16.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: osdmap e41: 6 total, 6 up, 6 in 2024-04-13T05:10:16.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:16.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: osdmap e41: 6 total, 6 up, 6 in 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:16.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: osdmap e41: 6 total, 6 up, 6 in 2024-04-13T05:10:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:17.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:17 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi189 2024-04-13T05:10:17.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:17 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi189 to 8544M 2024-04-13T05:10:17.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:17 smithi083 ceph-mon[35133]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-13T05:10:17.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:17 smithi083 ceph-mon[35133]: osdmap e42: 6 total, 6 up, 6 in 2024-04-13T05:10:17.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:17 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi189 2024-04-13T05:10:17.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:17 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi189 to 8544M 2024-04-13T05:10:17.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:17 smithi057 ceph-mon[32331]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-13T05:10:17.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:17 smithi057 ceph-mon[32331]: osdmap e42: 6 total, 6 up, 6 in 2024-04-13T05:10:17.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:17 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi189 2024-04-13T05:10:17.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:17 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi189 to 8544M 2024-04-13T05:10:17.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:17 smithi189 ceph-mon[35531]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-13T05:10:17.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:17 smithi189 ceph-mon[35531]: osdmap e42: 6 total, 6 up, 6 in 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 -- 172.21.15.57:0/868786244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 msgr2=0x7f097c074f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 --2- 172.21.15.57:0/868786244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c074f40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0970009990 tx=0x7f0970009ca0 comp rx=0 tx=0).stop 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 -- 172.21.15.57:0/868786244 shutdown_connections 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 --2- 172.21.15.57:0/868786244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 0x7f097c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 --2- 172.21.15.57:0/868786244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c074f40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 --2- 172.21.15.57:0/868786244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 0x7f097c075990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:18.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 -- 172.21.15.57:0/868786244 >> 172.21.15.57:0/868786244 conn(0x7f097c0fdfe0 msgr2=0x7f097c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:10:18.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 -- 172.21.15.57:0/868786244 shutdown_connections 2024-04-13T05:10:18.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.076+0000 7f0981e00700 1 -- 172.21.15.57:0/868786244 wait complete. 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.077+0000 7f0981e00700 1 Processor -- start 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.077+0000 7f0981e00700 1 -- start start 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0981e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 0x7f097c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0981e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0981e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 0x7f097c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0981e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1106b0 con 0x7f097c076ae0 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0981e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f097c110530 con 0x7f097c10a910 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0981e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f097c110830 con 0x7f097c075510 2024-04-13T05:10:18.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f097be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:18.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f097be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38764/0 (socket says 172.21.15.57:38764) 2024-04-13T05:10:18.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f097be00700 1 -- 172.21.15.57:0/111755703 learned_addr learned my addr 172.21.15.57:0/111755703 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:10:18.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.078+0000 7f0980a00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 0x7f097c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:18.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f0981400700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 0x7f097c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f097be00700 1 -- 172.21.15.57:0/111755703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 msgr2=0x7f097c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f097be00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 0x7f097c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f097be00700 1 -- 172.21.15.57:0/111755703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 msgr2=0x7f097c1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f097be00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 0x7f097c1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f097be00700 1 -- 172.21.15.57:0/111755703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c1a2970 con 0x7f097c076ae0 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f0981400700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 0x7f097c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f0980a00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 0x7f097c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.079+0000 7f097be00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c19b260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f096800d980 tx=0x7f096800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:10:18.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.080+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968017030 con 0x7f097c076ae0 2024-04-13T05:10:18.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.080+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09680044d0 con 0x7f097c076ae0 2024-04-13T05:10:18.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.080+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968003aa0 con 0x7f097c076ae0 2024-04-13T05:10:18.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.080+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c1a2c60 con 0x7f097c076ae0 2024-04-13T05:10:18.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.080+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c1a3150 con 0x7f097c076ae0 2024-04-13T05:10:18.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.081+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c0697a0 con 0x7f097c076ae0 2024-04-13T05:10:18.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.084+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f096801a070 con 0x7f097c076ae0 2024-04-13T05:10:18.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.087+0000 7f0979600700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09640a3cc0 0x7f09640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.087+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f09680cf320 con 0x7f097c076ae0 2024-04-13T05:10:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.087+0000 7f0980a00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09640a3cc0 0x7f09640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:18.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.088+0000 7f0980a00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09640a3cc0 0x7f09640a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0970003780 tx=0x7f0970003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:10:18.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.092+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f096809cba0 con 0x7f097c076ae0 2024-04-13T05:10:18.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:18.336+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f097c062ef0 con 0x7f09640a3cc0 2024-04-13T05:10:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:19 smithi083 ceph-mon[35133]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-13T05:10:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:10:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:10:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:19 smithi057 ceph-mon[32331]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-13T05:10:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:10:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:10:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:19.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:19 smithi189 ceph-mon[35531]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-13T05:10:19.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:10:19.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:10:19.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:20.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:20 smithi083 ceph-mon[35133]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:10:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:20 smithi057 ceph-mon[32331]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:10:20.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:20 smithi189 ceph-mon[35531]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:10:21.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:21 smithi083 ceph-mon[35133]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-13T05:10:21.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:21 smithi057 ceph-mon[32331]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-13T05:10:21.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:21 smithi189 ceph-mon[35531]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-13T05:10:23.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:23 smithi189 ceph-mon[35531]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-13T05:10:23.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:23 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1735417477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e957fe-81a0-4abe-828b-8c1cb7b14983"}]: dispatch 2024-04-13T05:10:23.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:23 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1735417477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e957fe-81a0-4abe-828b-8c1cb7b14983"}]': finished 2024-04-13T05:10:23.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:23 smithi189 ceph-mon[35531]: osdmap e43: 7 total, 6 up, 7 in 2024-04-13T05:10:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:23 smithi083 ceph-mon[35133]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-13T05:10:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:23 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1735417477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e957fe-81a0-4abe-828b-8c1cb7b14983"}]: dispatch 2024-04-13T05:10:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:23 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1735417477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e957fe-81a0-4abe-828b-8c1cb7b14983"}]': finished 2024-04-13T05:10:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:23 smithi083 ceph-mon[35133]: osdmap e43: 7 total, 6 up, 7 in 2024-04-13T05:10:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:23 smithi057 ceph-mon[32331]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-13T05:10:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:23 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1735417477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e957fe-81a0-4abe-828b-8c1cb7b14983"}]: dispatch 2024-04-13T05:10:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:23 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1735417477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e957fe-81a0-4abe-828b-8c1cb7b14983"}]': finished 2024-04-13T05:10:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:23 smithi057 ceph-mon[32331]: osdmap e43: 7 total, 6 up, 7 in 2024-04-13T05:10:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:24.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:24 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3567814237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:10:24.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:24 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3567814237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:10:24.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:24 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3567814237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:10:25.483 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:25 smithi189 ceph-mon[35531]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:25.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:25 smithi083 ceph-mon[35133]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:25.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:25 smithi057 ceph-mon[32331]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:27.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:27 smithi189 ceph-mon[35531]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:27.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:27 smithi083 ceph-mon[35133]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:27.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:27 smithi057 ceph-mon[32331]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:29.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:29 smithi083 ceph-mon[35133]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:29.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:29 smithi057 ceph-mon[32331]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:29.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:29 smithi189 ceph-mon[35531]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:31.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:31 smithi083 ceph-mon[35133]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:31.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:31 smithi057 ceph-mon[32331]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:31 smithi189 ceph-mon[35531]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:32.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-13T05:10:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:32.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-13T05:10:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-13T05:10:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:33.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:33 smithi057 ceph-mon[32331]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:33.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:33 smithi057 ceph-mon[32331]: Deploying daemon osd.6 on smithi057 2024-04-13T05:10:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:33 smithi083 ceph-mon[35133]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:33 smithi083 ceph-mon[35133]: Deploying daemon osd.6 on smithi057 2024-04-13T05:10:33.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:33 smithi189 ceph-mon[35531]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:33.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:33 smithi189 ceph-mon[35531]: Deploying daemon osd.6 on smithi057 2024-04-13T05:10:35.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:35 smithi083 ceph-mon[35133]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:35.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:35 smithi057 ceph-mon[32331]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:35 smithi189 ceph-mon[35531]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:37.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:37 smithi083 ceph-mon[35133]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:37.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:37 smithi057 ceph-mon[32331]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:37.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:37 smithi189 ceph-mon[35531]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:39.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:39 smithi057 ceph-mon[32331]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:39.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:39 smithi189 ceph-mon[35531]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:39.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:39 smithi083 ceph-mon[35133]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:40.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:40 smithi189 ceph-mon[35531]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:40.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:40 smithi057 ceph-mon[32331]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:40.749 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:40 smithi083 ceph-mon[35133]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:42.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:42.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:42.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:42.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:42.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:42.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:42.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:42.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:42.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:43.249 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:42 smithi083 ceph-mon[35133]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:43.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:42 smithi057 ceph-mon[32331]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:43.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:42 smithi189 ceph-mon[35531]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:45.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:45.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.203 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:45.203 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:45.203 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.203 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:44 smithi057 ceph-mon[32331]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:44 smithi083 ceph-mon[35133]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-13T05:10:45.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:45.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:45.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:45.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:45.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:44 smithi189 ceph-mon[35531]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-13T05:10:46.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.238+0000 7f0979600700 1 -- 172.21.15.57:0/111755703 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f097c062ef0 con 0x7f09640a3cc0 2024-04-13T05:10:46.239 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 6 on host 'smithi057' 2024-04-13T05:10:46.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.242+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09640a3cc0 msgr2=0x7f09640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.242+0000 7f0981e00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09640a3cc0 0x7f09640a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0970003780 tx=0x7f0970003680 comp rx=0 tx=0).stop 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.242+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 msgr2=0x7f097c19b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.242+0000 7f0981e00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c19b260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f096800d980 tx=0x7f096800dd40 comp rx=0 tx=0).stop 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 shutdown_connections 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f097c075510 0x7f097c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09640a3cc0 0x7f09640a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f097c10a910 0x7f097c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 --2- 172.21.15.57:0/111755703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f097c076ae0 0x7f097c19b260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 >> 172.21.15.57:0/111755703 conn(0x7f097c0fdfe0 msgr2=0x7f097c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 shutdown_connections 2024-04-13T05:10:46.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:10:46.243+0000 7f0981e00700 1 -- 172.21.15.57:0/111755703 wait complete. 2024-04-13T05:10:47.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:47.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-13T05:10:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: osdmap e44: 7 total, 6 up, 7 in 2024-04-13T05:10:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:10:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:47.118 DEBUG:teuthology.orchestra.run.smithi057:osd.6> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.6.service 2024-04-13T05:10:47.122 INFO:tasks.cephadm:Deploying osd.7 on smithi083 with /dev/vg_nvme/lv_2... 2024-04-13T05:10:47.123 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: osdmap e44: 7 total, 6 up, 7 in 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:47.210 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:47.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: osdmap e44: 7 total, 6 up, 7 in 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:47.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:47.558 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:10:47.977 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:47 smithi083 ceph-mon[35133]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:10:47.977 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:47 smithi083 ceph-mon[35133]: osdmap e45: 7 total, 6 up, 7 in 2024-04-13T05:10:47.977 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:47 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:47.977 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:47 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:48.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:47 smithi057 ceph-mon[32331]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:10:48.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:47 smithi057 ceph-mon[32331]: osdmap e45: 7 total, 6 up, 7 in 2024-04-13T05:10:48.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:47 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:48.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:47 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:47 smithi189 ceph-mon[35531]: from='osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:10:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:47 smithi189 ceph-mon[35531]: osdmap e45: 7 total, 6 up, 7 in 2024-04-13T05:10:48.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:47 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:48.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:47 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] boot 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: osdmap e46: 7 total, 7 up, 7 in 2024-04-13T05:10:49.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] boot 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: osdmap e46: 7 total, 7 up, 7 in 2024-04-13T05:10:49.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:49.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:10:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:10:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-13T05:10:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: osd.6 [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] boot 2024-04-13T05:10:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: osdmap e46: 7 total, 7 up, 7 in 2024-04-13T05:10:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-13T05:10:50.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:49 smithi083 ceph-mon[35133]: osdmap e47: 7 total, 7 up, 7 in 2024-04-13T05:10:50.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:49 smithi057 ceph-mon[32331]: osdmap e47: 7 total, 7 up, 7 in 2024-04-13T05:10:50.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:49 smithi189 ceph-mon[35531]: osdmap e47: 7 total, 7 up, 7 in 2024-04-13T05:10:50.428 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-13T05:10:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-13T05:10:50.429 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-13T05:10:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-13T05:10:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0392294 s, 267 MB/s 2024-04-13T05:10:50.429 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-13T05:10:51.097 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:50 smithi083 ceph-mon[35133]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:51.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:50 smithi057 ceph-mon[32331]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:51.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:50 smithi189 ceph-mon[35531]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:51.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-04-13T05:10:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:10:53.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:53.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi057 2024-04-13T05:10:53.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi057 to 5698M 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:53 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi057 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi057 to 5698M 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:53.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:53 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi057 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi057 to 5698M 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:10:53.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:53 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 -- 172.21.15.83:0/9926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 msgr2=0x7f7a70105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 --2- 172.21.15.83:0/9926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a70105170 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a600099c0 tx=0x7f7a60009cd0 comp rx=0 tx=0).stop 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 -- 172.21.15.83:0/9926178 shutdown_connections 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 --2- 172.21.15.83:0/9926178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701068f0 0x7f7a7010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 --2- 172.21.15.83:0/9926178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a70105170 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 --2- 172.21.15.83:0/9926178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a70105f50 0x7f7a701063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 -- 172.21.15.83:0/9926178 >> 172.21.15.83:0/9926178 conn(0x7f7a70100310 msgr2=0x7f7a70102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.800+0000 7f7a75800700 1 -- 172.21.15.83:0/9926178 shutdown_connections 2024-04-13T05:10:53.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.801+0000 7f7a75800700 1 -- 172.21.15.83:0/9926178 wait complete. 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 Processor -- start 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 -- start start 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a7019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a70105f50 0x7f7a7019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a701068f0 0x7f7a701a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70110890 con 0x7f7a701068f0 2024-04-13T05:10:53.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70110710 con 0x7f7a70104d50 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a75800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70110a10 con 0x7f7a70105f50 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.802+0000 7f7a6ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a7019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a7019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60330/0 (socket says 172.21.15.83:60330) 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 -- 172.21.15.83:0/794143815 learned_addr learned my addr 172.21.15.83:0/794143815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6f400700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a701068f0 0x7f7a701a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6e000700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a70105f50 0x7f7a7019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 -- 172.21.15.83:0/794143815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a70105f50 msgr2=0x7f7a7019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a70105f50 0x7f7a7019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 -- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a701068f0 msgr2=0x7f7a701a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a701068f0 0x7f7a701a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 -- 172.21.15.83:0/794143815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a701a2b60 con 0x7f7a70104d50 2024-04-13T05:10:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6f400700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a701068f0 0x7f7a701a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:10:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.803+0000 7f7a6ea00700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a7019af00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7a600057c0 tx=0x7f7a60009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:10:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.804+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60026070 con 0x7f7a70104d50 2024-04-13T05:10:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.804+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a60004210 con 0x7f7a70104d50 2024-04-13T05:10:53.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.804+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a701a2df0 con 0x7f7a70104d50 2024-04-13T05:10:53.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.804+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60021410 con 0x7f7a70104d50 2024-04-13T05:10:53.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.804+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a3210 con 0x7f7a70104d50 2024-04-13T05:10:53.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.805+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a700697a0 con 0x7f7a70104d50 2024-04-13T05:10:53.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.807+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7a60029070 con 0x7f7a70104d50 2024-04-13T05:10:53.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.808+0000 7f7a67400700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a540a3c40 0x7f7a540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:10:53.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.809+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f7a600d09e0 con 0x7f7a70104d50 2024-04-13T05:10:53.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.809+0000 7f7a6e000700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a540a3c40 0x7f7a540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:10:53.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.810+0000 7f7a6e000700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a540a3c40 0x7f7a540a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005fd0 tx=0x7f7a5c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:10:53.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:53.812+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a60030080 con 0x7f7a70104d50 2024-04-13T05:10:54.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:10:54.048+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7a70062ef0 con 0x7f7a540a3c40 2024-04-13T05:10:54.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:10:54.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:10:54.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:54.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:10:54.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:10:54.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:54.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:10:54.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:10:54.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:10:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:55 smithi083 ceph-mon[35133]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:55 smithi083 ceph-mon[35133]: from='client.34277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:10:55.483 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:55 smithi189 ceph-mon[35531]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:55.483 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:55 smithi189 ceph-mon[35531]: from='client.34277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:10:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:55 smithi057 ceph-mon[32331]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:55 smithi057 ceph-mon[32331]: from='client.34277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:10:57.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:57 smithi083 ceph-mon[35133]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:57.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:57 smithi057 ceph-mon[32331]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:57.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:57 smithi189 ceph-mon[35531]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:58.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/1297340278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]: dispatch 2024-04-13T05:10:58.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:58 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]: dispatch 2024-04-13T05:10:58.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:58 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]': finished 2024-04-13T05:10:58.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:58 smithi083 ceph-mon[35133]: osdmap e48: 8 total, 7 up, 8 in 2024-04-13T05:10:58.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:10:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/1297340278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]: dispatch 2024-04-13T05:10:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:58 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]: dispatch 2024-04-13T05:10:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:58 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]': finished 2024-04-13T05:10:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:58 smithi057 ceph-mon[32331]: osdmap e48: 8 total, 7 up, 8 in 2024-04-13T05:10:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:10:58.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/1297340278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]: dispatch 2024-04-13T05:10:58.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:58 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]: dispatch 2024-04-13T05:10:58.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:58 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b5cd3c-1c3d-4896-949e-5f77372aac7b"}]': finished 2024-04-13T05:10:58.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:58 smithi189 ceph-mon[35531]: osdmap e48: 8 total, 7 up, 8 in 2024-04-13T05:10:58.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:10:59.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:59 smithi083 ceph-mon[35133]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:59.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:10:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/2724609359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:10:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:59 smithi057 ceph-mon[32331]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:10:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/2724609359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:10:59.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:59 smithi189 ceph-mon[35531]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:10:59.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:10:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/2724609359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:11:01.483 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:01 smithi083 ceph-mon[35133]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:01.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:01 smithi057 ceph-mon[32331]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:01 smithi189 ceph-mon[35531]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:03.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:03 smithi083 ceph-mon[35133]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:03.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:03 smithi057 ceph-mon[32331]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:03.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:03 smithi189 ceph-mon[35531]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:05.498 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:05 smithi083 ceph-mon[35133]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:05 smithi057 ceph-mon[32331]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:05.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:05 smithi189 ceph-mon[35531]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:07.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:07 smithi083 ceph-mon[35133]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:07.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-13T05:11:07.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:07.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:07 smithi057 ceph-mon[32331]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:07.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-13T05:11:07.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:07.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:07 smithi189 ceph-mon[35531]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:07.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-13T05:11:07.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:08.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:08 smithi083 ceph-mon[35133]: Deploying daemon osd.7 on smithi083 2024-04-13T05:11:08.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:08 smithi057 ceph-mon[32331]: Deploying daemon osd.7 on smithi083 2024-04-13T05:11:08.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:08 smithi189 ceph-mon[35531]: Deploying daemon osd.7 on smithi083 2024-04-13T05:11:09.469 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:09 smithi083 ceph-mon[35133]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:09.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:09 smithi057 ceph-mon[32331]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:09.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:09 smithi189 ceph-mon[35531]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:10.483 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:10 smithi083 ceph-mon[35133]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:10 smithi057 ceph-mon[32331]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:10.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:10 smithi189 ceph-mon[35531]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:12.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:12 smithi189 ceph-mon[35531]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:13.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:12 smithi083 ceph-mon[35133]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:13.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:12 smithi057 ceph-mon[32331]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:14.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:14 smithi189 ceph-mon[35531]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:14.833 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:14 smithi083 ceph-mon[35133]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:15.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:14 smithi057 ceph-mon[32331]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:16.462 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:16.462 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:16.462 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:16.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:17.450 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:17 smithi083 ceph-mon[35133]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:17.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:17 smithi057 ceph-mon[32331]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:17.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:17 smithi189 ceph-mon[35531]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-13T05:11:19.478 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:19 smithi083 ceph-mon[35133]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-13T05:11:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-13T05:11:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:19 smithi057 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-13T05:11:19.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:19.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-13T05:11:19.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:19 smithi189 ceph-mon[35531]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-13T05:11:19.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.615+0000 7f7a67400700 1 -- 172.21.15.83:0/794143815 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7a70062ef0 con 0x7f7a540a3c40 2024-04-13T05:11:19.617 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 7 on host 'smithi083' 2024-04-13T05:11:19.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a540a3c40 msgr2=0x7f7a540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:19.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a540a3c40 0x7f7a540a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005fd0 tx=0x7f7a5c00a560 comp rx=0 tx=0).stop 2024-04-13T05:11:19.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 msgr2=0x7f7a7019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a7019af00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7a600057c0 tx=0x7f7a60009f80 comp rx=0 tx=0).stop 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 shutdown_connections 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a540a3c40 0x7f7a540a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a701068f0 0x7f7a701a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70104d50 0x7f7a7019af00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 --2- 172.21.15.83:0/794143815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a70105f50 0x7f7a7019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 >> 172.21.15.83:0/794143815 conn(0x7f7a70100310 msgr2=0x7f7a70109370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 shutdown_connections 2024-04-13T05:11:19.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:11:19.620+0000 7f7a75800700 1 -- 172.21.15.83:0/794143815 wait complete. 2024-04-13T05:11:20.476 DEBUG:teuthology.orchestra.run.smithi083:osd.7> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.7.service 2024-04-13T05:11:20.480 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-04-13T05:11:20.480 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: osdmap e49: 8 total, 7 up, 8 in 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:20.722 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:11:20.723 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:20 smithi083 ceph-mon[35133]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:11:20.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:20.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:20.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:20.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:20.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-13T05:11:20.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: osdmap e49: 8 total, 7 up, 8 in 2024-04-13T05:11:20.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:20.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:11:20.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:20 smithi189 ceph-mon[35531]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:11:20.917 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:11:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-13T05:11:21.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: osdmap e49: 8 total, 7 up, 8 in 2024-04-13T05:11:21.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:21.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:11:21.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:20 smithi057 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:11:21.500 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:11:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:11:21.086+0000 7fa23e200700 -1 osd.7 0 waiting for initial osdmap 2024-04-13T05:11:21.500 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:11:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:11:21.103+0000 7fa239000700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:11:22.220 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:22 smithi083 ceph-mon[35133]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:11:22.220 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:22 smithi083 ceph-mon[35133]: osdmap e50: 8 total, 7 up, 8 in 2024-04-13T05:11:22.220 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:22 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:22.220 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:22 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:22.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:22 smithi189 ceph-mon[35531]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:11:22.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:22 smithi189 ceph-mon[35531]: osdmap e50: 8 total, 7 up, 8 in 2024-04-13T05:11:22.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:22 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:22.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:22 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:22.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:22 smithi057 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:11:22.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:22 smithi057 ceph-mon[32331]: osdmap e50: 8 total, 7 up, 8 in 2024-04-13T05:11:22.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:22 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:22.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:22 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:23.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:11:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:11:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] boot 2024-04-13T05:11:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: osdmap e51: 8 total, 8 up, 8 in 2024-04-13T05:11:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:23 smithi189 ceph-mon[35531]: osdmap e52: 8 total, 8 up, 8 in 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] boot 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: osdmap e51: 8 total, 8 up, 8 in 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:23.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:23 smithi057 ceph-mon[32331]: osdmap e52: 8 total, 8 up, 8 in 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: osd.7 [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] boot 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: osdmap e51: 8 total, 8 up, 8 in 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-13T05:11:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:23 smithi083 ceph-mon[35133]: osdmap e52: 8 total, 8 up, 8 in 2024-04-13T05:11:24.021 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-13T05:11:24.021 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-13T05:11:24.021 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-13T05:11:24.021 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-13T05:11:24.021 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036299 s, 289 MB/s 2024-04-13T05:11:24.021 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-13T05:11:24.974 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-04-13T05:11:25.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:25 smithi189 ceph-mon[35531]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:25.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:25 smithi189 ceph-mon[35531]: osdmap e53: 8 total, 8 up, 8 in 2024-04-13T05:11:25.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:25 smithi189 ceph-mon[35531]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-13T05:11:25.411 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:11:25.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:25 smithi083 ceph-mon[35133]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:25.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:25 smithi083 ceph-mon[35133]: osdmap e53: 8 total, 8 up, 8 in 2024-04-13T05:11:25.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:25 smithi083 ceph-mon[35133]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-13T05:11:25.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:25 smithi057 ceph-mon[32331]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:25.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:25 smithi057 ceph-mon[32331]: osdmap e53: 8 total, 8 up, 8 in 2024-04-13T05:11:25.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:25 smithi057 ceph-mon[32331]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-13T05:11:26.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: osdmap e54: 8 total, 8 up, 8 in 2024-04-13T05:11:26.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi083 2024-04-13T05:11:26.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:26 smithi189 ceph-mon[35531]: osdmap e55: 8 total, 8 up, 8 in 2024-04-13T05:11:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: osdmap e54: 8 total, 8 up, 8 in 2024-04-13T05:11:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi083 2024-04-13T05:11:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:26 smithi083 ceph-mon[35133]: osdmap e55: 8 total, 8 up, 8 in 2024-04-13T05:11:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: osdmap e54: 8 total, 8 up, 8 in 2024-04-13T05:11:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi083 2024-04-13T05:11:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:26 smithi057 ceph-mon[32331]: osdmap e55: 8 total, 8 up, 8 in 2024-04-13T05:11:27.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:27 smithi189 ceph-mon[35531]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:27.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:27 smithi083 ceph-mon[35133]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:27.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:27 smithi057 ceph-mon[32331]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 -- 172.21.15.189:0/2741976607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 msgr2=0x7f4828109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 --2- 172.21.15.189:0/2741976607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f4828109570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4818009990 tx=0x7f4818009ca0 comp rx=0 tx=0).stop 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 -- 172.21.15.189:0/2741976607 shutdown_connections 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 --2- 172.21.15.189:0/2741976607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f4828109570 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 --2- 172.21.15.189:0/2741976607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f482810acf0 0x7f48281115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 --2- 172.21.15.189:0/2741976607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 0x7f482810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.664+0000 7f482ec00700 1 -- 172.21.15.189:0/2741976607 >> 172.21.15.189:0/2741976607 conn(0x7f4828078120 msgr2=0x7f482807a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:11:27.666 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.665+0000 7f482ec00700 1 -- 172.21.15.189:0/2741976607 shutdown_connections 2024-04-13T05:11:27.666 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.665+0000 7f482ec00700 1 -- 172.21.15.189:0/2741976607 wait complete. 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 Processor -- start 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 -- start start 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f482819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 0x7f482819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f482810acf0 0x7f48281a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4828114bc0 con 0x7f482810acf0 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4828114a40 con 0x7f482810a350 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f482ec00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4828114d40 con 0x7f4828109150 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.666+0000 7f4827e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f482819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:11:27.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f482819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52166/0 (socket says 172.21.15.189:52166) 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 -- 172.21.15.189:0/2158230714 learned_addr learned my addr 172.21.15.189:0/2158230714 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f482cc00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f482810acf0 0x7f48281a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827400700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 0x7f482819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 -- 172.21.15.189:0/2158230714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 msgr2=0x7f482819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 0x7f482819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 -- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f482810acf0 msgr2=0x7f48281a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f482810acf0 0x7f48281a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 -- 172.21.15.189:0/2158230714 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48281a6ec0 con 0x7f4828109150 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827400700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 0x7f482819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.667+0000 7f4827e00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f482819f260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4818000c00 tx=0x7f48180047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.668+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818003c90 con 0x7f4828109150 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.668+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4818004830 con 0x7f4828109150 2024-04-13T05:11:27.669 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.668+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48281a7150 con 0x7f4828109150 2024-04-13T05:11:27.670 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.668+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818005380 con 0x7f4828109150 2024-04-13T05:11:27.670 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.668+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48281a75a0 con 0x7f4828109150 2024-04-13T05:11:27.670 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.669+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48280697a0 con 0x7f4828109150 2024-04-13T05:11:27.671 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.671+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48180049e0 con 0x7f4828109150 2024-04-13T05:11:27.672 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.672+0000 7f4824c00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f480c0a3cc0 0x7f480c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:11:27.672 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.673+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f48180d0e70 con 0x7f4828109150 2024-04-13T05:11:27.673 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.673+0000 7f4827400700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f480c0a3cc0 0x7f480c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:11:27.674 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.675+0000 7f4827400700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f480c0a3cc0 0x7f480c0a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4814005fd0 tx=0x7f4814005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:11:27.675 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.676+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4818013070 con 0x7f4828109150 2024-04-13T05:11:27.915 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:27.915+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4828062ef0 con 0x7f480c0a3cc0 2024-04-13T05:11:28.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:11:28.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:11:28.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:28.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:11:28.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:11:28.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:28.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:11:28.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:11:28.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:29.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:29 smithi189 ceph-mon[35531]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:29.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:29 smithi189 ceph-mon[35531]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:11:29.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:29 smithi083 ceph-mon[35133]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:29.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:29 smithi083 ceph-mon[35133]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:11:29.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:29 smithi057 ceph-mon[32331]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:29.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:29 smithi057 ceph-mon[32331]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:11:30.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:30 smithi189 ceph-mon[35531]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-13T05:11:30.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:30 smithi189 ceph-mon[35531]: Cluster is now healthy 2024-04-13T05:11:30.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:30 smithi083 ceph-mon[35133]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-13T05:11:30.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:30 smithi083 ceph-mon[35133]: Cluster is now healthy 2024-04-13T05:11:30.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:30 smithi057 ceph-mon[32331]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-13T05:11:30.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:30 smithi057 ceph-mon[32331]: Cluster is now healthy 2024-04-13T05:11:31.499 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:31 smithi083 ceph-mon[35133]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:31.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:31 smithi057 ceph-mon[32331]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:31 smithi189 ceph-mon[35531]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:32.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:32 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1104723860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]: dispatch 2024-04-13T05:11:32.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:32 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]: dispatch 2024-04-13T05:11:32.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:32 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]': finished 2024-04-13T05:11:32.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:32 smithi189 ceph-mon[35531]: osdmap e56: 9 total, 8 up, 9 in 2024-04-13T05:11:32.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:32 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1104723860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]: dispatch 2024-04-13T05:11:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:32 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]: dispatch 2024-04-13T05:11:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:32 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]': finished 2024-04-13T05:11:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:32 smithi083 ceph-mon[35133]: osdmap e56: 9 total, 8 up, 9 in 2024-04-13T05:11:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:32 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1104723860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]: dispatch 2024-04-13T05:11:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:32 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]: dispatch 2024-04-13T05:11:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:32 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "041dc02b-fe99-4ac2-836c-33c8ac7ccc1e"}]': finished 2024-04-13T05:11:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:32 smithi057 ceph-mon[32331]: osdmap e56: 9 total, 8 up, 9 in 2024-04-13T05:11:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:33.402 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:33 smithi083 ceph-mon[35133]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:33.402 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:33 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/1490393717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:11:33.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:33 smithi057 ceph-mon[32331]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:33.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:33 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/1490393717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:11:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:33 smithi189 ceph-mon[35531]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:33 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/1490393717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:11:35.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:35 smithi083 ceph-mon[35133]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:35.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:35 smithi057 ceph-mon[32331]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:35 smithi189 ceph-mon[35531]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:37.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:37 smithi189 ceph-mon[35531]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:37.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:37 smithi083 ceph-mon[35133]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:37.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:37 smithi057 ceph-mon[32331]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:39.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:39 smithi189 ceph-mon[35531]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:39.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:39 smithi083 ceph-mon[35133]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:39.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:39 smithi057 ceph-mon[32331]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:41.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:41 smithi189 ceph-mon[35531]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:41.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:41 smithi083 ceph-mon[35133]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:41.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:41 smithi057 ceph-mon[32331]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:42.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-13T05:11:42.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:42.530 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-13T05:11:42.531 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:42.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-13T05:11:42.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:43.451 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:43 smithi189 ceph-mon[35531]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:43.451 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:43 smithi189 ceph-mon[35531]: Deploying daemon osd.8 on smithi189 2024-04-13T05:11:43.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:43 smithi083 ceph-mon[35133]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:43.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:43 smithi083 ceph-mon[35133]: Deploying daemon osd.8 on smithi189 2024-04-13T05:11:43.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:43 smithi057 ceph-mon[32331]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:43.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:43 smithi057 ceph-mon[32331]: Deploying daemon osd.8 on smithi189 2024-04-13T05:11:45.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:45 smithi189 ceph-mon[35531]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:45.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:45 smithi083 ceph-mon[35133]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:45.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:45 smithi057 ceph-mon[32331]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:47.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:47 smithi189 ceph-mon[35531]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:47.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:47 smithi083 ceph-mon[35133]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:47.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:47 smithi057 ceph-mon[32331]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:49.437 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:49 smithi189 ceph-mon[35531]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:49.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:49 smithi083 ceph-mon[35133]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:49.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:49 smithi057 ceph-mon[32331]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:51.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:51 smithi083 ceph-mon[35133]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:51.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:51.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:51.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:51.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:51 smithi189 ceph-mon[35531]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:51.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:51.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:51.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:51.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:51 smithi057 ceph-mon[32331]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:51.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:51.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:51.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:53.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:53 smithi083 ceph-mon[35133]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:53.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:53 smithi057 ceph-mon[32331]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:53.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:53 smithi189 ceph-mon[35531]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:54.966 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:54 smithi189 ceph-mon[35531]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:54.966 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:54.966 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:54.966 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:54 smithi189 ceph-mon[35531]: from='osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-13T05:11:54.966 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:54 smithi189 ceph-mon[35531]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-13T05:11:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:54 smithi083 ceph-mon[35133]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:54 smithi083 ceph-mon[35133]: from='osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-13T05:11:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:54 smithi083 ceph-mon[35133]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-13T05:11:55.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:54 smithi057 ceph-mon[32331]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:55.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:55.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:55.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:54 smithi057 ceph-mon[32331]: from='osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-13T05:11:55.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:54 smithi057 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-13T05:11:55.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.648+0000 7f4824c00700 1 -- 172.21.15.189:0/2158230714 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4828062ef0 con 0x7f480c0a3cc0 2024-04-13T05:11:55.651 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f480c0a3cc0 msgr2=0x7f480c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f480c0a3cc0 0x7f480c0a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4814005fd0 tx=0x7f4814005dc0 comp rx=0 tx=0).stop 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 msgr2=0x7f482819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f482819f260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4818000c00 tx=0x7f48180047b0 comp rx=0 tx=0).stop 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 shutdown_connections 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4828109150 0x7f482819f260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f480c0a3cc0 0x7f480c0a6180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f482810acf0 0x7f48281a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 --2- 172.21.15.189:0/2158230714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482810a350 0x7f482819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.653+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 >> 172.21.15.189:0/2158230714 conn(0x7f4828078120 msgr2=0x7f482810d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.654+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 shutdown_connections 2024-04-13T05:11:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:11:55.654+0000 7f482ec00700 1 -- 172.21.15.189:0/2158230714 wait complete. 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: osdmap e57: 9 total, 8 up, 9 in 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:11:56.056 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:56.057 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.057 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: osdmap e57: 9 total, 8 up, 9 in 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:11:56.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: osdmap e57: 9 total, 8 up, 9 in 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:11:56.430 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.8.service 2024-04-13T05:11:56.435 INFO:tasks.cephadm:Deploying osd.9 on smithi057 with /dev/vg_nvme/lv_1... 2024-04-13T05:11:56.436 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-13T05:11:56.913 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:11:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:57 smithi057 ceph-mon[32331]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:57 smithi057 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:11:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:57 smithi057 ceph-mon[32331]: osdmap e58: 9 total, 8 up, 9 in 2024-04-13T05:11:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:57.521 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:57 smithi189 ceph-mon[35531]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:57.521 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:57 smithi189 ceph-mon[35531]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:11:57.521 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:57 smithi189 ceph-mon[35531]: osdmap e58: 9 total, 8 up, 9 in 2024-04-13T05:11:57.521 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:57.521 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:57.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:57 smithi083 ceph-mon[35133]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:57 smithi083 ceph-mon[35133]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:11:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:57 smithi083 ceph-mon[35133]: osdmap e58: 9 total, 8 up, 9 in 2024-04-13T05:11:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:11:58.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:11:58.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] boot 2024-04-13T05:11:58.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: osdmap e59: 9 total, 9 up, 9 in 2024-04-13T05:11:58.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:58 smithi189 ceph-mon[35531]: osdmap e60: 9 total, 9 up, 9 in 2024-04-13T05:11:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:11:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:11:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] boot 2024-04-13T05:11:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: osdmap e59: 9 total, 9 up, 9 in 2024-04-13T05:11:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:58 smithi057 ceph-mon[32331]: osdmap e60: 9 total, 9 up, 9 in 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: osd.8 [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] boot 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: osdmap e59: 9 total, 9 up, 9 in 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-13T05:11:58.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:58 smithi083 ceph-mon[35133]: osdmap e60: 9 total, 9 up, 9 in 2024-04-13T05:11:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:59 smithi057 ceph-mon[32331]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:11:59 smithi057 ceph-mon[32331]: osdmap e61: 9 total, 9 up, 9 in 2024-04-13T05:11:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:59 smithi189 ceph-mon[35531]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:11:59 smithi189 ceph-mon[35531]: osdmap e61: 9 total, 9 up, 9 in 2024-04-13T05:11:59.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:59 smithi083 ceph-mon[35133]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-13T05:11:59.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:11:59 smithi083 ceph-mon[35133]: osdmap e61: 9 total, 9 up, 9 in 2024-04-13T05:12:00.107 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-13T05:12:00.107 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-13T05:12:00.107 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-04-13T05:12:00.107 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-04-13T05:12:00.107 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0395284 s, 265 MB/s 2024-04-13T05:12:00.107 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-04-13T05:12:00.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:00 smithi189 ceph-mon[35531]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:00.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:00 smithi057 ceph-mon[32331]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:00.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:00 smithi083 ceph-mon[35133]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:01.079 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_1 2024-04-13T05:12:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:12:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: osdmap e62: 9 total, 9 up, 9 in 2024-04-13T05:12:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi189 2024-04-13T05:12:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi189 to 5696M 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.704 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: osdmap e62: 9 total, 9 up, 9 in 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi189 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi189 to 5696M 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:01.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: osdmap e62: 9 total, 9 up, 9 in 2024-04-13T05:12:01.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi189 2024-04-13T05:12:01.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi189 to 5696M 2024-04-13T05:12:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:02.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:02 smithi189 ceph-mon[35531]: osdmap e63: 9 total, 9 up, 9 in 2024-04-13T05:12:02.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:02 smithi189 ceph-mon[35531]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:02.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:02 smithi057 ceph-mon[32331]: osdmap e63: 9 total, 9 up, 9 in 2024-04-13T05:12:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:02 smithi057 ceph-mon[32331]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:02.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:02 smithi083 ceph-mon[35133]: osdmap e63: 9 total, 9 up, 9 in 2024-04-13T05:12:02.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:02 smithi083 ceph-mon[35133]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 -- 172.21.15.57:0/1746228392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8410a240 msgr2=0x7f3a8410a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 --2- 172.21.15.57:0/1746228392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8410a240 0x7f3a8410a6a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0099c0 tx=0x7f3a6c009cd0 comp rx=0 tx=0).stop 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 -- 172.21.15.57:0/1746228392 shutdown_connections 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 --2- 172.21.15.57:0/1746228392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a84109040 0x7f3a84109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 --2- 172.21.15.57:0/1746228392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 0x7f3a84111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 --2- 172.21.15.57:0/1746228392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a8410a240 0x7f3a8410a6a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 -- 172.21.15.57:0/1746228392 >> 172.21.15.57:0/1746228392 conn(0x7f3a84078010 msgr2=0x7f3a8407a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:12:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 -- 172.21.15.57:0/1746228392 shutdown_connections 2024-04-13T05:12:03.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.885+0000 7f3a8a600700 1 -- 172.21.15.57:0/1746228392 wait complete. 2024-04-13T05:12:03.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.887+0000 7f3a8a600700 1 Processor -- start 2024-04-13T05:12:03.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 -- start start 2024-04-13T05:12:03.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 0x7f3a8419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:03.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8410a240 0x7f3a8419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:03.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 0x7f3a841a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:03.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84114b30 con 0x7f3a84109040 2024-04-13T05:12:03.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841149b0 con 0x7f3a8410abe0 2024-04-13T05:12:03.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.888+0000 7f3a8a600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84114cb0 con 0x7f3a8410a240 2024-04-13T05:12:03.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 0x7f3a8419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:03.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 0x7f3a8419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36830/0 (socket says 172.21.15.57:36830) 2024-04-13T05:12:03.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 -- 172.21.15.57:0/541303621 learned_addr learned my addr 172.21.15.57:0/541303621 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:12:03.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a7be00700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8410a240 0x7f3a8419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:03.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89c00700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 0x7f3a841a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:03.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 -- 172.21.15.57:0/541303621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8410a240 msgr2=0x7f3a8419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:03.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8410a240 0x7f3a8419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:03.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 -- 172.21.15.57:0/541303621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 msgr2=0x7f3a841a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:03.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 0x7f3a841a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:03.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89200700 1 -- 172.21.15.57:0/541303621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a841a6d40 con 0x7f3a84109040 2024-04-13T05:12:03.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.889+0000 7f3a89c00700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 0x7f3a841a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.890+0000 7f3a89200700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 0x7f3a8419f0e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d9b0 tx=0x7f3a7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.890+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74017070 con 0x7f3a84109040 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.890+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a74009de0 con 0x7f3a84109040 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.890+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a841a6fd0 con 0x7f3a84109040 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.890+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400f3c0 con 0x7f3a84109040 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.890+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a841a7430 con 0x7f3a84109040 2024-04-13T05:12:03.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.891+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a840697a0 con 0x7f3a84109040 2024-04-13T05:12:03.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.894+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3a74009930 con 0x7f3a84109040 2024-04-13T05:12:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.895+0000 7f3a82000700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a680a3c40 0x7f3a680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.895+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f3a740cf620 con 0x7f3a84109040 2024-04-13T05:12:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.895+0000 7f3a7be00700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a680a3c40 0x7f3a680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.896+0000 7f3a7be00700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a680a3c40 0x7f3a680a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009990 tx=0x7f3a6c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:12:03.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:03.898+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a7409c910 con 0x7f3a84109040 2024-04-13T05:12:04.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:04.145+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a84062fb0 con 0x7f3a680a3c40 2024-04-13T05:12:05.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:05 smithi189 ceph-mon[35531]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-13T05:12:05.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:05 smithi189 ceph-mon[35531]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:12:05.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:12:05.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:12:05.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:05.334 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:05 smithi057 ceph-mon[32331]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-13T05:12:05.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:05 smithi057 ceph-mon[32331]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:12:05.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:12:05.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:12:05.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:05.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:05 smithi083 ceph-mon[35133]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-13T05:12:05.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:05 smithi083 ceph-mon[35133]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:12:05.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:12:05.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:12:05.455 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:07.140 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:07 smithi083 ceph-mon[35133]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:12:07.167 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:07 smithi057 ceph-mon[32331]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:12:07.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:07 smithi189 ceph-mon[35531]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:12:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:09 smithi057 ceph-mon[32331]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-13T05:12:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3643855149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f43fea2e-9521-419d-85de-fc3068b68ffd"}]: dispatch 2024-04-13T05:12:09.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3643855149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f43fea2e-9521-419d-85de-fc3068b68ffd"}]': finished 2024-04-13T05:12:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:09 smithi057 ceph-mon[32331]: osdmap e64: 10 total, 9 up, 10 in 2024-04-13T05:12:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:09 smithi189 ceph-mon[35531]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-13T05:12:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3643855149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f43fea2e-9521-419d-85de-fc3068b68ffd"}]: dispatch 2024-04-13T05:12:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3643855149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f43fea2e-9521-419d-85de-fc3068b68ffd"}]': finished 2024-04-13T05:12:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:09 smithi189 ceph-mon[35531]: osdmap e64: 10 total, 9 up, 10 in 2024-04-13T05:12:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:09.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:09 smithi083 ceph-mon[35133]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-13T05:12:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3643855149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f43fea2e-9521-419d-85de-fc3068b68ffd"}]: dispatch 2024-04-13T05:12:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3643855149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f43fea2e-9521-419d-85de-fc3068b68ffd"}]': finished 2024-04-13T05:12:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:09 smithi083 ceph-mon[35133]: osdmap e64: 10 total, 9 up, 10 in 2024-04-13T05:12:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:10.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:10 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2822350146' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:12:10.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:10 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2822350146' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:12:10.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:10 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2822350146' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:12:11.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:11 smithi189 ceph-mon[35531]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-13T05:12:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:11 smithi083 ceph-mon[35133]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-13T05:12:11.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:11 smithi057 ceph-mon[32331]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-13T05:12:13.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:13 smithi189 ceph-mon[35531]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:12:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:13 smithi083 ceph-mon[35133]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:12:13.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:13 smithi057 ceph-mon[32331]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-13T05:12:15.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:15 smithi083 ceph-mon[35133]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:15.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:15 smithi057 ceph-mon[32331]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:15.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:15 smithi189 ceph-mon[35531]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:17.493 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:17 smithi083 ceph-mon[35133]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:17.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:17 smithi057 ceph-mon[32331]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:17.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:17 smithi189 ceph-mon[35531]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:19.386 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:19 smithi057 ceph-mon[32331]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:19.386 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-13T05:12:19.386 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:19.386 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:19 smithi057 ceph-mon[32331]: Deploying daemon osd.9 on smithi057 2024-04-13T05:12:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:19 smithi083 ceph-mon[35133]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-13T05:12:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:19 smithi083 ceph-mon[35133]: Deploying daemon osd.9 on smithi057 2024-04-13T05:12:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:19 smithi189 ceph-mon[35531]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-13T05:12:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:19 smithi189 ceph-mon[35531]: Deploying daemon osd.9 on smithi057 2024-04-13T05:12:21.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:21 smithi057 ceph-mon[32331]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:21.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:21 smithi083 ceph-mon[35133]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:21.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:21 smithi189 ceph-mon[35531]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:23.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:23 smithi057 ceph-mon[32331]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:23 smithi083 ceph-mon[35133]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:23.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:23 smithi189 ceph-mon[35531]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:25.361 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:25 smithi057 ceph-mon[32331]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:25.384 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:25 smithi189 ceph-mon[35531]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:25.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:25 smithi083 ceph-mon[35133]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:27.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:27 smithi057 ceph-mon[32331]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:27.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:27 smithi189 ceph-mon[35531]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:27.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:27 smithi083 ceph-mon[35133]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:28.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:28.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:28.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:28 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:12:28.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:28.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:28.375 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:28 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:12:28.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:28.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:28.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:28 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:12:29.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:29 smithi057 ceph-mon[32331]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:29.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:29 smithi083 ceph-mon[35133]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:29.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:29 smithi189 ceph-mon[35531]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:31.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:31 smithi083 ceph-mon[35133]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:31.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:31.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:31.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:31 smithi057 ceph-mon[32331]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:31.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:31.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:31 smithi189 ceph-mon[35531]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:31.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:31.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:32.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.193+0000 7f3a82000700 1 -- 172.21.15.57:0/541303621 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a84062fb0 con 0x7f3a680a3c40 2024-04-13T05:12:32.194 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 9 on host 'smithi057' 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a680a3c40 msgr2=0x7f3a680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a680a3c40 0x7f3a680a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009990 tx=0x7f3a6c005e80 comp rx=0 tx=0).stop 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 msgr2=0x7f3a8419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 0x7f3a8419f0e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d9b0 tx=0x7f3a7400dcc0 comp rx=0 tx=0).stop 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 shutdown_connections 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a680a3c40 0x7f3a680a6100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8410abe0 0x7f3a841a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a84109040 0x7f3a8419f0e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 --2- 172.21.15.57:0/541303621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a8410a240 0x7f3a8419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:32.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.196+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 >> 172.21.15.57:0/541303621 conn(0x7f3a84078010 msgr2=0x7f3a8410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:12:32.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.197+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 shutdown_connections 2024-04-13T05:12:32.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:12:32.197+0000 7f3a8a600700 1 -- 172.21.15.57:0/541303621 wait complete. 2024-04-13T05:12:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:32 smithi083 ceph-mon[35133]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-13T05:12:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:32.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:32 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:32 smithi057 ceph-mon[32331]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-13T05:12:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:32.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:32 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:32.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:32 smithi189 ceph-mon[35531]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-13T05:12:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:32.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:32 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:33.088 DEBUG:teuthology.orchestra.run.smithi057:osd.9> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.9.service 2024-04-13T05:12:33.092 INFO:tasks.cephadm:Deploying osd.10 on smithi083 with /dev/vg_nvme/lv_1... 2024-04-13T05:12:33.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-13T05:12:33.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-mon[32331]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:33.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-mon[32331]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-13T05:12:33.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-mon[32331]: osdmap e65: 10 total, 9 up, 10 in 2024-04-13T05:12:33.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-mon[32331]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:12:33.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:33.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:12:33.209+0000 7f73b3000700 -1 osd.9 0 waiting for initial osdmap 2024-04-13T05:12:33.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:12:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:12:33.225+0000 7f73ade00700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:12:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:33 smithi083 ceph-mon[35133]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:33 smithi083 ceph-mon[35133]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-13T05:12:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:33 smithi083 ceph-mon[35133]: osdmap e65: 10 total, 9 up, 10 in 2024-04-13T05:12:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:33 smithi083 ceph-mon[35133]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:12:33.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:33 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:12:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:33 smithi189 ceph-mon[35531]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:33 smithi189 ceph-mon[35531]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-13T05:12:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:33 smithi189 ceph-mon[35531]: osdmap e65: 10 total, 9 up, 10 in 2024-04-13T05:12:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:33 smithi189 ceph-mon[35531]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-04-13T05:12:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:33 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:34.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:34 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:12:34.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:34 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:12:34.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:34 smithi083 ceph-mon[35133]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:12:34.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:34 smithi083 ceph-mon[35133]: osdmap e66: 10 total, 9 up, 10 in 2024-04-13T05:12:34.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:34 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:34.487 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:34 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:34 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:12:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:34 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:12:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:34 smithi057 ceph-mon[32331]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:12:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:34 smithi057 ceph-mon[32331]: osdmap e66: 10 total, 9 up, 10 in 2024-04-13T05:12:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:34 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:12:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:34 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:12:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:34 smithi189 ceph-mon[35531]: from='osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-04-13T05:12:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:34 smithi189 ceph-mon[35531]: osdmap e66: 10 total, 9 up, 10 in 2024-04-13T05:12:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:34 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:34 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:35.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:35 smithi083 ceph-mon[35133]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:35.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:35 smithi083 ceph-mon[35133]: osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] boot 2024-04-13T05:12:35.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:35 smithi083 ceph-mon[35133]: osdmap e67: 10 total, 10 up, 10 in 2024-04-13T05:12:35.494 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:35.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:35 smithi057 ceph-mon[32331]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:35.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:35 smithi057 ceph-mon[32331]: osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] boot 2024-04-13T05:12:35.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:35 smithi057 ceph-mon[32331]: osdmap e67: 10 total, 10 up, 10 in 2024-04-13T05:12:35.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:35.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:35 smithi189 ceph-mon[35531]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-13T05:12:35.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:35 smithi189 ceph-mon[35531]: osd.9 [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] boot 2024-04-13T05:12:35.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:35 smithi189 ceph-mon[35531]: osdmap e67: 10 total, 10 up, 10 in 2024-04-13T05:12:35.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-13T05:12:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-13T05:12:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-13T05:12:36.395 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-13T05:12:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-13T05:12:36.395 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0337431 s, 311 MB/s 2024-04-13T05:12:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-13T05:12:36.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:36 smithi057 ceph-mon[32331]: osdmap e68: 10 total, 10 up, 10 in 2024-04-13T05:12:36.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:36 smithi189 ceph-mon[35531]: osdmap e68: 10 total, 10 up, 10 in 2024-04-13T05:12:36.747 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:36 smithi083 ceph-mon[35133]: osdmap e68: 10 total, 10 up, 10 in 2024-04-13T05:12:37.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-04-13T05:12:37.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:37 smithi083 ceph-mon[35133]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:37.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:37 smithi057 ceph-mon[32331]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:37.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:37 smithi189 ceph-mon[35531]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:37.649 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:12:39.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:39.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi057 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi057 to 4273M 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:39.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.394 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:39.394 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi057 2024-04-13T05:12:39.394 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.394 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi057 to 4273M 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:39.395 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi057 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:12:39.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi057 to 4273M 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:12:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:12:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.852+0000 7ff989400700 1 -- 172.21.15.83:0/132129131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff984109150 msgr2=0x7ff984109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 --2- 172.21.15.83:0/132129131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff984109150 0x7ff984109570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9740099c0 tx=0x7ff974009cd0 comp rx=0 tx=0).stop 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 -- 172.21.15.83:0/132129131 shutdown_connections 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 --2- 172.21.15.83:0/132129131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff98410acf0 0x7ff9841115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 --2- 172.21.15.83:0/132129131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff984109150 0x7ff984109570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 --2- 172.21.15.83:0/132129131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff98410a350 0x7ff98410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 -- 172.21.15.83:0/132129131 >> 172.21.15.83:0/132129131 conn(0x7ff984078120 msgr2=0x7ff98407a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 -- 172.21.15.83:0/132129131 shutdown_connections 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.853+0000 7ff989400700 1 -- 172.21.15.83:0/132129131 wait complete. 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 Processor -- start 2024-04-13T05:12:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 -- start start 2024-04-13T05:12:39.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff984109150 0x7ff98419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:39.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 0x7ff98419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff98410acf0 0x7ff9841a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff984114c10 con 0x7ff98410acf0 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff984114a90 con 0x7ff98410a350 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.854+0000 7ff989400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff984114d90 con 0x7ff984109150 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 0x7ff98419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 0x7ff98419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41400/0 (socket says 172.21.15.83:41400) 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 -- 172.21.15.83:0/3962106122 learned_addr learned my addr 172.21.15.83:0/3962106122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff982800700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff984109150 0x7ff98419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff983200700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff98410acf0 0x7ff9841a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 -- 172.21.15.83:0/3962106122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff984109150 msgr2=0x7ff98419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff984109150 0x7ff98419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 -- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff98410acf0 msgr2=0x7ff9841a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:12:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff98410acf0 0x7ff9841a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.855+0000 7ff981e00700 1 -- 172.21.15.83:0/3962106122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9841a6e80 con 0x7ff98410a350 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.856+0000 7ff981e00700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 0x7ff98419f760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c9b0 tx=0x7ff97000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.856+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970007b00 con 0x7ff98410a350 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.856+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff970007ca0 con 0x7ff98410a350 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.856+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700186f0 con 0x7ff98410a350 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.856+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9841a7170 con 0x7ff98410a350 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.856+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9841a7660 con 0x7ff98410a350 2024-04-13T05:12:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.857+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9840697a0 con 0x7ff98410a350 2024-04-13T05:12:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.860+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff97000f450 con 0x7ff98410a350 2024-04-13T05:12:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.861+0000 7ff97b400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9680a3c40 0x7ff9680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:12:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.862+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7ff9700cffe0 con 0x7ff98410a350 2024-04-13T05:12:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.862+0000 7ff982800700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9680a3c40 0x7ff9680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:12:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.863+0000 7ff982800700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9680a3c40 0x7ff9680a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff974000c00 tx=0x7ff974005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:12:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:39.863+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff970014030 con 0x7ff98410a350 2024-04-13T05:12:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:12:40.102+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff984062ef0 con 0x7ff9680a3c40 2024-04-13T05:12:41.319 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:41 smithi083 ceph-mon[35133]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:41.320 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:41 smithi083 ceph-mon[35133]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:12:41.320 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:12:41.320 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:12:41.320 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:41 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:41.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:41 smithi189 ceph-mon[35531]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:41.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:41 smithi189 ceph-mon[35531]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:12:41.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:12:41.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:12:41.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:41 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:41.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:41 smithi057 ceph-mon[32331]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:41.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:41 smithi057 ceph-mon[32331]: from='client.34325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:12:41.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:12:41.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:12:41.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:41 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:43.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:43 smithi083 ceph-mon[35133]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:43.331 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:43 smithi189 ceph-mon[35531]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:43.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:43 smithi057 ceph-mon[32331]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:44 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/327863301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]: dispatch 2024-04-13T05:12:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:44 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]: dispatch 2024-04-13T05:12:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:44 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]': finished 2024-04-13T05:12:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:44 smithi189 ceph-mon[35531]: osdmap e69: 11 total, 10 up, 11 in 2024-04-13T05:12:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:12:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:44 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/327863301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]: dispatch 2024-04-13T05:12:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:44 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]: dispatch 2024-04-13T05:12:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:44 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]': finished 2024-04-13T05:12:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:44 smithi083 ceph-mon[35133]: osdmap e69: 11 total, 10 up, 11 in 2024-04-13T05:12:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:12:44.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:44 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/327863301' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]: dispatch 2024-04-13T05:12:44.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:44 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]: dispatch 2024-04-13T05:12:44.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:44 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdef097c-4d96-492e-9d28-1eba9cc759fe"}]': finished 2024-04-13T05:12:44.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:44 smithi057 ceph-mon[32331]: osdmap e69: 11 total, 10 up, 11 in 2024-04-13T05:12:44.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:12:45.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:45 smithi083 ceph-mon[35133]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:45.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:45 smithi083 ceph-mon[35133]: from='client.? 172.21.15.83:0/3641574492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:12:45.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:45 smithi057 ceph-mon[32331]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:45.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:45 smithi057 ceph-mon[32331]: from='client.? 172.21.15.83:0/3641574492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:12:45.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:45 smithi189 ceph-mon[35531]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:45.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:45 smithi189 ceph-mon[35531]: from='client.? 172.21.15.83:0/3641574492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:12:47.450 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:47 smithi083 ceph-mon[35133]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:47.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:47 smithi057 ceph-mon[32331]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:47.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:47 smithi189 ceph-mon[35531]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:49.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:49 smithi083 ceph-mon[35133]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:49.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:49 smithi057 ceph-mon[32331]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:49.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:49 smithi189 ceph-mon[35531]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:51.437 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:51 smithi083 ceph-mon[35133]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:51.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:51 smithi057 ceph-mon[32331]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:51.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:51 smithi189 ceph-mon[35531]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:53.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:53 smithi083 ceph-mon[35133]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:53.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:53 smithi057 ceph-mon[32331]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:53.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:53 smithi189 ceph-mon[35531]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-13T05:12:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:54.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:54 smithi083 ceph-mon[35133]: Deploying daemon osd.10 on smithi083 2024-04-13T05:12:54.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-13T05:12:54.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:54.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:54 smithi057 ceph-mon[32331]: Deploying daemon osd.10 on smithi083 2024-04-13T05:12:54.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-13T05:12:54.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:12:54.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:54 smithi189 ceph-mon[35531]: Deploying daemon osd.10 on smithi083 2024-04-13T05:12:55.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:55 smithi083 ceph-mon[35133]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:55.483 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:55 smithi189 ceph-mon[35531]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:55.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:55 smithi057 ceph-mon[32331]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:57.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:57 smithi083 ceph-mon[35133]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:57.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:57 smithi057 ceph-mon[32331]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:57.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:57 smithi189 ceph-mon[35531]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:59.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:12:59 smithi083 ceph-mon[35133]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:12:59 smithi057 ceph-mon[32331]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:12:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:12:59 smithi189 ceph-mon[35531]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:01.377 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:01 smithi083 ceph-mon[35133]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:01.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:01 smithi057 ceph-mon[32331]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:01.581 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:01 smithi189 ceph-mon[35531]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:02.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:02 smithi189 ceph-mon[35531]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:02.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:02.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:02.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:02.880 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:02 smithi083 ceph-mon[35133]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:02.880 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:02.881 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:02.881 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:03.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:02 smithi057 ceph-mon[32331]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:03.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:03.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:03.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:04.667 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:04 smithi083 ceph-mon[35133]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:04.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:04 smithi189 ceph-mon[35531]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:05.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:04 smithi057 ceph-mon[32331]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:06.489 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:06.489 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:06.489 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:06 smithi083 ceph-mon[35133]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:06.489 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:06 smithi083 ceph-mon[35133]: from='osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-13T05:13:06.489 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:06 smithi083 ceph-mon[35133]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-13T05:13:06.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:06.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:06.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:06 smithi057 ceph-mon[32331]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:06.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:06 smithi057 ceph-mon[32331]: from='osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-13T05:13:06.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:06 smithi057 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-13T05:13:06.587 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 10 on host 'smithi083' 2024-04-13T05:13:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.584+0000 7ff97b400700 1 -- 172.21.15.83:0/3962106122 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff984062ef0 con 0x7ff9680a3c40 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.587+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9680a3c40 msgr2=0x7ff9680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.587+0000 7ff989400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9680a3c40 0x7ff9680a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff974000c00 tx=0x7ff974005fb0 comp rx=0 tx=0).stop 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.587+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 msgr2=0x7ff98419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.587+0000 7ff989400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 0x7ff98419f760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c9b0 tx=0x7ff97000cd70 comp rx=0 tx=0).stop 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 shutdown_connections 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff984109150 0x7ff98419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9680a3c40 0x7ff9680a6100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff98410acf0 0x7ff9841a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 --2- 172.21.15.83:0/3962106122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff98410a350 0x7ff98419f760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 >> 172.21.15.83:0/3962106122 conn(0x7ff984078120 msgr2=0x7ff98410d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 shutdown_connections 2024-04-13T05:13:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-13T05:13:06.588+0000 7ff989400700 1 -- 172.21.15.83:0/3962106122 wait complete. 2024-04-13T05:13:06.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:06.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:06.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:06 smithi189 ceph-mon[35531]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:06.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:06 smithi189 ceph-mon[35531]: from='osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-13T05:13:06.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:06 smithi189 ceph-mon[35531]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-13T05:13:07.420 DEBUG:teuthology.orchestra.run.smithi083:osd.10> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.10.service 2024-04-13T05:13:07.424 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-04-13T05:13:07.425 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-13T05:13:07.622 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-13T05:13:07.622 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: from='osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:13:07.622 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: osdmap e70: 11 total, 10 up, 11 in 2024-04-13T05:13:07.623 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:07.623 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:13:07.623 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:07.623 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: from='osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: osdmap e70: 11 total, 10 up, 11 in 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:07.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:07.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-13T05:13:07.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: from='osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:13:07.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: osdmap e70: 11 total, 10 up, 11 in 2024-04-13T05:13:07.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:07.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-13T05:13:07.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:07.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:07.867 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:13:08.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: osdmap e71: 11 total, 10 up, 11 in 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: osdmap e72: 11 total, 10 up, 11 in 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-mon[35133]: from='osd.10 ' entity='osd.10' 2024-04-13T05:13:08.502 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:13:08.206+0000 7f830b600700 -1 osd.10 0 waiting for initial osdmap 2024-04-13T05:13:08.502 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:13:08 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:13:08.263+0000 7f8301400700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: osdmap e71: 11 total, 10 up, 11 in 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: osdmap e72: 11 total, 10 up, 11 in 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:08 smithi189 ceph-mon[35531]: from='osd.10 ' entity='osd.10' 2024-04-13T05:13:08.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: osdmap e71: 11 total, 10 up, 11 in 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: osdmap e72: 11 total, 10 up, 11 in 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:08.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:08 smithi057 ceph-mon[32331]: from='osd.10 ' entity='osd.10' 2024-04-13T05:13:09.699 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:09 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:13:09.699 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:09 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:13:09.699 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:09.700 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:09 smithi189 ceph-mon[35531]: osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] boot 2024-04-13T05:13:09.700 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:09 smithi189 ceph-mon[35531]: osdmap e73: 11 total, 11 up, 11 in 2024-04-13T05:13:09.700 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:09 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:09 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:13:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:09 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:13:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:09 smithi083 ceph-mon[35133]: osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] boot 2024-04-13T05:13:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:09 smithi083 ceph-mon[35133]: osdmap e73: 11 total, 11 up, 11 in 2024-04-13T05:13:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:09 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:09.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:09 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:13:09.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:09 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:13:09.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:09.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:09 smithi057 ceph-mon[32331]: osd.10 [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] boot 2024-04-13T05:13:09.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:09 smithi057 ceph-mon[32331]: osdmap e73: 11 total, 11 up, 11 in 2024-04-13T05:13:09.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:09 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-13T05:13:11.036 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-13T05:13:11.036 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-13T05:13:11.036 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-13T05:13:11.036 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-13T05:13:11.036 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0352556 s, 297 MB/s 2024-04-13T05:13:11.036 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-13T05:13:11.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:11 smithi189 ceph-mon[35531]: pgmap v302: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:11.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:11 smithi189 ceph-mon[35531]: osdmap e74: 11 total, 11 up, 11 in 2024-04-13T05:13:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:11 smithi083 ceph-mon[35133]: pgmap v302: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:11.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:11 smithi083 ceph-mon[35133]: osdmap e74: 11 total, 11 up, 11 in 2024-04-13T05:13:11.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:11 smithi057 ceph-mon[32331]: pgmap v302: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:11.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:11 smithi057 ceph-mon[32331]: osdmap e74: 11 total, 11 up, 11 in 2024-04-13T05:13:11.964 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-04-13T05:13:12.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: osdmap e75: 11 total, 11 up, 11 in 2024-04-13T05:13:12.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:12.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: osdmap e75: 11 total, 11 up, 11 in 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:12.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:12.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.399 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:13:12.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: osdmap e75: 11 total, 11 up, 11 in 2024-04-13T05:13:12.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:13.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:13.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi083 2024-04-13T05:13:13.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-13T05:13:13.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: osdmap e76: 11 total, 11 up, 11 in 2024-04-13T05:13:13.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:13.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:13.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi083 2024-04-13T05:13:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-13T05:13:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: osdmap e76: 11 total, 11 up, 11 in 2024-04-13T05:13:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:13.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi083 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: osdmap e76: 11 total, 11 up, 11 in 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:14.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:14 smithi189 ceph-mon[35531]: osdmap e77: 11 total, 11 up, 11 in 2024-04-13T05:13:14.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:14 smithi083 ceph-mon[35133]: osdmap e77: 11 total, 11 up, 11 in 2024-04-13T05:13:14.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:14 smithi057 ceph-mon[32331]: osdmap e77: 11 total, 11 up, 11 in 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 -- 172.21.15.189:0/889033052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4105f50 msgr2=0x7fa7b41063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/889033052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4105f50 0x7fa7b41063b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009990 tx=0x7fa7a0009ca0 comp rx=0 tx=0).stop 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 -- 172.21.15.189:0/889033052 shutdown_connections 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/889033052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b41068f0 0x7fa7b410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/889033052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b4104d50 0x7fa7b4105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/889033052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4105f50 0x7fa7b41063b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 -- 172.21.15.189:0/889033052 >> 172.21.15.189:0/889033052 conn(0x7fa7b4100310 msgr2=0x7fa7b4102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 -- 172.21.15.189:0/889033052 shutdown_connections 2024-04-13T05:13:14.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.771+0000 7fa7b9c00700 1 -- 172.21.15.189:0/889033052 wait complete. 2024-04-13T05:13:14.773 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 Processor -- start 2024-04-13T05:13:14.773 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 -- start start 2024-04-13T05:13:14.773 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 0x7fa7b419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:14.773 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b4105f50 0x7fa7b419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:14.773 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b41068f0 0x7fa7b41a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:14.774 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4110830 con 0x7fa7b41068f0 2024-04-13T05:13:14.774 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41106b0 con 0x7fa7b4105f50 2024-04-13T05:13:14.774 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.773+0000 7fa7b9c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41109b0 con 0x7fa7b4104d50 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 0x7fa7b419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 0x7fa7b419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:38500/0 (socket says 172.21.15.189:38500) 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 -- 172.21.15.189:0/3116139633 learned_addr learned my addr 172.21.15.189:0/3116139633 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b3400700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b41068f0 0x7fa7b41a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2000700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b4105f50 0x7fa7b419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 -- 172.21.15.189:0/3116139633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b4105f50 msgr2=0x7fa7b419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b4105f50 0x7fa7b419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 -- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b41068f0 msgr2=0x7fa7b41a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b41068f0 0x7fa7b41a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:14.775 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 -- 172.21.15.189:0/3116139633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b41a2b00 con 0x7fa7b4104d50 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.774+0000 7fa7b2a00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 0x7fa7b419aea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d9b0 tx=0x7fa7a400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.775+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4017070 con 0x7fa7b4104d50 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.775+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7a4010460 con 0x7fa7b4104d50 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.775+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b41a2df0 con 0x7fa7b4104d50 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.775+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400f620 con 0x7fa7b4104d50 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.775+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a3240 con 0x7fa7b4104d50 2024-04-13T05:13:14.776 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.776+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b402dfa0 con 0x7fa7b4104d50 2024-04-13T05:13:14.778 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.778+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7a4010610 con 0x7fa7b4104d50 2024-04-13T05:13:14.780 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.780+0000 7fa7ab400700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7980a3cc0 0x7fa7980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:14.781 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.781+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7fa7a40cf980 con 0x7fa7b4104d50 2024-04-13T05:13:14.781 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.781+0000 7fa7b2000700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7980a3cc0 0x7fa7980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:14.782 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.782+0000 7fa7b2000700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7980a3cc0 0x7fa7980a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0003920 tx=0x7fa7a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:14.786 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:14.786+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7a409c910 con 0x7fa7b4104d50 2024-04-13T05:13:15.025 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:15.024+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa7b4062ef0 con 0x7fa7980a3cc0 2024-04-13T05:13:15.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:15 smithi189 ceph-mon[35531]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-13T05:13:15.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:15 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:13:15.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:15 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:13:15.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:15 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:15.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:15 smithi083 ceph-mon[35133]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-13T05:13:15.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:15 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:13:15.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:15 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:13:15.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:15 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:15.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:15 smithi057 ceph-mon[32331]: pgmap v308: 1 pgs: 1 peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-13T05:13:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-13T05:13:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-13T05:13:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:16.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:16 smithi083 ceph-mon[35133]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:13:16.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:16 smithi189 ceph-mon[35531]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:13:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:16 smithi057 ceph-mon[32331]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:13:17.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:17 smithi189 ceph-mon[35531]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-13T05:13:17.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:17 smithi083 ceph-mon[35133]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-13T05:13:17.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:17 smithi057 ceph-mon[32331]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-13T05:13:19.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:19 smithi083 ceph-mon[35133]: pgmap v310: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:13:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:19 smithi057 ceph-mon[32331]: pgmap v310: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:13:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:19 smithi189 ceph-mon[35531]: pgmap v310: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-13T05:13:20.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:20 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/2839466634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]: dispatch 2024-04-13T05:13:20.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:20 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]: dispatch 2024-04-13T05:13:20.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:20 smithi083 ceph-mon[35133]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]': finished 2024-04-13T05:13:20.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:20 smithi083 ceph-mon[35133]: osdmap e78: 12 total, 11 up, 12 in 2024-04-13T05:13:20.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:20 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:20.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:20 smithi083 ceph-mon[35133]: from='client.? 172.21.15.189:0/4171752398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:13:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:20 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/2839466634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]: dispatch 2024-04-13T05:13:20.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:20 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]: dispatch 2024-04-13T05:13:20.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:20 smithi057 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]': finished 2024-04-13T05:13:20.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:20 smithi057 ceph-mon[32331]: osdmap e78: 12 total, 11 up, 12 in 2024-04-13T05:13:20.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:20 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:20.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:20 smithi057 ceph-mon[32331]: from='client.? 172.21.15.189:0/4171752398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:13:20.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:20 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/2839466634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]: dispatch 2024-04-13T05:13:20.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:20 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]: dispatch 2024-04-13T05:13:20.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:20 smithi189 ceph-mon[35531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "476f1e68-924c-49f6-87b8-bcb1267ff3e2"}]': finished 2024-04-13T05:13:20.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:20 smithi189 ceph-mon[35531]: osdmap e78: 12 total, 11 up, 12 in 2024-04-13T05:13:20.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:20 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:20.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:20 smithi189 ceph-mon[35531]: from='client.? 172.21.15.189:0/4171752398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-13T05:13:21.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:21 smithi083 ceph-mon[35133]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-13T05:13:21.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:21 smithi057 ceph-mon[32331]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-13T05:13:21.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:21 smithi189 ceph-mon[35531]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-13T05:13:23.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:23 smithi083 ceph-mon[35133]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:23 smithi057 ceph-mon[32331]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:23 smithi189 ceph-mon[35531]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:25.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:25 smithi057 ceph-mon[32331]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:25.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:25 smithi189 ceph-mon[35531]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:25.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:25 smithi083 ceph-mon[35133]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:27.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:27 smithi189 ceph-mon[35531]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:27.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:27 smithi057 ceph-mon[32331]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:27.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:27 smithi083 ceph-mon[35133]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:29.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:29 smithi057 ceph-mon[32331]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:29.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:29 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-13T05:13:29.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:29 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:29.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:29 smithi189 ceph-mon[35531]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:29.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:29 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-13T05:13:29.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:29 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:29.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:29 smithi083 ceph-mon[35133]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:29.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:29 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-13T05:13:29.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:29 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:30.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:30 smithi189 ceph-mon[35531]: Deploying daemon osd.11 on smithi189 2024-04-13T05:13:30.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:30 smithi057 ceph-mon[32331]: Deploying daemon osd.11 on smithi189 2024-04-13T05:13:30.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:30 smithi083 ceph-mon[35133]: Deploying daemon osd.11 on smithi189 2024-04-13T05:13:31.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:31 smithi189 ceph-mon[35531]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:31.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:31 smithi083 ceph-mon[35133]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:31.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:31 smithi057 ceph-mon[32331]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:33.560 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:33 smithi189 ceph-mon[35531]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:33.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:33 smithi083 ceph-mon[35133]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:33.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:33 smithi057 ceph-mon[32331]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:34 smithi189 ceph-mon[35531]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:34.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:34 smithi083 ceph-mon[35133]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:34.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:34 smithi057 ceph-mon[32331]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:37.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:36 smithi083 ceph-mon[35133]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:37.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:36 smithi057 ceph-mon[32331]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:37.060 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:36 smithi189 ceph-mon[35531]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:38.907 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:38 smithi189 ceph-mon[35531]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:38.908 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:38 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:38.908 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:38 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:38.908 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:38 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:38 smithi083 ceph-mon[35133]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:38 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:38 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:38 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:39.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:38 smithi057 ceph-mon[32331]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:39.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:38 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:39.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:38 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:39.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:38 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:40 smithi189 ceph-mon[35531]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:41.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:40 smithi083 ceph-mon[35133]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:41.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:40 smithi057 ceph-mon[32331]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:42.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-13T05:13:42.534 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:42 smithi189 ceph-mon[35531]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-13T05:13:42.696 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.694+0000 7fa7ab400700 1 -- 172.21.15.189:0/3116139633 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa7b4062ef0 con 0x7fa7980a3cc0 2024-04-13T05:13:42.696 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7980a3cc0 msgr2=0x7fa7980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7980a3cc0 0x7fa7980a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0003920 tx=0x7fa7a0005fd0 comp rx=0 tx=0).stop 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 msgr2=0x7fa7b419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 0x7fa7b419aea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d9b0 tx=0x7fa7a400dd70 comp rx=0 tx=0).stop 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 shutdown_connections 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4104d50 0x7fa7b419aea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7980a3cc0 0x7fa7980a6180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa7b41068f0 0x7fa7b41a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 --2- 172.21.15.189:0/3116139633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b4105f50 0x7fa7b419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 >> 172.21.15.189:0/3116139633 conn(0x7fa7b4100310 msgr2=0x7fa7b41092d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 shutdown_connections 2024-04-13T05:13:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:42.698+0000 7fa7b9c00700 1 -- 172.21.15.189:0/3116139633 wait complete. 2024-04-13T05:13:42.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-13T05:13:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:42 smithi083 ceph-mon[35133]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-13T05:13:42.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-13T05:13:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:42 smithi057 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-13T05:13:43.546 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.11.service 2024-04-13T05:13:43.551 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-13T05:13:43.551 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd stat -f json 2024-04-13T05:13:43.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: osdmap e79: 12 total, 11 up, 12 in 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-13T05:13:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: osdmap e79: 12 total, 11 up, 12 in 2024-04-13T05:13:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:13:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:13:43.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:43.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:43.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:43.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-13T05:13:43.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: osdmap e79: 12 total, 11 up, 12 in 2024-04-13T05:13:43.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:13:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-13T05:13:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:44.033 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:13:44.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:13:44.206+0000 7fd0d0800700 -1 osd.11 0 waiting for initial osdmap 2024-04-13T05:13:44.339 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:13:44.251+0000 7fd0c6600700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-13T05:13:44.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:44 smithi057 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:13:44.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:44 smithi057 ceph-mon[32331]: osdmap e80: 12 total, 11 up, 12 in 2024-04-13T05:13:44.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:44.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:44.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:44 smithi057 ceph-mon[32331]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:44.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:44 smithi057 ceph-mon[32331]: from='osd.11 ' entity='osd.11' 2024-04-13T05:13:44.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:44 smithi083 ceph-mon[35133]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:13:44.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:44 smithi083 ceph-mon[35133]: osdmap e80: 12 total, 11 up, 12 in 2024-04-13T05:13:44.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:44.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:44.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:44 smithi083 ceph-mon[35133]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:44.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:44 smithi083 ceph-mon[35133]: from='osd.11 ' entity='osd.11' 2024-04-13T05:13:44.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-mon[35531]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-13T05:13:44.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-mon[35531]: osdmap e80: 12 total, 11 up, 12 in 2024-04-13T05:13:44.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:44.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:44.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-mon[35531]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-13T05:13:44.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:44 smithi189 ceph-mon[35531]: from='osd.11 ' entity='osd.11' 2024-04-13T05:13:45.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:45 smithi189 ceph-mon[35531]: purged_snaps scrub starts 2024-04-13T05:13:45.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:45 smithi189 ceph-mon[35531]: purged_snaps scrub ok 2024-04-13T05:13:45.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:45.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:45 smithi189 ceph-mon[35531]: osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] boot 2024-04-13T05:13:45.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:45 smithi189 ceph-mon[35531]: osdmap e81: 12 total, 12 up, 12 in 2024-04-13T05:13:45.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:45 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:45.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:45 smithi057 ceph-mon[32331]: purged_snaps scrub starts 2024-04-13T05:13:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:45 smithi057 ceph-mon[32331]: purged_snaps scrub ok 2024-04-13T05:13:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:45 smithi057 ceph-mon[32331]: osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] boot 2024-04-13T05:13:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:45 smithi057 ceph-mon[32331]: osdmap e81: 12 total, 12 up, 12 in 2024-04-13T05:13:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:45 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:45.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:45 smithi083 ceph-mon[35133]: purged_snaps scrub starts 2024-04-13T05:13:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:45 smithi083 ceph-mon[35133]: purged_snaps scrub ok 2024-04-13T05:13:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:45 smithi083 ceph-mon[35133]: osd.11 [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] boot 2024-04-13T05:13:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:45 smithi083 ceph-mon[35133]: osdmap e81: 12 total, 12 up, 12 in 2024-04-13T05:13:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:45 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 -- 172.21.15.57:0/3118481348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 msgr2=0x7f4e4c105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3118481348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c105170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0099c0 tx=0x7f4e3c009cd0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 -- 172.21.15.57:0/3118481348 shutdown_connections 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3118481348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c1068f0 0x7f4e4c10d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3118481348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c105170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3118481348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4c105f50 0x7f4e4c1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 -- 172.21.15.57:0/3118481348 >> 172.21.15.57:0/3118481348 conn(0x7f4e4c100310 msgr2=0x7f4e4c102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:46.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 -- 172.21.15.57:0/3118481348 shutdown_connections 2024-04-13T05:13:46.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.491+0000 7f4e52a00700 1 -- 172.21.15.57:0/3118481348 wait complete. 2024-04-13T05:13:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.492+0000 7f4e52a00700 1 Processor -- start 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 -- start start 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4c105f50 0x7f4e4c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c1068f0 0x7f4e4c1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c110810 con 0x7f4e4c104d50 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c110690 con 0x7f4e4c105f50 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e52a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c110990 con 0x7f4e4c1068f0 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e4be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e4be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37484/0 (socket says 172.21.15.57:37484) 2024-04-13T05:13:46.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.493+0000 7f4e4be00700 1 -- 172.21.15.57:0/3109396491 learned_addr learned my addr 172.21.15.57:0/3109396491 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4b400700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4c105f50 0x7f4e4c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e50a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c1068f0 0x7f4e4c1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4be00700 1 -- 172.21.15.57:0/3109396491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c1068f0 msgr2=0x7f4e4c1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4be00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c1068f0 0x7f4e4c1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4be00700 1 -- 172.21.15.57:0/3109396491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4c105f50 msgr2=0x7f4e4c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4be00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4c105f50 0x7f4e4c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4be00700 1 -- 172.21.15.57:0/3109396491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c1a2a50 con 0x7f4e4c104d50 2024-04-13T05:13:46.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.494+0000 7f4e4be00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c19adf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0057f0 tx=0x7f4e3c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:46.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.495+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c027070 con 0x7f4e4c104d50 2024-04-13T05:13:46.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.495+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e3c022470 con 0x7f4e4c104d50 2024-04-13T05:13:46.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.495+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1a2ce0 con 0x7f4e4c104d50 2024-04-13T05:13:46.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.495+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00f6c0 con 0x7f4e4c104d50 2024-04-13T05:13:46.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.495+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1a3130 con 0x7f4e4c104d50 2024-04-13T05:13:46.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.496+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c0697a0 con 0x7f4e4c104d50 2024-04-13T05:13:46.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.498+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e3c02a070 con 0x7f4e4c104d50 2024-04-13T05:13:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.500+0000 7f4e48c00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4e300a3c40 0x7f4e300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.500+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4e3c0d10a0 con 0x7f4e4c104d50 2024-04-13T05:13:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.500+0000 7f4e4b400700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4e300a3c40 0x7f4e300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:46.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.501+0000 7f4e4b400700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4e300a3c40 0x7f4e300a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e38005ea0 tx=0x7f4e38005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:46.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.502+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e3c013070 con 0x7f4e4c104d50 2024-04-13T05:13:46.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.733+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4e4c02e1d0 con 0x7f4e4c104d50 2024-04-13T05:13:46.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:46 smithi057 ceph-mon[32331]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:46.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.734+0000 7f4e48c00700 1 -- 172.21.15.57:0/3109396491 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f4e3c09df10 con 0x7f4e4c104d50 2024-04-13T05:13:46.735 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:13:46.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4e300a3c40 msgr2=0x7f4e300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:46.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4e300a3c40 0x7f4e300a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e38005ea0 tx=0x7f4e38005e30 comp rx=0 tx=0).stop 2024-04-13T05:13:46.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 msgr2=0x7f4e4c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:46.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c19adf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0057f0 tx=0x7f4e3c009f80 comp rx=0 tx=0).stop 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 shutdown_connections 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c1068f0 0x7f4e4c1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4e300a3c40 0x7f4e300a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e4c104d50 0x7f4e4c19adf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 --2- 172.21.15.57:0/3109396491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4c105f50 0x7f4e4c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.737+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 >> 172.21.15.57:0/3109396491 conn(0x7f4e4c100310 msgr2=0x7f4e4c109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.738+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 shutdown_connections 2024-04-13T05:13:46.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:46.738+0000 7f4e52a00700 1 -- 172.21.15.57:0/3109396491 wait complete. 2024-04-13T05:13:46.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:46 smithi083 ceph-mon[35133]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:46.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:46 smithi189 ceph-mon[35531]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:47.650 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1712985225,"num_in_osds":12,"osd_in_since":1712985199,"num_remapped_pgs":0} 2024-04-13T05:13:47.651 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd dump --format=json 2024-04-13T05:13:47.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:47 smithi057 ceph-mon[32331]: osdmap e82: 12 total, 12 up, 12 in 2024-04-13T05:13:47.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:47 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3109396491' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-13T05:13:47.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:47 smithi083 ceph-mon[35133]: osdmap e82: 12 total, 12 up, 12 in 2024-04-13T05:13:47.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:47 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3109396491' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-13T05:13:47.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:47 smithi189 ceph-mon[35531]: osdmap e82: 12 total, 12 up, 12 in 2024-04-13T05:13:47.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:47 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3109396491' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-13T05:13:48.120 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:13:48.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:48 smithi057 ceph-mon[32331]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:48.738 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:48 smithi083 ceph-mon[35133]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:48.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:48 smithi189 ceph-mon[35531]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: Detected new or changed devices on smithi189 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi189 to 4272M 2024-04-13T05:13:49.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:49.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:49.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:49.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: Detected new or changed devices on smithi189 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi189 to 4272M 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:50.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: Detected new or changed devices on smithi189 2024-04-13T05:13:50.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi189 to 4272M 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:50.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 -- 172.21.15.57:0/615627339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d81034c0 msgr2=0x7f73d810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 --2- 172.21.15.57:0/615627339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d81034c0 0x7f73d810f920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b0a0 tx=0x7f73d000b3b0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 -- 172.21.15.57:0/615627339 shutdown_connections 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 --2- 172.21.15.57:0/615627339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d81034c0 0x7f73d810f920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 --2- 172.21.15.57:0/615627339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73d8108ac0 0x7f73d8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 --2- 172.21.15.57:0/615627339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73d8102b00 0x7f73d8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 -- 172.21.15.57:0/615627339 >> 172.21.15.57:0/615627339 conn(0x7f73d8079c10 msgr2=0x7f73d807a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 -- 172.21.15.57:0/615627339 shutdown_connections 2024-04-13T05:13:50.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.343+0000 7f73dec00700 1 -- 172.21.15.57:0/615627339 wait complete. 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.344+0000 7f73dec00700 1 Processor -- start 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.344+0000 7f73dec00700 1 -- start start 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dec00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 0x7f73d8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dec00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73d81034c0 0x7f73d8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dec00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73d8108ac0 0x7f73d819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dec00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8112c60 con 0x7f73d8102b00 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dec00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8112ae0 con 0x7f73d81034c0 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dec00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8112de0 con 0x7f73d8108ac0 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73d7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 0x7f73d8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:50.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73d7e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 0x7f73d8199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37496/0 (socket says 172.21.15.57:37496) 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73d7e00700 1 -- 172.21.15.57:0/699983691 learned_addr learned my addr 172.21.15.57:0/699983691 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73cfe00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73d81034c0 0x7f73d8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.345+0000 7f73dcc00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73d8108ac0 0x7f73d819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d7e00700 1 -- 172.21.15.57:0/699983691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73d8108ac0 msgr2=0x7f73d819d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d7e00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73d8108ac0 0x7f73d819d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d7e00700 1 -- 172.21.15.57:0/699983691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73d81034c0 msgr2=0x7f73d8199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d7e00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73d81034c0 0x7f73d8199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d7e00700 1 -- 172.21.15.57:0/699983691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d819e110 con 0x7f73d8102b00 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d7e00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 0x7f73d8199150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73c800c920 tx=0x7f73c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8007b00 con 0x7f73d8102b00 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73c8004ce0 con 0x7f73d8102b00 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d819e400 con 0x7f73d8102b00 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.346+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8005600 con 0x7f73d8102b00 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.347+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d81a5c80 con 0x7f73d8102b00 2024-04-13T05:13:50.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.348+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d80697a0 con 0x7f73d8102b00 2024-04-13T05:13:50.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.350+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f73c8007ca0 con 0x7f73d8102b00 2024-04-13T05:13:50.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.351+0000 7f73d5600700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73bc0a3c40 0x7f73bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:50.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.351+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f73c80cfbb0 con 0x7f73d8102b00 2024-04-13T05:13:50.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.351+0000 7f73cfe00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73bc0a3c40 0x7f73bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:50.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.352+0000 7f73cfe00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73bc0a3c40 0x7f73bc0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73c0005fd0 tx=0x7f73c000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:50.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.356+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73c8014030 con 0x7f73d8102b00 2024-04-13T05:13:50.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.589+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73d802dfa0 con 0x7f73d8102b00 2024-04-13T05:13:50.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.591+0000 7f73d5600700 1 -- 172.21.15.57:0/699983691 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18382 (secure 0 0 0) 0x7f73c809c990 con 0x7f73d8102b00 2024-04-13T05:13:50.593 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:13:50.593 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":82,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:13:46.414789+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.594+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73bc0a3c40 msgr2=0x7f73bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.594+0000 7f73dec00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73bc0a3c40 0x7f73bc0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73c0005fd0 tx=0x7f73c000a560 comp rx=0 tx=0).stop 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 msgr2=0x7f73d8199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 0x7f73d8199150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f73c800c920 tx=0x7f73c800cce0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 shutdown_connections 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73d8108ac0 0x7f73d819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73bc0a3c40 0x7f73bc0a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d8102b00 0x7f73d8199150 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 --2- 172.21.15.57:0/699983691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73d81034c0 0x7f73d8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:50.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 >> 172.21.15.57:0/699983691 conn(0x7f73d8079c10 msgr2=0x7f73d8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:50.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 shutdown_connections 2024-04-13T05:13:50.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:50.595+0000 7f73dec00700 1 -- 172.21.15.57:0/699983691 wait complete. 2024-04-13T05:13:50.977 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:50 smithi057 ceph-mon[32331]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:50.977 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:50 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/699983691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:13:51.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:50 smithi083 ceph-mon[35133]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:51.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:50 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/699983691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:13:51.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:50 smithi189 ceph-mon[35531]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:51.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:50 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/699983691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:13:51.497 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T05:13:51.498 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-13T05:13:51.961 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:13:52.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:52 smithi057 ceph-mon[32331]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:53.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:52 smithi083 ceph-mon[35133]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:53.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:52 smithi189 ceph-mon[35531]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:54.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 -- 172.21.15.57:0/942331770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109340 msgr2=0x7f0670109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 --2- 172.21.15.57:0/942331770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109340 0x7f0670109720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f066000b0d0 tx=0x7f066000b3e0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 -- 172.21.15.57:0/942331770 shutdown_connections 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 --2- 172.21.15.57:0/942331770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067010e420 0x7f0670115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 --2- 172.21.15.57:0/942331770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109340 0x7f0670109720 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 --2- 172.21.15.57:0/942331770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0670109cf0 0x7f067010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 -- 172.21.15.57:0/942331770 >> 172.21.15.57:0/942331770 conn(0x7f0670076280 msgr2=0x7f06701078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 -- 172.21.15.57:0/942331770 shutdown_connections 2024-04-13T05:13:54.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.094+0000 7f0675e00700 1 -- 172.21.15.57:0/942331770 wait complete. 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.095+0000 7f0675e00700 1 Processor -- start 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.095+0000 7f0675e00700 1 -- start start 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0675e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0670109340 0x7f067019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0675e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 0x7f067019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0675e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067010e420 0x7f06701a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0675e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0670119300 con 0x7f0670109cf0 2024-04-13T05:13:54.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0675e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0670119180 con 0x7f067010e420 2024-04-13T05:13:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0675e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0670119480 con 0x7f0670109340 2024-04-13T05:13:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0666000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 0x7f067019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0666000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 0x7f067019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37518/0 (socket says 172.21.15.57:37518) 2024-04-13T05:13:54.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f0666000700 1 -- 172.21.15.57:0/2473544709 learned_addr learned my addr 172.21.15.57:0/2473544709 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f066f400700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067010e420 0x7f06701a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.096+0000 7f066ea00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0670109340 0x7f067019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0666000700 1 -- 172.21.15.57:0/2473544709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0670109340 msgr2=0x7f067019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0666000700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0670109340 0x7f067019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0666000700 1 -- 172.21.15.57:0/2473544709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067010e420 msgr2=0x7f06701a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0666000700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067010e420 0x7f06701a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0666000700 1 -- 172.21.15.57:0/2473544709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06701a4760 con 0x7f0670109cf0 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0666000700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 0x7f067019fc90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f065800c8f0 tx=0x7f065800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:54.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658007b00 con 0x7f0670109cf0 2024-04-13T05:13:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0658007ca0 con 0x7f0670109cf0 2024-04-13T05:13:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658018660 con 0x7f0670109cf0 2024-04-13T05:13:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06701a4a50 con 0x7f0670109cf0 2024-04-13T05:13:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.097+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701ac2d0 con 0x7f0670109cf0 2024-04-13T05:13:54.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.099+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06700697a0 con 0x7f0670109cf0 2024-04-13T05:13:54.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.101+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f065800f450 con 0x7f0670109cf0 2024-04-13T05:13:54.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.102+0000 7f0674a00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06540a3cc0 0x7f06540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.102+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f06580cff90 con 0x7f0670109cf0 2024-04-13T05:13:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.103+0000 7f066ea00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06540a3cc0 0x7f06540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:54.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.103+0000 7f066ea00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06540a3cc0 0x7f06540a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0660009fd0 tx=0x7f066000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:54.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.107+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0658014030 con 0x7f0670109cf0 2024-04-13T05:13:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.341+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f067002dfa0 con 0x7f0670109cf0 2024-04-13T05:13:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.342+0000 7f0674a00700 1 -- 172.21.15.57:0/2473544709 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f065809cd70 con 0x7f0670109cf0 2024-04-13T05:13:54.344 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:13:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.345+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06540a3cc0 msgr2=0x7f06540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.345+0000 7f0675e00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06540a3cc0 0x7f06540a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0660009fd0 tx=0x7f066000b040 comp rx=0 tx=0).stop 2024-04-13T05:13:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.345+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 msgr2=0x7f067019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.345+0000 7f0675e00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 0x7f067019fc90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f065800c8f0 tx=0x7f065800ccb0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 shutdown_connections 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0670109340 0x7f067019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06540a3cc0 0x7f06540a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f067010e420 0x7f06701a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 --2- 172.21.15.57:0/2473544709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0670109cf0 0x7f067019fc90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 >> 172.21.15.57:0/2473544709 conn(0x7f0670076280 msgr2=0x7f0670077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 shutdown_connections 2024-04-13T05:13:54.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:54.346+0000 7f0675e00700 1 -- 172.21.15.57:0/2473544709 wait complete. 2024-04-13T05:13:55.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:54 smithi083 ceph-mon[35133]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:55.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:54 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2473544709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:13:55.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:54 smithi057 ceph-mon[32331]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:55.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:54 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2473544709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:13:55.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:54 smithi189 ceph-mon[35531]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:55.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:54 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2473544709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:13:55.133 INFO:tasks.cephadm:Adding mds.a on smithi057 2024-04-13T05:13:55.133 INFO:tasks.cephadm:Adding mds.d on smithi057 2024-04-13T05:13:55.133 INFO:tasks.cephadm:Adding mds.g on smithi057 2024-04-13T05:13:55.133 INFO:tasks.cephadm:Adding mds.j on smithi057 2024-04-13T05:13:55.133 INFO:tasks.cephadm:Adding mds.b on smithi083 2024-04-13T05:13:55.133 INFO:tasks.cephadm:Adding mds.e on smithi083 2024-04-13T05:13:55.134 INFO:tasks.cephadm:Adding mds.h on smithi083 2024-04-13T05:13:55.134 INFO:tasks.cephadm:Adding mds.k on smithi083 2024-04-13T05:13:55.134 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-04-13T05:13:55.134 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-04-13T05:13:55.134 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-04-13T05:13:55.134 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-04-13T05:13:55.134 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph orch apply mds all '12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-04-13T05:13:55.581 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:13:57.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:56 smithi083 ceph-mon[35133]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:57.008 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:56 smithi189 ceph-mon[35531]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:57.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:56 smithi057 ceph-mon[32331]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 -- 172.21.15.189:0/2278915631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d4109150 msgr2=0x7ff7d4109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/2278915631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d4109150 0x7ff7d4109570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009990 tx=0x7ff7c4009ca0 comp rx=0 tx=0).stop 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 -- 172.21.15.189:0/2278915631 shutdown_connections 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/2278915631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d4109150 0x7ff7d4109570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/2278915631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d410acf0 0x7ff7d41115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/2278915631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7d410a350 0x7ff7d410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:57.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 -- 172.21.15.189:0/2278915631 >> 172.21.15.189:0/2278915631 conn(0x7ff7d4078120 msgr2=0x7ff7d407a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:57.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 -- 172.21.15.189:0/2278915631 shutdown_connections 2024-04-13T05:13:57.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.858+0000 7ff7d9a00700 1 -- 172.21.15.189:0/2278915631 wait complete. 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.859+0000 7ff7d9a00700 1 Processor -- start 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.859+0000 7ff7d9a00700 1 -- start start 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7d4109150 0x7ff7d419f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d9a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 0x7ff7d419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d9a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d410acf0 0x7ff7d41a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d9a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4114c30 con 0x7ff7d4109150 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d9a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4114ab0 con 0x7ff7d410acf0 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d9a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4114db0 con 0x7ff7d410a350 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d2000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 0x7ff7d419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:57.861 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d2000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 0x7ff7d419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:55138/0 (socket says 172.21.15.189:55138) 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d2000700 1 -- 172.21.15.189:0/227293560 learned_addr learned my addr 172.21.15.189:0/227293560 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d2a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7d4109150 0x7ff7d419f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.860+0000 7ff7d3400700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d410acf0 0x7ff7d41a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d2000700 1 -- 172.21.15.189:0/227293560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d410acf0 msgr2=0x7ff7d41a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d2000700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d410acf0 0x7ff7d41a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d2000700 1 -- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7d4109150 msgr2=0x7ff7d419f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d2000700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7d4109150 0x7ff7d419f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:57.862 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d2000700 1 -- 172.21.15.189:0/227293560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d41a6e70 con 0x7ff7d410a350 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d2000700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 0x7ff7d419f750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c9e0 tx=0x7ff7c000cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0007ac0 con 0x7ff7d410a350 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d41a7160 con 0x7ff7d410a350 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c0007c60 con 0x7ff7d410a350 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0018660 con 0x7ff7d410a350 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.861+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41a76a0 con 0x7ff7d410a350 2024-04-13T05:13:57.863 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.862+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d40697a0 con 0x7ff7d410a350 2024-04-13T05:13:57.865 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.865+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7c000f450 con 0x7ff7d410a350 2024-04-13T05:13:57.867 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.867+0000 7ff7cb400700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff7b80a3c40 0x7ff7b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:57.867 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.867+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff7c00cffe0 con 0x7ff7d410a350 2024-04-13T05:13:57.868 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.867+0000 7ff7d2a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff7b80a3c40 0x7ff7b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:57.869 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.868+0000 7ff7d2a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff7b80a3c40 0x7ff7b80a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b5c0 tx=0x7ff7c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:57.872 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:57.871+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7c0014030 con 0x7ff7d410a350 2024-04-13T05:13:58.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.108+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) -- 0x7ff7d40630f0 con 0x7ff7b80a3c40 2024-04-13T05:13:58.126 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.126+0000 7ff7cb400700 1 -- 172.21.15.189:0/227293560 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff7d40630f0 con 0x7ff7b80a3c40 2024-04-13T05:13:58.127 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-04-13T05:13:58.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff7b80a3c40 msgr2=0x7ff7b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff7b80a3c40 0x7ff7b80a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b5c0 tx=0x7ff7c4005fd0 comp rx=0 tx=0).stop 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 msgr2=0x7ff7d419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 0x7ff7d419f750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c9e0 tx=0x7ff7c000cda0 comp rx=0 tx=0).stop 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 shutdown_connections 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff7b80a3c40 0x7ff7b80a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7d410acf0 0x7ff7d41a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7d4109150 0x7ff7d419f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 --2- 172.21.15.189:0/227293560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7d410a350 0x7ff7d419f750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 >> 172.21.15.189:0/227293560 conn(0x7ff7d4078120 msgr2=0x7ff7d410d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 shutdown_connections 2024-04-13T05:13:58.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-13T05:13:58.129+0000 7ff7d9a00700 1 -- 172.21.15.189:0/227293560 wait complete. 2024-04-13T05:13:59.169 DEBUG:teuthology.orchestra.run.smithi057:mds.a> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.a.service 2024-04-13T05:13:59.178 DEBUG:teuthology.orchestra.run.smithi057:mds.d> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.d.service 2024-04-13T05:13:59.225 DEBUG:teuthology.orchestra.run.smithi057:mds.g> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.g.service 2024-04-13T05:13:59.271 DEBUG:teuthology.orchestra.run.smithi057:mds.j> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.j.service 2024-04-13T05:13:59.318 DEBUG:teuthology.orchestra.run.smithi083:mds.b> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.b.service 2024-04-13T05:13:59.326 DEBUG:teuthology.orchestra.run.smithi083:mds.e> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.e.service 2024-04-13T05:13:59.371 DEBUG:teuthology.orchestra.run.smithi083:mds.h> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.h.service 2024-04-13T05:13:59.377 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:59.377 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:59.378 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:13:59 smithi057 ceph-mon[32331]: Deploying daemon mds.l on smithi189 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:59.405 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:59.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:59.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:13:59.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:13:59.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:59.406 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:13:59 smithi083 ceph-mon[35133]: Deploying daemon mds.l on smithi189 2024-04-13T05:13:59.408 DEBUG:teuthology.orchestra.run.smithi083:mds.k> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.k.service 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:13:59.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:13:59.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:13:59.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:13:59 smithi189 ceph-mon[35531]: Deploying daemon mds.l on smithi189 2024-04-13T05:13:59.453 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.c.service 2024-04-13T05:13:59.461 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.f.service 2024-04-13T05:13:59.508 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.i.service 2024-04-13T05:13:59.555 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.l.service 2024-04-13T05:13:59.600 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-13T05:13:59.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:13:59.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 -- 172.21.15.57:0/1824709140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102090 msgr2=0x7f867c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 --2- 172.21.15.57:0/1824709140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102090 0x7f867c102510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8664009900 tx=0x7f866402bc30 comp rx=0 tx=0).stop 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 -- 172.21.15.57:0/1824709140 shutdown_connections 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 --2- 172.21.15.57:0/1824709140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867c102a50 0x7f867c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 --2- 172.21.15.57:0/1824709140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102090 0x7f867c102510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 --2- 172.21.15.57:0/1824709140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c100eb0 0x7f867c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 -- 172.21.15.57:0/1824709140 >> 172.21.15.57:0/1824709140 conn(0x7f867c0fc470 msgr2=0x7f867c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 -- 172.21.15.57:0/1824709140 shutdown_connections 2024-04-13T05:13:59.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.847+0000 7f8683000640 1 -- 172.21.15.57:0/1824709140 wait complete. 2024-04-13T05:13:59.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 Processor -- start 2024-04-13T05:13:59.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 -- start start 2024-04-13T05:13:59.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867c100eb0 0x7f867c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c102090 0x7f867c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 0x7f867c1a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f867c10cb90 con 0x7f867c102a50 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f867c10ca10 con 0x7f867c102090 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.848+0000 7f8683000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f867c10cd10 con 0x7f867c100eb0 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 0x7f867c1a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 0x7f867c1a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33042/0 (socket says 172.21.15.57:33042) 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 -- 172.21.15.57:0/1855201219 learned_addr learned my addr 172.21.15.57:0/1855201219 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f867be00640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867c100eb0 0x7f867c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f867b400640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c102090 0x7f867c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 -- 172.21.15.57:0/1855201219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867c100eb0 msgr2=0x7f867c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867c100eb0 0x7f867c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 -- 172.21.15.57:0/1855201219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c102090 msgr2=0x7f867c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:13:59.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c102090 0x7f867c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 -- 172.21.15.57:0/1855201219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c1a2200 con 0x7f867c102a50 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f867b400640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c102090 0x7f867c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.849+0000 7f8681000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 0x7f867c1a1b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8674009fd0 tx=0x7f867400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.850+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674011830 con 0x7f867c102a50 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.850+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8674016070 con 0x7f867c102a50 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.850+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674010de0 con 0x7f867c102a50 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.850+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c1a24f0 con 0x7f867c102a50 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.850+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c1a2b10 con 0x7f867c102a50 2024-04-13T05:13:59.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.851+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c1a2dd0 con 0x7f867c102a50 2024-04-13T05:13:59.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.854+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f86740119d0 con 0x7f867c102a50 2024-04-13T05:13:59.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.856+0000 7f8678c00640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86600a3c30 0x7f86600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:13:59.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.856+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f86740cfd80 con 0x7f867c102a50 2024-04-13T05:13:59.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.856+0000 7f867be00640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86600a3c30 0x7f86600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:13:59.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.857+0000 7f867be00640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86600a3c30 0x7f86600a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f867c101c00 tx=0x7f866c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:13:59.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:13:59.860+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f867409cb60 con 0x7f867c102a50 2024-04-13T05:14:00.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.067+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f867c1a2fa0 con 0x7f867c102a50 2024-04-13T05:14:00.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.070+0000 7f8678c00640 1 -- 172.21.15.57:0/1855201219 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18382 (secure 0 0 0) 0x7f867409c460 con 0x7f867c102a50 2024-04-13T05:14:00.072 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:00.072 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":82,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:13:46.414789+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:14:00.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.072+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86600a3c30 msgr2=0x7f86600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.072+0000 7f8683000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86600a3c30 0x7f86600a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f867c101c00 tx=0x7f866c0094f0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.072+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 msgr2=0x7f867c1a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.072+0000 7f8683000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 0x7f867c1a1b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8674009fd0 tx=0x7f867400b8e0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 shutdown_connections 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86600a3c30 0x7f86600a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f867c102090 0x7f867c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f867c102a50 0x7f867c1a1b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 --2- 172.21.15.57:0/1855201219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867c100eb0 0x7f867c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 >> 172.21.15.57:0/1855201219 conn(0x7f867c0fc470 msgr2=0x7f867c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 shutdown_connections 2024-04-13T05:14:00.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.073+0000 7f8683000640 1 -- 172.21.15.57:0/1855201219 wait complete. 2024-04-13T05:14:00.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T05:14:00.094 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:14:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.320+0000 7f8d33200640 1 -- 172.21.15.57:0/1668034786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34105cf0 msgr2=0x7f8d3410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.320+0000 7f8d33200640 1 --2- 172.21.15.57:0/1668034786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34105cf0 0x7f8d3410c580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009ca0 tx=0x7f8d2c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 -- 172.21.15.57:0/1668034786 shutdown_connections 2024-04-13T05:14:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 --2- 172.21.15.57:0/1668034786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34105cf0 0x7f8d3410c580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 --2- 172.21.15.57:0/1668034786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d34105330 0x7f8d341057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 --2- 172.21.15.57:0/1668034786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34104140 0x7f8d34104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 -- 172.21.15.57:0/1668034786 >> 172.21.15.57:0/1668034786 conn(0x7f8d340ff6d0 msgr2=0x7f8d34101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:00.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 -- 172.21.15.57:0/1668034786 shutdown_connections 2024-04-13T05:14:00.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.321+0000 7f8d33200640 1 -- 172.21.15.57:0/1668034786 wait complete. 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 Processor -- start 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 -- start start 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 0x7f8d3419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d34105330 0x7f8d3419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34105cf0 0x7f8d341a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410fc80 con 0x7f8d34104140 2024-04-13T05:14:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410fb00 con 0x7f8d34105cf0 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d33200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3410fe00 con 0x7f8d34105330 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.322+0000 7f8d31e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 0x7f8d3419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 0x7f8d3419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33064/0 (socket says 172.21.15.57:33064) 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 -- 172.21.15.57:0/804063771 learned_addr learned my addr 172.21.15.57:0/804063771 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d32800640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34105cf0 0x7f8d341a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 -- 172.21.15.57:0/804063771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d34105330 msgr2=0x7f8d3419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31400640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d34105330 0x7f8d3419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d34105330 0x7f8d3419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 -- 172.21.15.57:0/804063771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34105cf0 msgr2=0x7f8d341a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34105cf0 0x7f8d341a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 -- 172.21.15.57:0/804063771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d341a1e50 con 0x7f8d34104140 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d32800640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34105cf0 0x7f8d341a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.323+0000 7f8d31e00640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 0x7f8d3419a230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d770 tx=0x7f8d1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.324+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c017070 con 0x7f8d34104140 2024-04-13T05:14:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.324+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d1c009c30 con 0x7f8d34104140 2024-04-13T05:14:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.324+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341a2140 con 0x7f8d34104140 2024-04-13T05:14:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.324+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c003d00 con 0x7f8d34104140 2024-04-13T05:14:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.324+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341a2690 con 0x7f8d34104140 2024-04-13T05:14:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.325+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3410b460 con 0x7f8d34104140 2024-04-13T05:14:00.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.327+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8d1c003ea0 con 0x7f8d34104140 2024-04-13T05:14:00.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.329+0000 7f8d2aa00640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d180a3bb0 0x7f8d180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.329+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8d1c0cfdc0 con 0x7f8d34104140 2024-04-13T05:14:00.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.330+0000 7f8d31400640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d180a3bb0 0x7f8d180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.330+0000 7f8d31400640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d180a3bb0 0x7f8d180a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d3419b740 tx=0x7f8d24005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:00.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.334+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d1c09cba0 con 0x7f8d34104140 2024-04-13T05:14:00.467 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1855201219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:14:00.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1855201219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:14:00.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1855201219' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:14:00.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.542+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d34062bc0 con 0x7f8d34104140 2024-04-13T05:14:00.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.543+0000 7f8d2aa00640 1 -- 172.21.15.57:0/804063771 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8d1c009de0 con 0x7f8d34104140 2024-04-13T05:14:00.544 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:14:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d180a3bb0 msgr2=0x7f8d180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d180a3bb0 0x7f8d180a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d3419b740 tx=0x7f8d24005cc0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 msgr2=0x7f8d3419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 0x7f8d3419a230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d770 tx=0x7f8d1c00dc30 comp rx=0 tx=0).stop 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 shutdown_connections 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d180a3bb0 0x7f8d180a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d34105cf0 0x7f8d341a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d34105330 0x7f8d3419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 --2- 172.21.15.57:0/804063771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d34104140 0x7f8d3419a230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 >> 172.21.15.57:0/804063771 conn(0x7f8d340ff6d0 msgr2=0x7f8d34108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 shutdown_connections 2024-04-13T05:14:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.546+0000 7f8d33200640 1 -- 172.21.15.57:0/804063771 wait complete. 2024-04-13T05:14:00.570 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.802+0000 7fc626600640 1 -- 172.21.15.57:0/2678722350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620104240 msgr2=0x7fc620104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.802+0000 7fc626600640 1 --2- 172.21.15.57:0/2678722350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620104240 0x7fc620104660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc610009900 tx=0x7fc61002bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.802+0000 7fc626600640 1 -- 172.21.15.57:0/2678722350 shutdown_connections 2024-04-13T05:14:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.802+0000 7fc626600640 1 --2- 172.21.15.57:0/2678722350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc620105430 0x7fc6201058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.802+0000 7fc626600640 1 --2- 172.21.15.57:0/2678722350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc620105df0 0x7fc62010c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.802+0000 7fc626600640 1 --2- 172.21.15.57:0/2678722350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620104240 0x7fc620104660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.803+0000 7fc626600640 1 -- 172.21.15.57:0/2678722350 >> 172.21.15.57:0/2678722350 conn(0x7fc6200ff7b0 msgr2=0x7fc620101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:00.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.803+0000 7fc626600640 1 -- 172.21.15.57:0/2678722350 shutdown_connections 2024-04-13T05:14:00.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.803+0000 7fc626600640 1 -- 172.21.15.57:0/2678722350 wait complete. 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 Processor -- start 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 -- start start 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc620104240 0x7fc62019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 0x7fc62019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc620105df0 0x7fc6201a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010fdd0 con 0x7fc620105430 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010fc50 con 0x7fc620105df0 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc626600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010ff50 con 0x7fc620104240 2024-04-13T05:14:00.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 0x7fc62019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 0x7fc62019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33078/0 (socket says 172.21.15.57:33078) 2024-04-13T05:14:00.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.804+0000 7fc61ea00640 1 -- 172.21.15.57:0/3933279344 learned_addr learned my addr 172.21.15.57:0/3933279344 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:00.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61f400640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc620104240 0x7fc62019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61fe00640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc620105df0 0x7fc6201a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61ea00640 1 -- 172.21.15.57:0/3933279344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc620104240 msgr2=0x7fc62019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61ea00640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc620104240 0x7fc62019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61ea00640 1 -- 172.21.15.57:0/3933279344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc620105df0 msgr2=0x7fc6201a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61ea00640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc620105df0 0x7fc6201a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61ea00640 1 -- 172.21.15.57:0/3933279344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201a2070 con 0x7fc620105430 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc61ea00640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 0x7fc62019a990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc60800eab0 tx=0x7fc60800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60800ce40 con 0x7fc620105430 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201a2360 con 0x7fc620105430 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc608011920 con 0x7fc620105430 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.805+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608010a00 con 0x7fc620105430 2024-04-13T05:14:00.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.806+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a2900 con 0x7fc620105430 2024-04-13T05:14:00.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.807+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62010b5e0 con 0x7fc620105430 2024-04-13T05:14:00.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.809+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc608011ad0 con 0x7fc620105430 2024-04-13T05:14:00.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.810+0000 7fc617e00640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc6000a3c30 0x7fc6000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:00.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.810+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc6080d06b0 con 0x7fc620105430 2024-04-13T05:14:00.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.811+0000 7fc61f400640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc6000a3c30 0x7fc6000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:00.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.811+0000 7fc61f400640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc6000a3c30 0x7fc6000a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc61002c240 tx=0x7fc610005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:00.815+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc608016030 con 0x7fc620105430 2024-04-13T05:14:01.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.063+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc62002d5e0 con 0x7fc620105430 2024-04-13T05:14:01.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.064+0000 7fc617e00640 1 -- 172.21.15.57:0/3933279344 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fc60809d490 con 0x7fc620105430 2024-04-13T05:14:01.065 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:01.066 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-13T05:14:01.066 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 1 2024-04-13T05:14:01.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc6000a3c30 msgr2=0x7fc6000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:01.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc6000a3c30 0x7fc6000a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc61002c240 tx=0x7fc610005fd0 comp rx=0 tx=0).stop 2024-04-13T05:14:01.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 msgr2=0x7fc62019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:01.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 0x7fc62019a990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc60800eab0 tx=0x7fc60800ef70 comp rx=0 tx=0).stop 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 shutdown_connections 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc6000a3c30 0x7fc6000a60f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc620105df0 0x7fc6201a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc620105430 0x7fc62019a990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 --2- 172.21.15.57:0/3933279344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc620104240 0x7fc62019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.067+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 >> 172.21.15.57:0/3933279344 conn(0x7fc6200ff7b0 msgr2=0x7fc6201087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:01.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.068+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 shutdown_connections 2024-04-13T05:14:01.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:01.068+0000 7fc626600640 1 -- 172.21.15.57:0/3933279344 wait complete. 2024-04-13T05:14:01.493 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:01 smithi189 ceph-mon[35531]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:01.493 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/804063771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:14:01.493 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3933279344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:01.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:01 smithi083 ceph-mon[35133]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:01.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/804063771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:14:01.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3933279344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:01 smithi057 ceph-mon[32331]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/804063771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:14:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3933279344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:01.834 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:01 smithi189 systemd[1]: Starting Ceph mds.l for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:02.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:01 smithi189 podman[65285]: 2024-04-13 05:14:01.755353278 +0000 UTC m=+0.033930938 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:02.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:01 smithi189 podman[65285]: 2024-04-13 05:14:01.873689949 +0000 UTC m=+0.152267646 container create 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-13T05:14:02.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:02 smithi189 podman[65285]: 2024-04-13 05:14:02.169497871 +0000 UTC m=+0.448075575 container init 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:14:02.582 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:02 smithi189 podman[65285]: 2024-04-13 05:14:02.179648818 +0000 UTC m=+0.458226522 container start 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-13T05:14:02.583 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:02 smithi189 ceph-mds[65305]: -- 172.21.15.189:0/2536975026 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627fb9c5a40 con 0x5627fb998c00 2024-04-13T05:14:02.583 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l[65301]: starting mds.l at 2024-04-13T05:14:02.583 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:02 smithi189 bash[65285]: 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d 2024-04-13T05:14:02.583 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:14:02 smithi189 systemd[1]: Started Ceph mds.l for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:03.088 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:03.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 -- 172.21.15.57:0/2721056475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c074a20 msgr2=0x7fb41c074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:03.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 --2- 172.21.15.57:0/2721056475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c074a20 0x7fb41c074ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb408009930 tx=0x7fb40802bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:03.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 -- 172.21.15.57:0/2721056475 shutdown_connections 2024-04-13T05:14:03.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 --2- 172.21.15.57:0/2721056475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb41c10e400 0x7fb41c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 --2- 172.21.15.57:0/2721056475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c074a20 0x7fb41c074ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 --2- 172.21.15.57:0/2721056475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c076000 0x7fb41c074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 -- 172.21.15.57:0/2721056475 >> 172.21.15.57:0/2721056475 conn(0x7fb41c0fd3a0 msgr2=0x7fb41c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:03.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 -- 172.21.15.57:0/2721056475 shutdown_connections 2024-04-13T05:14:03.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.323+0000 7fb422600640 1 -- 172.21.15.57:0/2721056475 wait complete. 2024-04-13T05:14:03.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.324+0000 7fb422600640 1 Processor -- start 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.324+0000 7fb422600640 1 -- start start 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.324+0000 7fb422600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb41c074a20 0x7fb41c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb422600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c076000 0x7fb41c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb422600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 0x7fb41c1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb422600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c113fc0 con 0x7fb41c10e400 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb422600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c113e40 con 0x7fb41c076000 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb422600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c114140 con 0x7fb41c074a20 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 0x7fb41c1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 0x7fb41c1a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33102/0 (socket says 172.21.15.57:33102) 2024-04-13T05:14:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 -- 172.21.15.57:0/2655235575 learned_addr learned my addr 172.21.15.57:0/2655235575 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb41be00640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c076000 0x7fb41c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421200640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb41c074a20 0x7fb41c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 -- 172.21.15.57:0/2655235575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb41c074a20 msgr2=0x7fb41c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb41c074a20 0x7fb41c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 -- 172.21.15.57:0/2655235575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c076000 msgr2=0x7fb41c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c076000 0x7fb41c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.325+0000 7fb421c00640 1 -- 172.21.15.57:0/2655235575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c1a62a0 con 0x7fb41c10e400 2024-04-13T05:14:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.326+0000 7fb421c00640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 0x7fb41c1a5c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cad0 tx=0x7fb41400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.326+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400ae20 con 0x7fb41c10e400 2024-04-13T05:14:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.326+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb414005020 con 0x7fb41c10e400 2024-04-13T05:14:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.326+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400f6b0 con 0x7fb41c10e400 2024-04-13T05:14:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.326+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c1a6590 con 0x7fb41c10e400 2024-04-13T05:14:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.326+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1a6a60 con 0x7fb41c10e400 2024-04-13T05:14:03.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.327+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c1045d0 con 0x7fb41c10e400 2024-04-13T05:14:03.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.329+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4140051d0 con 0x7fb41c10e400 2024-04-13T05:14:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.331+0000 7fb419600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4040a3bb0 0x7fb4040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.331+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb4140cfff0 con 0x7fb41c10e400 2024-04-13T05:14:03.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.331+0000 7fb421200640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4040a3bb0 0x7fb4040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:03.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.332+0000 7fb421200640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4040a3bb0 0x7fb4040a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb410009b60 tx=0x7fb410005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:03.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.335+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb414014030 con 0x7fb41c10e400 2024-04-13T05:14:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.189:6832/550433387,v1:172.21.15.189:6833/550433387] up:boot 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: fsmap 1 up:standby 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:03.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:03.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.189:6832/550433387,v1:172.21.15.189:6833/550433387] up:boot 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: fsmap 1 up:standby 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:03.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.189:6832/550433387,v1:172.21.15.189:6833/550433387] up:boot 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: fsmap 1 up:standby 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:03.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.584+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb41c02d5e0 con 0x7fb41c10e400 2024-04-13T05:14:03.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.585+0000 7fb419600640 1 -- 172.21.15.57:0/2655235575 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fb41409ce10 con 0x7fb41c10e400 2024-04-13T05:14:03.585 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:03.586 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-13T05:14:03.586 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 2 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4040a3bb0 msgr2=0x7fb4040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4040a3bb0 0x7fb4040a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb410009b60 tx=0x7fb410005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 msgr2=0x7fb41c1a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 0x7fb41c1a5c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cad0 tx=0x7fb41400cf90 comp rx=0 tx=0).stop 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 shutdown_connections 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4040a3bb0 0x7fb4040a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb41c076000 0x7fb41c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb41c10e400 0x7fb41c1a5c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 --2- 172.21.15.57:0/2655235575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb41c074a20 0x7fb41c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.588+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 >> 172.21.15.57:0/2655235575 conn(0x7fb41c0fd3a0 msgr2=0x7fb41c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:03.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.589+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 shutdown_connections 2024-04-13T05:14:03.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:03.589+0000 7fb422600640 1 -- 172.21.15.57:0/2655235575 wait complete. 2024-04-13T05:14:04.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:04 smithi057 ceph-mon[32331]: Deploying daemon mds.k on smithi083 2024-04-13T05:14:04.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:04 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2655235575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:04.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:04 smithi189 ceph-mon[35531]: Deploying daemon mds.k on smithi083 2024-04-13T05:14:04.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:04 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2655235575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:04.697 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:04 smithi083 ceph-mon[35133]: Deploying daemon mds.k on smithi083 2024-04-13T05:14:04.697 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:04 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2655235575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:05.495 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:05 smithi083 ceph-mon[35133]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:05 smithi057 ceph-mon[32331]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:05.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:05 smithi189 ceph-mon[35531]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:05.611 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:05.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 -- 172.21.15.57:0/2082843915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c102f80 msgr2=0x7fa10c103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:05.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 --2- 172.21.15.57:0/2082843915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c102f80 0x7fa10c103400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009900 tx=0x7fa0f402bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 -- 172.21.15.57:0/2082843915 shutdown_connections 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 --2- 172.21.15.57:0/2082843915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa10c103940 0x7fa10c10a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 --2- 172.21.15.57:0/2082843915 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c102f80 0x7fa10c103400 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 --2- 172.21.15.57:0/2082843915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10c101e30 0x7fa10c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 -- 172.21.15.57:0/2082843915 >> 172.21.15.57:0/2082843915 conn(0x7fa10c0fd3a0 msgr2=0x7fa10c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 -- 172.21.15.57:0/2082843915 shutdown_connections 2024-04-13T05:14:05.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.846+0000 7fa111c00640 1 -- 172.21.15.57:0/2082843915 wait complete. 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 Processor -- start 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 -- start start 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa10c101e30 0x7fa10c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10c102f80 0x7fa10c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 0x7fa10c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10d9e0 con 0x7fa10c103940 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10d860 con 0x7fa10c102f80 2024-04-13T05:14:05.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.847+0000 7fa111c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10db60 con 0x7fa10c101e30 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 0x7fa10c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 0x7fa10c19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52442/0 (socket says 172.21.15.57:52442) 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 -- 172.21.15.57:0/2595713811 learned_addr learned my addr 172.21.15.57:0/2595713811 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa103400640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10c102f80 0x7fa10c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa10be00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa10c101e30 0x7fa10c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 -- 172.21.15.57:0/2595713811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa10c101e30 msgr2=0x7fa10c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa10c101e30 0x7fa10c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 -- 172.21.15.57:0/2595713811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10c102f80 msgr2=0x7fa10c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10c102f80 0x7fa10c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 -- 172.21.15.57:0/2595713811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c19fd20 con 0x7fa10c103940 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.848+0000 7fa111200640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 0x7fa10c19f6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa10400d750 tx=0x7fa10400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.849+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104017070 con 0x7fa10c103940 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.849+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1040054d0 con 0x7fa10c103940 2024-04-13T05:14:05.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.849+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104004c60 con 0x7fa10c103940 2024-04-13T05:14:05.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.849+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c19ffb0 con 0x7fa10c103940 2024-04-13T05:14:05.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.849+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1a0520 con 0x7fa10c103940 2024-04-13T05:14:05.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.850+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10c109200 con 0x7fa10c103940 2024-04-13T05:14:05.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.852+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa104004e00 con 0x7fa10c103940 2024-04-13T05:14:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.853+0000 7fa109600640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0f00a3c30 0x7fa0f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.854+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1040cf940 con 0x7fa10c103940 2024-04-13T05:14:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.854+0000 7fa10be00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0f00a3c30 0x7fa0f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:05.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.855+0000 7fa10be00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0f00a3c30 0x7fa0f00a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005fd0 tx=0x7fa0fc009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:05.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:05.858+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa10409c720 con 0x7fa10c103940 2024-04-13T05:14:06.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.107+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa10c062bc0 con 0x7fa10c103940 2024-04-13T05:14:06.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.108+0000 7fa109600640 1 -- 172.21.15.57:0/2595713811 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fa10409c020 con 0x7fa10c103940 2024-04-13T05:14:06.108 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:06.108 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-13T05:14:06.108 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 2 2024-04-13T05:14:06.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0f00a3c30 msgr2=0x7fa0f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:06.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0f00a3c30 0x7fa0f00a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc005fd0 tx=0x7fa0fc009520 comp rx=0 tx=0).stop 2024-04-13T05:14:06.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 msgr2=0x7fa10c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 0x7fa10c19f6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa10400d750 tx=0x7fa10400dc10 comp rx=0 tx=0).stop 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 shutdown_connections 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0f00a3c30 0x7fa0f00a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10c102f80 0x7fa10c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa10c103940 0x7fa10c19f6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 --2- 172.21.15.57:0/2595713811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa10c101e30 0x7fa10c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 >> 172.21.15.57:0/2595713811 conn(0x7fa10c0fd3a0 msgr2=0x7fa10c1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 shutdown_connections 2024-04-13T05:14:06.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:06.111+0000 7fa111c00640 1 -- 172.21.15.57:0/2595713811 wait complete. 2024-04-13T05:14:06.117 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:05 smithi083 podman[64894]: 2024-04-13 05:14:05.856403204 +0000 UTC m=+0.143689179 container create 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-13T05:14:06.117 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:06 smithi083 podman[64894]: 2024-04-13 05:14:06.114584846 +0000 UTC m=+0.401870848 container init 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-13T05:14:06.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:06 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2595713811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:06.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:06 smithi083 podman[64894]: 2024-04-13 05:14:06.127407021 +0000 UTC m=+0.414693023 container start 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:14:06.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:06 smithi083 ceph-mds[64920]: -- 172.21.15.83:0/4061960671 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfed915a40 con 0x55cfed8e8c00 2024-04-13T05:14:06.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k[64914]: starting mds.k at 2024-04-13T05:14:06.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:06 smithi083 bash[64894]: 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 2024-04-13T05:14:06.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:14:06 smithi083 systemd[1]: Started Ceph mds.k for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:06.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:06 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2595713811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:06.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:06 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2595713811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:07.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6832/617814721,v1:172.21.15.83:6833/617814721] up:boot 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: fsmap 2 up:standby 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6832/617814721,v1:172.21.15.83:6833/617814721] up:boot 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: fsmap 2 up:standby 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.610 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:07.611 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:07.611 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6832/617814721,v1:172.21.15.83:6833/617814721] up:boot 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: fsmap 2 up:standby 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:08.133 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 -- 172.21.15.57:0/3862412632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103bc0 msgr2=0x7f471c10a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 --2- 172.21.15.57:0/3862412632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103bc0 0x7f471c10a450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4714009ca0 tx=0x7f471402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 -- 172.21.15.57:0/3862412632 shutdown_connections 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 --2- 172.21.15.57:0/3862412632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103bc0 0x7f471c10a450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 --2- 172.21.15.57:0/3862412632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f471c103200 0x7f471c103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 --2- 172.21.15.57:0/3862412632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471c102010 0x7f471c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.365+0000 7f4723600640 1 -- 172.21.15.57:0/3862412632 >> 172.21.15.57:0/3862412632 conn(0x7f471c0fd5c0 msgr2=0x7f471c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:08.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.366+0000 7f4723600640 1 -- 172.21.15.57:0/3862412632 shutdown_connections 2024-04-13T05:14:08.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.366+0000 7f4723600640 1 -- 172.21.15.57:0/3862412632 wait complete. 2024-04-13T05:14:08.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.367+0000 7f4723600640 1 Processor -- start 2024-04-13T05:14:08.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.367+0000 7f4723600640 1 -- start start 2024-04-13T05:14:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.367+0000 7f4723600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f471c102010 0x7f471c075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f4723600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 0x7f471c0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f4723600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471c103bc0 0x7f471c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f4723600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f471c10db50 con 0x7f471c103200 2024-04-13T05:14:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f4723600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f471c10d9d0 con 0x7f471c103bc0 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f4723600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f471c10dcd0 con 0x7f471c102010 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 0x7f471c0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 0x7f471c0743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52456/0 (socket says 172.21.15.57:52456) 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 -- 172.21.15.57:0/494146771 learned_addr learned my addr 172.21.15.57:0/494146771 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f4720c00640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f471c102010 0x7f471c075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 -- 172.21.15.57:0/494146771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f471c102010 msgr2=0x7f471c075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f471c102010 0x7f471c075d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 -- 172.21.15.57:0/494146771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471c103bc0 msgr2=0x7f471c0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471c103bc0 0x7f471c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.368+0000 7f471be00640 1 -- 172.21.15.57:0/494146771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c075180 con 0x7f471c103200 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.369+0000 7f471be00640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 0x7f471c0743b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f470800d770 tx=0x7f470800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.369+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708017070 con 0x7f471c103200 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.369+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47080054d0 con 0x7f471c103200 2024-04-13T05:14:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.369+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c1a2330 con 0x7f471c103200 2024-04-13T05:14:08.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.369+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708004cf0 con 0x7f471c103200 2024-04-13T05:14:08.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.369+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c1a2830 con 0x7f471c103200 2024-04-13T05:14:08.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.370+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471c109350 con 0x7f471c103200 2024-04-13T05:14:08.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.373+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4708004e90 con 0x7f471c103200 2024-04-13T05:14:08.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.374+0000 7f4719600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47040a3bb0 0x7f47040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:08.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.374+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47080cfb30 con 0x7f471c103200 2024-04-13T05:14:08.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.375+0000 7f4720c00640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47040a3bb0 0x7f47040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:08.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.375+0000 7f4720c00640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47040a3bb0 0x7f47040a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f4710005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:08.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.379+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f470809c910 con 0x7f471c103200 2024-04-13T05:14:08.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:08 smithi057 ceph-mon[32331]: Deploying daemon mds.j on smithi057 2024-04-13T05:14:08.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:08 smithi189 ceph-mon[35531]: Deploying daemon mds.j on smithi057 2024-04-13T05:14:08.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.630+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f471c062bc0 con 0x7f471c103200 2024-04-13T05:14:08.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.631+0000 7f4719600640 1 -- 172.21.15.57:0/494146771 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f470809c210 con 0x7f471c103200 2024-04-13T05:14:08.631 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:08.632 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-13T05:14:08.632 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2024-04-13T05:14:08.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.633+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47040a3bb0 msgr2=0x7f47040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:08.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.633+0000 7f4723600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47040a3bb0 0x7f47040a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f4710005850 comp rx=0 tx=0).stop 2024-04-13T05:14:08.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 msgr2=0x7f471c0743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:08.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 0x7f471c0743b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f470800d770 tx=0x7f470800dc30 comp rx=0 tx=0).stop 2024-04-13T05:14:08.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 shutdown_connections 2024-04-13T05:14:08.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47040a3bb0 0x7f47040a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f471c103bc0 0x7f471c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f471c103200 0x7f471c0743b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 --2- 172.21.15.57:0/494146771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f471c102010 0x7f471c075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:08.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 >> 172.21.15.57:0/494146771 conn(0x7f471c0fd5c0 msgr2=0x7f471c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:08.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 shutdown_connections 2024-04-13T05:14:08.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:08.634+0000 7f4723600640 1 -- 172.21.15.57:0/494146771 wait complete. 2024-04-13T05:14:08.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:08 smithi083 ceph-mon[35133]: Deploying daemon mds.j on smithi057 2024-04-13T05:14:09.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:09 smithi189 ceph-mon[35531]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:09.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/494146771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:09.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:09 smithi057 ceph-mon[32331]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:09.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/494146771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:09.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:09 smithi083 ceph-mon[35133]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/494146771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:09.994 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:09 smithi057 systemd[1]: Starting Ceph mds.j for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:10.272 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:09 smithi057 podman[73836]: 2024-04-13 05:14:09.991600836 +0000 UTC m=+0.123702032 container create a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-13T05:14:10.272 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:09 smithi057 podman[73836]: 2024-04-13 05:14:09.904169691 +0000 UTC m=+0.036270854 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:10.272 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:10 smithi057 podman[73836]: 2024-04-13 05:14:10.270394743 +0000 UTC m=+0.402495946 container init a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:14:10.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:10 smithi057 ceph-mon[32331]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:10.532 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:10 smithi057 podman[73836]: 2024-04-13 05:14:10.281924478 +0000 UTC m=+0.414025644 container start a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-13T05:14:10.532 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:10 smithi057 ceph-mds[73855]: -- 172.21.15.57:0/3151264026 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627eb2f1a40 con 0x5627eb2c5000 2024-04-13T05:14:10.532 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j[73851]: starting mds.j at 2024-04-13T05:14:10.532 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:10 smithi057 bash[73836]: a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 2024-04-13T05:14:10.532 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:14:10 smithi057 systemd[1]: Started Ceph mds.j for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:10.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:10 smithi189 ceph-mon[35531]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:10.654 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:10.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:10 smithi083 ceph-mon[35133]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.870+0000 7f9727200640 1 -- 172.21.15.57:0/3263683116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728102fb0 msgr2=0x7f9728103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.870+0000 7f9727200640 1 --2- 172.21.15.57:0/3263683116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728102fb0 0x7f9728103430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971402bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 -- 172.21.15.57:0/3263683116 shutdown_connections 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 --2- 172.21.15.57:0/3263683116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9728103970 0x7f972810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 --2- 172.21.15.57:0/3263683116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728102fb0 0x7f9728103430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 --2- 172.21.15.57:0/3263683116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728101e60 0x7f9728102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 -- 172.21.15.57:0/3263683116 >> 172.21.15.57:0/3263683116 conn(0x7f97280fd3d0 msgr2=0x7f97280ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 -- 172.21.15.57:0/3263683116 shutdown_connections 2024-04-13T05:14:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.871+0000 7f9727200640 1 -- 172.21.15.57:0/3263683116 wait complete. 2024-04-13T05:14:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 Processor -- start 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 -- start start 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9728101e60 0x7f972819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728102fb0 0x7f972819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 0x7f97281a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f972810d9c0 con 0x7f9728103970 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f972810d840 con 0x7f9728102fb0 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.872+0000 7f9727200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f972810db40 con 0x7f9728101e60 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 0x7f97281a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 0x7f97281a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52488/0 (socket says 172.21.15.57:52488) 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 -- 172.21.15.57:0/206183047 learned_addr learned my addr 172.21.15.57:0/206183047 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9725e00640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9728101e60 0x7f972819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9725400640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728102fb0 0x7f972819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:10.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 -- 172.21.15.57:0/206183047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9728101e60 msgr2=0x7f972819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9728101e60 0x7f972819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 -- 172.21.15.57:0/206183047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728102fb0 msgr2=0x7f972819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728102fb0 0x7f972819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 -- 172.21.15.57:0/206183047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97281a1ea0 con 0x7f9728103970 2024-04-13T05:14:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9725400640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728102fb0 0x7f972819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.873+0000 7f9726800640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 0x7f97281a1830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f972000eaa0 tx=0x7f972000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.874+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972000cdb0 con 0x7f9728103970 2024-04-13T05:14:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.874+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97281a2190 con 0x7f9728103970 2024-04-13T05:14:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.874+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9720005020 con 0x7f9728103970 2024-04-13T05:14:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.874+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97200116d0 con 0x7f9728103970 2024-04-13T05:14:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.874+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a2700 con 0x7f9728103970 2024-04-13T05:14:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.875+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97281a29c0 con 0x7f9728103970 2024-04-13T05:14:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.878+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f97200051d0 con 0x7f9728103970 2024-04-13T05:14:10.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.879+0000 7f971ea00640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f970c0a3bb0 0x7f970c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:10.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.879+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f97200d0580 con 0x7f9728103970 2024-04-13T05:14:10.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.879+0000 7f9725e00640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f970c0a3bb0 0x7f970c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.880+0000 7f9725e00640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f970c0a3bb0 0x7f970c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f971800bbe0 tx=0x7f971800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:10.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:10.887+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9720016030 con 0x7f9728103970 2024-04-13T05:14:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.138+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97281a2b90 con 0x7f9728103970 2024-04-13T05:14:11.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.138+0000 7f971ea00640 1 -- 172.21.15.57:0/206183047 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f97200bbe20 con 0x7f9728103970 2024-04-13T05:14:11.139 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:11.139 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-13T05:14:11.139 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2024-04-13T05:14:11.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f970c0a3bb0 msgr2=0x7f970c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f970c0a3bb0 0x7f970c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f971800bbe0 tx=0x7f971800b390 comp rx=0 tx=0).stop 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 msgr2=0x7f97281a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 0x7f97281a1830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f972000eaa0 tx=0x7f972000ef60 comp rx=0 tx=0).stop 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 shutdown_connections 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f970c0a3bb0 0x7f970c0a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728102fb0 0x7f972819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9728103970 0x7f97281a1830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 --2- 172.21.15.57:0/206183047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9728101e60 0x7f972819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 >> 172.21.15.57:0/206183047 conn(0x7f97280fd3d0 msgr2=0x7f9728108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.141+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 shutdown_connections 2024-04-13T05:14:11.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:11.142+0000 7f9727200640 1 -- 172.21.15.57:0/206183047 wait complete. 2024-04-13T05:14:11.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:11.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: Deploying daemon mds.d on smithi057 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/206183047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.57:6834/378251169,v1:172.21.15.57:6835/378251169] up:boot 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: fsmap 3 up:standby 2024-04-13T05:14:11.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: Deploying daemon mds.d on smithi057 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/206183047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.57:6834/378251169,v1:172.21.15.57:6835/378251169] up:boot 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: fsmap 3 up:standby 2024-04-13T05:14:11.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: Deploying daemon mds.d on smithi057 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/206183047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:12.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.57:6834/378251169,v1:172.21.15.57:6835/378251169] up:boot 2024-04-13T05:14:12.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: fsmap 3 up:standby 2024-04-13T05:14:12.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-13T05:14:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:12 smithi057 ceph-mon[32331]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:12.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:12 smithi189 ceph-mon[35531]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:13.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:12 smithi083 ceph-mon[35133]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:13.162 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:13.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 -- 172.21.15.57:0/4259439522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105430 msgr2=0x7fa57c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 --2- 172.21.15.57:0/4259439522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105430 0x7fa57c1058b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa568009900 tx=0x7fa56802bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 -- 172.21.15.57:0/4259439522 shutdown_connections 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 --2- 172.21.15.57:0/4259439522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105430 0x7fa57c1058b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 --2- 172.21.15.57:0/4259439522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105df0 0x7fa57c10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 --2- 172.21.15.57:0/4259439522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 0x7fa57c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 -- 172.21.15.57:0/4259439522 >> 172.21.15.57:0/4259439522 conn(0x7fa57c0ff7b0 msgr2=0x7fa57c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 -- 172.21.15.57:0/4259439522 shutdown_connections 2024-04-13T05:14:13.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.445+0000 7fa582000640 1 -- 172.21.15.57:0/4259439522 wait complete. 2024-04-13T05:14:13.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.447+0000 7fa582000640 1 Processor -- start 2024-04-13T05:14:13.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.447+0000 7fa582000640 1 -- start start 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa582000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 0x7fa57c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa582000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 0x7fa57c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa582000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105df0 0x7fa57c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa582000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c10fd80 con 0x7fa57c105430 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa582000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c10fc00 con 0x7fa57c105df0 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa582000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c10ff00 con 0x7fa57c104240 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa57aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 0x7fa57c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa57aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 0x7fa57c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52494/0 (socket says 172.21.15.57:52494) 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa57aa00640 1 -- 172.21.15.57:0/791663485 learned_addr learned my addr 172.21.15.57:0/791663485 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa57b400640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 0x7fa57c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:13.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.448+0000 7fa57be00640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105df0 0x7fa57c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57aa00640 1 -- 172.21.15.57:0/791663485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 msgr2=0x7fa57c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57aa00640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 0x7fa57c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57aa00640 1 -- 172.21.15.57:0/791663485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105df0 msgr2=0x7fa57c1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57aa00640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105df0 0x7fa57c1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57aa00640 1 -- 172.21.15.57:0/791663485 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c1a20a0 con 0x7fa57c105430 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57be00640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105df0 0x7fa57c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:13.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57b400640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 0x7fa57c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa57aa00640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 0x7fa57c19a9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa568006010 tx=0x7fa568005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568043020 con 0x7fa57c105430 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5680043d0 con 0x7fa57c105430 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c1a2330 con 0x7fa57c105430 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.449+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568003b80 con 0x7fa57c105430 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.450+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c1a2880 con 0x7fa57c105430 2024-04-13T05:14:13.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.451+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c10b680 con 0x7fa57c105430 2024-04-13T05:14:13.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.454+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa568046020 con 0x7fa57c105430 2024-04-13T05:14:13.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.456+0000 7fa573e00640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5600a3bb0 0x7fa5600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:13.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.456+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5680f2340 con 0x7fa57c105430 2024-04-13T05:14:13.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.456+0000 7fa57b400640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5600a3bb0 0x7fa5600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:13.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.457+0000 7fa57b400640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5600a3bb0 0x7fa5600a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa57c105290 tx=0x7fa56c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:13.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.461+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa56804d020 con 0x7fa57c105430 2024-04-13T05:14:13.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.719+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa57c062bc0 con 0x7fa57c105430 2024-04-13T05:14:13.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.720+0000 7fa573e00640 1 -- 172.21.15.57:0/791663485 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3037 (secure 0 0 0) 0x7fa5680bf120 con 0x7fa57c105430 2024-04-13T05:14:13.720 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:13.720 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-13T05:14:13.721 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 4 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.722+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5600a3bb0 msgr2=0x7fa5600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.722+0000 7fa582000640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5600a3bb0 0x7fa5600a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa57c105290 tx=0x7fa56c005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 msgr2=0x7fa57c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 0x7fa57c19a9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa568006010 tx=0x7fa568005e40 comp rx=0 tx=0).stop 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 shutdown_connections 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5600a3bb0 0x7fa5600a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa57c105df0 0x7fa57c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa57c105430 0x7fa57c19a9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 --2- 172.21.15.57:0/791663485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa57c104240 0x7fa57c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:13.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 >> 172.21.15.57:0/791663485 conn(0x7fa57c0ff7b0 msgr2=0x7fa57c108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:13.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 shutdown_connections 2024-04-13T05:14:13.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:13.723+0000 7fa582000640 1 -- 172.21.15.57:0/791663485 wait complete. 2024-04-13T05:14:14.282 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:14:14 smithi057 podman[74223]: 2024-04-13 05:14:14.26904396 +0000 UTC m=+0.373771240 container start 5339d0a98846c613ac8c8f775b430a916b82a21be517e238dc3bb0be0c514cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:14:14.643 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mds[74242]: -- 172.21.15.57:0/2311632349 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595c60a7a40 con 0x5595c607a800 2024-04-13T05:14:14.643 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d[74238]: starting mds.d at 2024-04-13T05:14:14.643 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:14:14 smithi057 bash[74223]: 5339d0a98846c613ac8c8f775b430a916b82a21be517e238dc3bb0be0c514cd2 2024-04-13T05:14:14.643 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:14:14 smithi057 systemd[1]: Started Ceph mds.d for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/791663485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/791663485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:15.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:15.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/791663485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:15.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:15.744 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:15.874 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:15 smithi083 ceph-mon[35133]: Deploying daemon mds.e on smithi083 2024-04-13T05:14:15.874 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:15 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.57:6836/3282486019,v1:172.21.15.57:6837/3282486019] up:boot 2024-04-13T05:14:15.874 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:15 smithi083 ceph-mon[35133]: fsmap 4 up:standby 2024-04-13T05:14:15.874 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:15 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 -- 172.21.15.57:0/676958894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c101e30 msgr2=0x7efd0c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 --2- 172.21.15.57:0/676958894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c101e30 0x7efd0c102250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd00009900 tx=0x7efd0002bc40 comp rx=0 tx=0).stop 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 -- 172.21.15.57:0/676958894 shutdown_connections 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 --2- 172.21.15.57:0/676958894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c102f80 0x7efd0c103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 --2- 172.21.15.57:0/676958894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103940 0x7efd0c10a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 --2- 172.21.15.57:0/676958894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c101e30 0x7efd0c102250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 -- 172.21.15.57:0/676958894 >> 172.21.15.57:0/676958894 conn(0x7efd0c0fd3a0 msgr2=0x7efd0c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 -- 172.21.15.57:0/676958894 shutdown_connections 2024-04-13T05:14:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.979+0000 7efd13000640 1 -- 172.21.15.57:0/676958894 wait complete. 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.980+0000 7efd13000640 1 Processor -- start 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.980+0000 7efd13000640 1 -- start start 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd13000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c101e30 0x7efd0c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd13000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 0x7efd0c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd13000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103940 0x7efd0c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd13000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10d9b0 con 0x7efd0c102f80 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd13000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10d830 con 0x7efd0c101e30 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd13000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10db30 con 0x7efd0c103940 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd11200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 0x7efd0c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd11200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 0x7efd0c19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56150/0 (socket says 172.21.15.57:56150) 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd11200640 1 -- 172.21.15.57:0/3511541285 learned_addr learned my addr 172.21.15.57:0/3511541285 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd11c00640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c101e30 0x7efd0c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.981+0000 7efd12600640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103940 0x7efd0c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd11200640 1 -- 172.21.15.57:0/3511541285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103940 msgr2=0x7efd0c1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd11200640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103940 0x7efd0c1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd11200640 1 -- 172.21.15.57:0/3511541285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c101e30 msgr2=0x7efd0c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd11200640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c101e30 0x7efd0c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd11200640 1 -- 172.21.15.57:0/3511541285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c1a1e90 con 0x7efd0c102f80 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd11200640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 0x7efd0c19a7b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c970 tx=0x7efcf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8004220 con 0x7efd0c102f80 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf80043c0 con 0x7efd0c102f80 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c1a2180 con 0x7efd0c102f80 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800f450 con 0x7efd0c102f80 2024-04-13T05:14:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.982+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c1a26d0 con 0x7efd0c102f80 2024-04-13T05:14:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.984+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c1a2990 con 0x7efd0c102f80 2024-04-13T05:14:15.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.986+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efcf80079b0 con 0x7efd0c102f80 2024-04-13T05:14:15.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.987+0000 7efd0a000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7efcf40a3bb0 0x7efcf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:15.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.987+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efcf80cfda0 con 0x7efd0c102f80 2024-04-13T05:14:15.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.987+0000 7efd11c00640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7efcf40a3bb0 0x7efcf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:15.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.988+0000 7efd11c00640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7efcf40a3bb0 0x7efcf40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd00006010 tx=0x7efd00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:15.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:15.992+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efcf8014030 con 0x7efd0c102f80 2024-04-13T05:14:16.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:15 smithi057 ceph-mon[32331]: Deploying daemon mds.e on smithi083 2024-04-13T05:14:16.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:15 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.57:6836/3282486019,v1:172.21.15.57:6837/3282486019] up:boot 2024-04-13T05:14:16.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:15 smithi057 ceph-mon[32331]: fsmap 4 up:standby 2024-04-13T05:14:16.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-13T05:14:16.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:15 smithi189 ceph-mon[35531]: Deploying daemon mds.e on smithi083 2024-04-13T05:14:16.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:15 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.57:6836/3282486019,v1:172.21.15.57:6837/3282486019] up:boot 2024-04-13T05:14:16.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:15 smithi189 ceph-mon[35531]: fsmap 4 up:standby 2024-04-13T05:14:16.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:15 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-13T05:14:16.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.242+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd0c1a2b60 con 0x7efd0c102f80 2024-04-13T05:14:16.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.243+0000 7efd0a000640 1 -- 172.21.15.57:0/3511541285 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7efcf809cb80 con 0x7efd0c102f80 2024-04-13T05:14:16.244 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:16.244 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-13T05:14:16.244 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 5 2024-04-13T05:14:16.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.246+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7efcf40a3bb0 msgr2=0x7efcf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:16.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.246+0000 7efd13000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7efcf40a3bb0 0x7efcf40a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd00006010 tx=0x7efd00000bc0 comp rx=0 tx=0).stop 2024-04-13T05:14:16.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.246+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 msgr2=0x7efd0c19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:16.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.246+0000 7efd13000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 0x7efd0c19a7b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c970 tx=0x7efcf800ce30 comp rx=0 tx=0).stop 2024-04-13T05:14:16.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 shutdown_connections 2024-04-13T05:14:16.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7efcf40a3bb0 0x7efcf40a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:16.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103940 0x7efd0c1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:16.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd0c102f80 0x7efd0c19a7b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:16.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 --2- 172.21.15.57:0/3511541285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c101e30 0x7efd0c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:16.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 >> 172.21.15.57:0/3511541285 conn(0x7efd0c0fd3a0 msgr2=0x7efd0c0741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:16.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 shutdown_connections 2024-04-13T05:14:16.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:16.247+0000 7efd13000640 1 -- 172.21.15.57:0/3511541285 wait complete. 2024-04-13T05:14:16.750 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:16 smithi083 ceph-mon[35133]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:16.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:16 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3511541285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:17.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:16 smithi057 ceph-mon[32331]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:17.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:16 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3511541285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:17.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:16 smithi189 ceph-mon[35531]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:17.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:16 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3511541285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:18.001 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:17 smithi083 systemd[1]: Starting Ceph mds.e for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:18.270 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:18.278 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 podman[65206]: 2024-04-13 05:14:17.923107935 +0000 UTC m=+0.038061492 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:18.278 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 podman[65206]: 2024-04-13 05:14:18.020852271 +0000 UTC m=+0.135805859 container create b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-13T05:14:18.278 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 podman[65206]: 2024-04-13 05:14:18.266747619 +0000 UTC m=+0.381701214 container init b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-13T05:14:18.278 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 podman[65206]: 2024-04-13 05:14:18.276302547 +0000 UTC m=+0.391256105 container start b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-13T05:14:18.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 -- 172.21.15.57:0/1326701032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b40749a0 msgr2=0x7f05b4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 --2- 172.21.15.57:0/1326701032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b40749a0 0x7f05b4074e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f059c009900 tx=0x7f059c02bcc0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 -- 172.21.15.57:0/1326701032 shutdown_connections 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 --2- 172.21.15.57:0/1326701032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b4109d10 0x7f05b410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 --2- 172.21.15.57:0/1326701032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b40749a0 0x7f05b4074e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 --2- 172.21.15.57:0/1326701032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05b4075f80 0x7f05b40743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 -- 172.21.15.57:0/1326701032 >> 172.21.15.57:0/1326701032 conn(0x7f05b40fd3d0 msgr2=0x7f05b40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 -- 172.21.15.57:0/1326701032 shutdown_connections 2024-04-13T05:14:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.505+0000 7f05b8e00640 1 -- 172.21.15.57:0/1326701032 wait complete. 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.506+0000 7f05b8e00640 1 Processor -- start 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.506+0000 7f05b8e00640 1 -- start start 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05b40749a0 0x7f05b419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b8e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 0x7f05b419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b4109d10 0x7f05b41a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b8e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f05b410fc00 con 0x7f05b4075f80 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b8e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05b410fa80 con 0x7f05b40749a0 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b8e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05b410fd80 con 0x7f05b4109d10 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 0x7f05b419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 0x7f05b419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56164/0 (socket says 172.21.15.57:56164) 2024-04-13T05:14:18.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 -- 172.21.15.57:0/873517443 learned_addr learned my addr 172.21.15.57:0/873517443 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b3400640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05b40749a0 0x7f05b419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05b3e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b4109d10 0x7f05b41a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 -- 172.21.15.57:0/873517443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b4109d10 msgr2=0x7f05b41a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b4109d10 0x7f05b41a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 -- 172.21.15.57:0/873517443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05b40749a0 msgr2=0x7f05b419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05b40749a0 0x7f05b419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.507+0000 7f05aaa00640 1 -- 172.21.15.57:0/873517443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b41a1ea0 con 0x7f05b4075f80 2024-04-13T05:14:18.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.508+0000 7f05aaa00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 0x7f05b419a7c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f059c000c00 tx=0x7f059c02dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:18.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.508+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c043020 con 0x7f05b4075f80 2024-04-13T05:14:18.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.508+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f059c02de20 con 0x7f05b4075f80 2024-04-13T05:14:18.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.508+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c003c30 con 0x7f05b4075f80 2024-04-13T05:14:18.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.508+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b41a2130 con 0x7f05b4075f80 2024-04-13T05:14:18.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.508+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b41a2630 con 0x7f05b4075f80 2024-04-13T05:14:18.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.509+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b4108ad0 con 0x7f05b4075f80 2024-04-13T05:14:18.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.511+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f059c046020 con 0x7f05b4075f80 2024-04-13T05:14:18.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.513+0000 7f05b0c00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05980a3c30 0x7f05980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:18.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.513+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f059c0f2370 con 0x7f05b4075f80 2024-04-13T05:14:18.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.513+0000 7f05b3400640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05980a3c30 0x7f05980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:18.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.514+0000 7f05b3400640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05980a3c30 0x7f05980a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05a40058c0 tx=0x7f05a400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:18.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.517+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f059c04d020 con 0x7f05b4075f80 2024-04-13T05:14:18.680 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mds[65226]: -- 172.21.15.83:0/1600204060 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565198215a40 con 0x5651981e9000 2024-04-13T05:14:18.680 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e[65222]: starting mds.e at 2024-04-13T05:14:18.680 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 bash[65206]: b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 2024-04-13T05:14:18.681 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:14:18 smithi083 systemd[1]: Started Ceph mds.e for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:18.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.773+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05b402d5e0 con 0x7f05b4075f80 2024-04-13T05:14:18.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.775+0000 7f05b0c00640 1 -- 172.21.15.57:0/873517443 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7f059c0bf150 con 0x7f05b4075f80 2024-04-13T05:14:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:18.776 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-13T05:14:18.777 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.777+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05980a3c30 msgr2=0x7f05980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.777+0000 7f05b8e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05980a3c30 0x7f05980a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05a40058c0 tx=0x7f05a400a4c0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.777+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 msgr2=0x7f05b419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.777+0000 7f05b8e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 0x7f05b419a7c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f059c000c00 tx=0x7f059c02dc30 comp rx=0 tx=0).stop 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 shutdown_connections 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05980a3c30 0x7f05980a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b4109d10 0x7f05b41a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05b4075f80 0x7f05b419a7c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 --2- 172.21.15.57:0/873517443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05b40749a0 0x7f05b419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 >> 172.21.15.57:0/873517443 conn(0x7f05b40fd3d0 msgr2=0x7f05b4105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 shutdown_connections 2024-04-13T05:14:18.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:18.778+0000 7f05b8e00640 1 -- 172.21.15.57:0/873517443 wait complete. 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:19.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:18 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:19.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:18 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:19.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:19.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:19.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:19.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:19.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:18 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:20.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:19 smithi083 ceph-mon[35133]: Deploying daemon mds.f on smithi189 2024-04-13T05:14:20.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:19 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] up:boot 2024-04-13T05:14:20.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:19 smithi083 ceph-mon[35133]: fsmap 5 up:standby 2024-04-13T05:14:20.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:19 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-13T05:14:20.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:19 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/873517443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:20.012 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:19 smithi189 ceph-mon[35531]: Deploying daemon mds.f on smithi189 2024-04-13T05:14:20.012 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:19 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] up:boot 2024-04-13T05:14:20.012 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:19 smithi189 ceph-mon[35531]: fsmap 5 up:standby 2024-04-13T05:14:20.012 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:19 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-13T05:14:20.012 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:19 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/873517443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:20.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:19 smithi057 ceph-mon[32331]: Deploying daemon mds.f on smithi189 2024-04-13T05:14:20.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:19 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] up:boot 2024-04-13T05:14:20.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:19 smithi057 ceph-mon[32331]: fsmap 5 up:standby 2024-04-13T05:14:20.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:19 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-13T05:14:20.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:19 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/873517443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:20.797 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:20.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:20 smithi189 ceph-mon[35531]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:21.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:20 smithi083 ceph-mon[35133]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:21.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:20 smithi057 ceph-mon[32331]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 -- 172.21.15.57:0/2640804859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 msgr2=0x7f3c3c110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 --2- 172.21.15.57:0/2640804859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c110880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009900 tx=0x7f3c3402bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 -- 172.21.15.57:0/2640804859 shutdown_connections 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 --2- 172.21.15.57:0/2640804859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c110880 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 --2- 172.21.15.57:0/2640804859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c3c109630 0x7f3c3c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 --2- 172.21.15.57:0/2640804859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 0x7f3c3c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 -- 172.21.15.57:0/2640804859 >> 172.21.15.57:0/2640804859 conn(0x7f3c3c077400 msgr2=0x7f3c3c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.033+0000 7f3c42000640 1 -- 172.21.15.57:0/2640804859 shutdown_connections 2024-04-13T05:14:21.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.034+0000 7f3c42000640 1 -- 172.21.15.57:0/2640804859 wait complete. 2024-04-13T05:14:21.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.034+0000 7f3c42000640 1 Processor -- start 2024-04-13T05:14:21.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 -- start start 2024-04-13T05:14:21.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 0x7f3c3c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:21.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c3c109630 0x7f3c3c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c1a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c113fd0 con 0x7f3c3c109ff0 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c113e50 con 0x7f3c3c108440 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c42000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c114150 con 0x7f3c3c109630 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c41600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c1a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c41600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c1a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56188/0 (socket says 172.21.15.57:56188) 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.035+0000 7f3c41600640 1 -- 172.21.15.57:0/3035733258 learned_addr learned my addr 172.21.15.57:0/3035733258 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c40c00640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 0x7f3c3c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:21.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c33e00640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c3c109630 0x7f3c3c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c41600640 1 -- 172.21.15.57:0/3035733258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c3c109630 msgr2=0x7f3c3c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c41600640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c3c109630 0x7f3c3c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c41600640 1 -- 172.21.15.57:0/3035733258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 msgr2=0x7f3c3c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c41600640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 0x7f3c3c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c41600640 1 -- 172.21.15.57:0/3035733258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c1a6240 con 0x7f3c3c109ff0 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c40c00640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 0x7f3c3c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:21.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.036+0000 7f3c41600640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c1a5bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3c34036040 tx=0x7f3c34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:21.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.037+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34043020 con 0x7f3c3c109ff0 2024-04-13T05:14:21.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.037+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c1a64d0 con 0x7f3c3c109ff0 2024-04-13T05:14:21.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.037+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c34004490 con 0x7f3c3c109ff0 2024-04-13T05:14:21.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.037+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34003d10 con 0x7f3c3c109ff0 2024-04-13T05:14:21.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.037+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1a6a70 con 0x7f3c3c109ff0 2024-04-13T05:14:21.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.038+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3c10f760 con 0x7f3c3c109ff0 2024-04-13T05:14:21.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.040+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c34046020 con 0x7f3c3c109ff0 2024-04-13T05:14:21.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.041+0000 7f3c3a000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3c200a3bb0 0x7f3c200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:21.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.041+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3c340f2480 con 0x7f3c3c109ff0 2024-04-13T05:14:21.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.042+0000 7f3c40c00640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3c200a3bb0 0x7f3c200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:21.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.042+0000 7f3c40c00640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3c200a3bb0 0x7f3c200a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c109490 tx=0x7f3c2c009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:21.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.046+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c3404d020 con 0x7f3c3c109ff0 2024-04-13T05:14:21.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.294+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c3c062bc0 con 0x7f3c3c109ff0 2024-04-13T05:14:21.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.295+0000 7f3c3a000640 1 -- 172.21.15.57:0/3035733258 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7f3c340bf260 con 0x7f3c3c109ff0 2024-04-13T05:14:21.296 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:21.297 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-13T05:14:21.297 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2024-04-13T05:14:21.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.297+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3c200a3bb0 msgr2=0x7f3c200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:21.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.297+0000 7f3c42000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3c200a3bb0 0x7f3c200a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c109490 tx=0x7f3c2c009540 comp rx=0 tx=0).stop 2024-04-13T05:14:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.297+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 msgr2=0x7f3c3c1a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.297+0000 7f3c42000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c1a5bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3c34036040 tx=0x7f3c34004270 comp rx=0 tx=0).stop 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 shutdown_connections 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3c200a3bb0 0x7f3c200a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c3c109ff0 0x7f3c3c1a5bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c3c109630 0x7f3c3c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 --2- 172.21.15.57:0/3035733258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c3c108440 0x7f3c3c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 >> 172.21.15.57:0/3035733258 conn(0x7f3c3c077400 msgr2=0x7f3c3c10c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 shutdown_connections 2024-04-13T05:14:21.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:21.298+0000 7f3c42000640 1 -- 172.21.15.57:0/3035733258 wait complete. 2024-04-13T05:14:22.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:21 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3035733258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:22.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:21 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3035733258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:22.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:21 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3035733258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:22.083 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:21 smithi189 systemd[1]: Starting Ceph mds.f for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:22.394 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 podman[65585]: 2024-04-13 05:14:21.997516747 +0000 UTC m=+0.033704107 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:22.394 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 podman[65585]: 2024-04-13 05:14:22.119234014 +0000 UTC m=+0.155421388 container create 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:14:22.394 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 podman[65585]: 2024-04-13 05:14:22.389957015 +0000 UTC m=+0.426144433 container init 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:14:22.727 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 podman[65585]: 2024-04-13 05:14:22.400548348 +0000 UTC m=+0.436735728 container start 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True) 2024-04-13T05:14:22.727 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 ceph-mds[65605]: -- 172.21.15.189:0/3477438570 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55baef671a40 con 0x55baef644c00 2024-04-13T05:14:22.727 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f[65600]: starting mds.f at 2024-04-13T05:14:22.727 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 bash[65585]: 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 2024-04-13T05:14:22.727 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:14:22 smithi189 systemd[1]: Started Ceph mds.f for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:23.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:22 smithi083 ceph-mon[35133]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:23.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:22 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:23.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:22 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:23.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:22 smithi057 ceph-mon[32331]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:23.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:22 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:23.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:22 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:23.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:22 smithi189 ceph-mon[35531]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:23.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:22 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:23.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:22 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:23.319 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:23.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 -- 172.21.15.57:0/296517610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98103940 msgr2=0x7f9b9810a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:23.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 --2- 172.21.15.57:0/296517610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98103940 0x7f9b9810a260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009ca0 tx=0x7f9b9002c100 comp rx=0 tx=0).stop 2024-04-13T05:14:23.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 -- 172.21.15.57:0/296517610 shutdown_connections 2024-04-13T05:14:23.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 --2- 172.21.15.57:0/296517610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98103940 0x7f9b9810a260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 --2- 172.21.15.57:0/296517610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b98102f80 0x7f9b98103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 --2- 172.21.15.57:0/296517610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b98101e30 0x7f9b98102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 -- 172.21.15.57:0/296517610 >> 172.21.15.57:0/296517610 conn(0x7f9b980fd3a0 msgr2=0x7f9b980ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:23.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 -- 172.21.15.57:0/296517610 shutdown_connections 2024-04-13T05:14:23.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.554+0000 7f9b9d800640 1 -- 172.21.15.57:0/296517610 wait complete. 2024-04-13T05:14:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.555+0000 7f9b9d800640 1 Processor -- start 2024-04-13T05:14:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 -- start start 2024-04-13T05:14:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 0x7f9b9819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:23.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b98102f80 0x7f9b9819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b98103940 0x7f9b981a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9810d990 con 0x7f9b98101e30 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9810d810 con 0x7f9b98103940 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b9d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9810db10 con 0x7f9b98102f80 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b97e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 0x7f9b9819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b97e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 0x7f9b9819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56214/0 (socket says 172.21.15.57:56214) 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b97e00640 1 -- 172.21.15.57:0/3092598316 learned_addr learned my addr 172.21.15.57:0/3092598316 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:23.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.556+0000 7f9b97400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b98102f80 0x7f9b9819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b9ce00640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b98103940 0x7f9b981a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b97e00640 1 -- 172.21.15.57:0/3092598316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b98102f80 msgr2=0x7f9b9819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b97e00640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b98102f80 0x7f9b9819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b97e00640 1 -- 172.21.15.57:0/3092598316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b98103940 msgr2=0x7f9b981a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b97e00640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b98103940 0x7f9b981a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b97e00640 1 -- 172.21.15.57:0/3092598316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b981a1e90 con 0x7f9b98101e30 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b97e00640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 0x7f9b9819a270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800d950 tx=0x7f9b8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88017070 con 0x7f9b98101e30 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b88009c30 con 0x7f9b98101e30 2024-04-13T05:14:23.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800f620 con 0x7f9b98101e30 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b981a2180 con 0x7f9b98101e30 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.557+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b981a2680 con 0x7f9b98101e30 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.558+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b981a28e0 con 0x7f9b98101e30 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.560+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b880109a0 con 0x7f9b98101e30 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.562+0000 7f9b94c00640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b7c0a3c30 0x7f9b7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.562+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9b880cfbb0 con 0x7f9b98101e30 2024-04-13T05:14:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.562+0000 7f9b97400640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b7c0a3c30 0x7f9b7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:23.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.563+0000 7f9b97400640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b7c0a3c30 0x7f9b7c0a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b9819b780 tx=0x7f9b84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:23.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.567+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b88009de0 con 0x7f9b98101e30 2024-04-13T05:14:23.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.824+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b981a2ab0 con 0x7f9b98101e30 2024-04-13T05:14:23.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.825+0000 7f9b94c00640 1 -- 172.21.15.57:0/3092598316 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f9b8809c990 con 0x7f9b98101e30 2024-04-13T05:14:23.827 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:23.827 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-13T05:14:23.827 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 7 2024-04-13T05:14:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b7c0a3c30 msgr2=0x7f9b7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b7c0a3c30 0x7f9b7c0a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b9819b780 tx=0x7f9b84005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:23.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 msgr2=0x7f9b9819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 0x7f9b9819a270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800d950 tx=0x7f9b8800de10 comp rx=0 tx=0).stop 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 shutdown_connections 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9b7c0a3c30 0x7f9b7c0a60f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b98103940 0x7f9b981a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b98102f80 0x7f9b9819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 --2- 172.21.15.57:0/3092598316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b98101e30 0x7f9b9819a270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 >> 172.21.15.57:0/3092598316 conn(0x7f9b980fd3a0 msgr2=0x7f9b980741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 shutdown_connections 2024-04-13T05:14:23.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:23.828+0000 7f9b9d800640 1 -- 172.21.15.57:0/3092598316 wait complete. 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.189:6834/3836752416,v1:172.21.15.189:6835/3836752416] up:boot 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: fsmap 6 up:standby 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:24.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:23 smithi083 ceph-mon[35133]: Deploying daemon mds.c on smithi189 2024-04-13T05:14:24.027 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:24.027 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:24.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.189:6834/3836752416,v1:172.21.15.189:6835/3836752416] up:boot 2024-04-13T05:14:24.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: fsmap 6 up:standby 2024-04-13T05:14:24.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-13T05:14:24.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:24.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:24.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:23 smithi189 ceph-mon[35531]: Deploying daemon mds.c on smithi189 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.189:6834/3836752416,v1:172.21.15.189:6835/3836752416] up:boot 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: fsmap 6 up:standby 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:23 smithi057 ceph-mon[32331]: Deploying daemon mds.c on smithi189 2024-04-13T05:14:24.972 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:24 smithi189 ceph-mon[35531]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:24.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:24 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3092598316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:25.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:24 smithi083 ceph-mon[35133]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:25.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:24 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3092598316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:25.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:24 smithi057 ceph-mon[32331]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:25.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:24 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3092598316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:25.849 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:26.082 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:25 smithi189 systemd[1]: Starting Ceph mds.c for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 -- 172.21.15.57:0/1719260973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105230 msgr2=0x7f89ec1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 --2- 172.21.15.57:0/1719260973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105230 0x7f89ec1056b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009900 tx=0x7f89d802bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 -- 172.21.15.57:0/1719260973 shutdown_connections 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 --2- 172.21.15.57:0/1719260973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89ec105bf0 0x7f89ec10c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 --2- 172.21.15.57:0/1719260973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105230 0x7f89ec1056b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 --2- 172.21.15.57:0/1719260973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 0x7f89ec104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 -- 172.21.15.57:0/1719260973 >> 172.21.15.57:0/1719260973 conn(0x7f89ec0ff5d0 msgr2=0x7f89ec1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 -- 172.21.15.57:0/1719260973 shutdown_connections 2024-04-13T05:14:26.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.083+0000 7f89f1200640 1 -- 172.21.15.57:0/1719260973 wait complete. 2024-04-13T05:14:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.084+0000 7f89f1200640 1 Processor -- start 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.084+0000 7f89f1200640 1 -- start start 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.084+0000 7f89f1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 0x7f89ec19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.084+0000 7f89f1200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89ec105230 0x7f89ec19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89f1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 0x7f89ec1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89f1200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec10fbd0 con 0x7f89ec105bf0 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89f1200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec10fa50 con 0x7f89ec104040 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89f1200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec10fd50 con 0x7f89ec105230 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 0x7f89ec1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 0x7f89ec1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35344/0 (socket says 172.21.15.57:35344) 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 -- 172.21.15.57:0/1852283708 learned_addr learned my addr 172.21.15.57:0/1852283708 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89eaa00640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89ec105230 0x7f89ec19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89eb400640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 0x7f89ec19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 -- 172.21.15.57:0/1852283708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89ec105230 msgr2=0x7f89ec19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89ec105230 0x7f89ec19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 -- 172.21.15.57:0/1852283708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 msgr2=0x7f89ec19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 0x7f89ec19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89ebe00640 1 -- 172.21.15.57:0/1852283708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec1a1df0 con 0x7f89ec105bf0 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.085+0000 7f89eb400640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 0x7f89ec19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.086+0000 7f89ebe00640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 0x7f89ec1a1780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89e400cce0 tx=0x7f89e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.086+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e400f450 con 0x7f89ec105bf0 2024-04-13T05:14:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.086+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89e400faa0 con 0x7f89ec105bf0 2024-04-13T05:14:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.086+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec1a20e0 con 0x7f89ec105bf0 2024-04-13T05:14:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.086+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4018b70 con 0x7f89ec105bf0 2024-04-13T05:14:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.086+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec1a25b0 con 0x7f89ec105bf0 2024-04-13T05:14:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.087+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec10b360 con 0x7f89ec105bf0 2024-04-13T05:14:26.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.089+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f89e4018d10 con 0x7f89ec105bf0 2024-04-13T05:14:26.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.090+0000 7f89e3e00640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f89d00a3bb0 0x7f89d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:26.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.091+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f89e40d0430 con 0x7f89ec105bf0 2024-04-13T05:14:26.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.091+0000 7f89eb400640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f89d00a3bb0 0x7f89d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:26.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.091+0000 7f89eb400640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f89d00a3bb0 0x7f89d00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f89ec105090 tx=0x7f89dc005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:26.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.095+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89e4014030 con 0x7f89ec105bf0 2024-04-13T05:14:26.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.337+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89ec02d5e0 con 0x7f89ec105bf0 2024-04-13T05:14:26.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.338+0000 7f89e3e00640 1 -- 172.21.15.57:0/1852283708 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f89e409d210 con 0x7f89ec105bf0 2024-04-13T05:14:26.339 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:26.339 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-13T05:14:26.339 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 7 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f89d00a3bb0 msgr2=0x7f89d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f89d00a3bb0 0x7f89d00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f89ec105090 tx=0x7f89dc005d30 comp rx=0 tx=0).stop 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 msgr2=0x7f89ec1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 0x7f89ec1a1780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89e400cce0 tx=0x7f89e400a620 comp rx=0 tx=0).stop 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 shutdown_connections 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f89d00a3bb0 0x7f89d00a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89ec105bf0 0x7f89ec1a1780 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89ec105230 0x7f89ec19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 --2- 172.21.15.57:0/1852283708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89ec104040 0x7f89ec19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 >> 172.21.15.57:0/1852283708 conn(0x7f89ec0ff5d0 msgr2=0x7f89ec108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.340+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 shutdown_connections 2024-04-13T05:14:26.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:26.341+0000 7f89f1200640 1 -- 172.21.15.57:0/1852283708 wait complete. 2024-04-13T05:14:26.539 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 podman[65900]: 2024-04-13 05:14:26.084622532 +0000 UTC m=+0.039417488 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:26.540 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 podman[65900]: 2024-04-13 05:14:26.206934878 +0000 UTC m=+0.161729858 container create adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-13T05:14:26.540 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 podman[65900]: 2024-04-13 05:14:26.535942402 +0000 UTC m=+0.490737392 container init adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-13T05:14:26.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:26 smithi189 ceph-mon[35531]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:26.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:26 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1852283708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:26.833 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 podman[65900]: 2024-04-13 05:14:26.547598567 +0000 UTC m=+0.502393554 container start adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-13T05:14:26.833 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 ceph-mds[65919]: -- 172.21.15.189:0/913993731 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f629f6fa40 con 0x55f629f43000 2024-04-13T05:14:26.833 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c[65915]: starting mds.c at 2024-04-13T05:14:26.833 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 bash[65900]: adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 2024-04-13T05:14:26.833 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:14:26 smithi189 systemd[1]: Started Ceph mds.c for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:27.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:26 smithi057 ceph-mon[32331]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:27.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:26 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1852283708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:26 smithi083 ceph-mon[35133]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:27.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:26 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1852283708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:28.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.189:6836/563533422,v1:172.21.15.189:6837/563533422] up:boot 2024-04-13T05:14:28.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: fsmap 7 up:standby 2024-04-13T05:14:28.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-13T05:14:28.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:28.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:28.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:28.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:27 smithi057 ceph-mon[32331]: Deploying daemon mds.h on smithi083 2024-04-13T05:14:28.049 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.189:6836/563533422,v1:172.21.15.189:6837/563533422] up:boot 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: fsmap 7 up:standby 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:28.050 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:27 smithi083 ceph-mon[35133]: Deploying daemon mds.h on smithi083 2024-04-13T05:14:28.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.189:6836/563533422,v1:172.21.15.189:6837/563533422] up:boot 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: fsmap 7 up:standby 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:28.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:27 smithi189 ceph-mon[35531]: Deploying daemon mds.h on smithi083 2024-04-13T05:14:28.361 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:28.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 -- 172.21.15.57:0/4187574972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628101e60 msgr2=0x7fb628102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 --2- 172.21.15.57:0/4187574972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628101e60 0x7fb628102280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c02bc40 comp rx=0 tx=0).stop 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 -- 172.21.15.57:0/4187574972 shutdown_connections 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 --2- 172.21.15.57:0/4187574972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb628102fb0 0x7fb628103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 --2- 172.21.15.57:0/4187574972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628103970 0x7fb62810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 --2- 172.21.15.57:0/4187574972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628101e60 0x7fb628102280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 -- 172.21.15.57:0/4187574972 >> 172.21.15.57:0/4187574972 conn(0x7fb6280fd3d0 msgr2=0x7fb6280ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 -- 172.21.15.57:0/4187574972 shutdown_connections 2024-04-13T05:14:28.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.595+0000 7fb62e600640 1 -- 172.21.15.57:0/4187574972 wait complete. 2024-04-13T05:14:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.596+0000 7fb62e600640 1 Processor -- start 2024-04-13T05:14:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 -- start start 2024-04-13T05:14:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628101e60 0x7fb6281980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb628102fb0 0x7fb6281985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 0x7fb62819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb62810d9b0 con 0x7fb628103970 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb62810d830 con 0x7fb628102fb0 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.597+0000 7fb62e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb62810db30 con 0x7fb628101e60 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb62dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 0x7fb62819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb62d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628101e60 0x7fb6281980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb627e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb628102fb0 0x7fb6281985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb62dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 0x7fb62819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35366/0 (socket says 172.21.15.57:35366) 2024-04-13T05:14:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb62dc00640 1 -- 172.21.15.57:0/3435624684 learned_addr learned my addr 172.21.15.57:0/3435624684 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:28.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb62d200640 1 -- 172.21.15.57:0/3435624684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb628102fb0 msgr2=0x7fb6281985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:28.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.598+0000 7fb62d200640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb628102fb0 0x7fb6281985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.599+0000 7fb62d200640 1 -- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 msgr2=0x7fb62819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.599+0000 7fb62d200640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 0x7fb62819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.599+0000 7fb62d200640 1 -- 172.21.15.57:0/3435624684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62819fc80 con 0x7fb628101e60 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.599+0000 7fb62dc00640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 0x7fb62819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.599+0000 7fb62d200640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628101e60 0x7fb6281980b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000c00 tx=0x7fb61c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.599+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c043020 con 0x7fb628101e60 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.600+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62819ff10 con 0x7fb628101e60 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.600+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb61c004aa0 con 0x7fb628101e60 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.600+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003cd0 con 0x7fb628101e60 2024-04-13T05:14:28.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.600+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6281a03d0 con 0x7fb628101e60 2024-04-13T05:14:28.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.601+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb628109240 con 0x7fb628101e60 2024-04-13T05:14:28.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.604+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb61c046020 con 0x7fb628101e60 2024-04-13T05:14:28.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.605+0000 7fb625600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:28.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.606+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb61c0f2ac0 con 0x7fb628101e60 2024-04-13T05:14:28.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.606+0000 7fb627e00640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:28.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.606+0000 7fb627e00640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6100a3bb0 0x7fb6100a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb628199570 tx=0x7fb61400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:28.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.619+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb61c04d020 con 0x7fb628101e60 2024-04-13T05:14:28.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.872+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb62802d5e0 con 0x7fb628101e60 2024-04-13T05:14:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.874+0000 7fb625600640 1 -- 172.21.15.57:0/3435624684 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7fb61c0bf8a0 con 0x7fb628101e60 2024-04-13T05:14:28.876 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:28.876 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-13T05:14:28.876 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2024-04-13T05:14:28.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6100a3bb0 msgr2=0x7fb6100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6100a3bb0 0x7fb6100a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb628199570 tx=0x7fb61400b390 comp rx=0 tx=0).stop 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628101e60 msgr2=0x7fb6281980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628101e60 0x7fb6281980b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb61c000c00 tx=0x7fb61c004270 comp rx=0 tx=0).stop 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 shutdown_connections 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb628102fb0 0x7fb6281985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb628103970 0x7fb62819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 --2- 172.21.15.57:0/3435624684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb628101e60 0x7fb6281980b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 >> 172.21.15.57:0/3435624684 conn(0x7fb6280fd3d0 msgr2=0x7fb628106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 shutdown_connections 2024-04-13T05:14:28.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:28.877+0000 7fb62e600640 1 -- 172.21.15.57:0/3435624684 wait complete. 2024-04-13T05:14:29.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:28 smithi083 ceph-mon[35133]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:29.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:28 smithi057 ceph-mon[32331]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:29.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:28 smithi189 ceph-mon[35531]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:29.945 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:29 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3435624684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:29.945 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:29 smithi083 systemd[1]: Starting Ceph mds.h for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:30.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:29 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3435624684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:30.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:29 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3435624684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:30.559 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 podman[65508]: 2024-04-13 05:14:30.202887008 +0000 UTC m=+0.032810768 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:30.560 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 podman[65508]: 2024-04-13 05:14:30.310256094 +0000 UTC m=+0.140179875 container create 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-13T05:14:30.560 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 podman[65508]: 2024-04-13 05:14:30.556324445 +0000 UTC m=+0.386248265 container init 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:14:30.854 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 podman[65508]: 2024-04-13 05:14:30.566173728 +0000 UTC m=+0.396097511 container start 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-13T05:14:30.854 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mds[65527]: -- 172.21.15.83:0/2054106307 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b03fdba40 con 0x559b03faf000 2024-04-13T05:14:30.854 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: starting mds.h at 2024-04-13T05:14:30.854 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 bash[65508]: 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b 2024-04-13T05:14:30.854 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:14:30 smithi083 systemd[1]: Started Ceph mds.h for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:30.901 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 -- 172.21.15.57:0/1647796062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109ff0 msgr2=0x7f60d8110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 --2- 172.21.15.57:0/1647796062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109ff0 0x7f60d8110880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009900 tx=0x7f60d002bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 -- 172.21.15.57:0/1647796062 shutdown_connections 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 --2- 172.21.15.57:0/1647796062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60d8109630 0x7f60d8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 --2- 172.21.15.57:0/1647796062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109ff0 0x7f60d8110880 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 --2- 172.21.15.57:0/1647796062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8108440 0x7f60d8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 -- 172.21.15.57:0/1647796062 >> 172.21.15.57:0/1647796062 conn(0x7f60d8077400 msgr2=0x7f60d8079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.135+0000 7f60dd600640 1 -- 172.21.15.57:0/1647796062 shutdown_connections 2024-04-13T05:14:31.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.136+0000 7f60dd600640 1 -- 172.21.15.57:0/1647796062 wait complete. 2024-04-13T05:14:31.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 Processor -- start 2024-04-13T05:14:31.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 -- start start 2024-04-13T05:14:31.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60d8108440 0x7f60d819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:31.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 0x7f60d819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8109ff0 0x7f60d81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8113f80 con 0x7f60d8109630 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8113e00 con 0x7f60d8108440 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60dd600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8114100 con 0x7f60d8109ff0 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.137+0000 7f60d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 0x7f60d819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 0x7f60d819eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35388/0 (socket says 172.21.15.57:35388) 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 -- 172.21.15.57:0/2158515532 learned_addr learned my addr 172.21.15.57:0/2158515532 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60dcc00640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8109ff0 0x7f60d81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7e00640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60d8108440 0x7f60d819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 -- 172.21.15.57:0/2158515532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8109ff0 msgr2=0x7f60d81a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8109ff0 0x7f60d81a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 -- 172.21.15.57:0/2158515532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60d8108440 msgr2=0x7f60d819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60d8108440 0x7f60d819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 -- 172.21.15.57:0/2158515532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d81a6180 con 0x7f60d8109630 2024-04-13T05:14:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60dcc00640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8109ff0 0x7f60d81a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.138+0000 7f60d7400640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 0x7f60d819eaa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f60c400cce0 tx=0x7f60c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.139+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c400f650 con 0x7f60d8109630 2024-04-13T05:14:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.139+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60c400fca0 con 0x7f60d8109630 2024-04-13T05:14:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.139+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4018b90 con 0x7f60d8109630 2024-04-13T05:14:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.139+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d81a6470 con 0x7f60d8109630 2024-04-13T05:14:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.139+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d81a6970 con 0x7f60d8109630 2024-04-13T05:14:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.140+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d810f760 con 0x7f60d8109630 2024-04-13T05:14:31.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.143+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60c400f7f0 con 0x7f60d8109630 2024-04-13T05:14:31.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.144+0000 7f60d4c00640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60bc0a3bb0 0x7f60bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:31.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.144+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f60c40d0500 con 0x7f60d8109630 2024-04-13T05:14:31.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.145+0000 7f60d7e00640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60bc0a3bb0 0x7f60bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:31.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.145+0000 7f60d7e00640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60bc0a3bb0 0x7f60bc0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005d00 tx=0x7f60c8005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:31.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.149+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60c4014030 con 0x7f60d8109630 2024-04-13T05:14:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:31.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:31.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:30 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:31.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:30 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:30 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:31.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.399+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d802d5e0 con 0x7f60d8109630 2024-04-13T05:14:31.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.400+0000 7f60d4c00640 1 -- 172.21.15.57:0/2158515532 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7f60c409d2e0 con 0x7f60d8109630 2024-04-13T05:14:31.402 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:31.402 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:31.403 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 9 2024-04-13T05:14:31.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60bc0a3bb0 msgr2=0x7f60bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:31.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60bc0a3bb0 0x7f60bc0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005d00 tx=0x7f60c8005c50 comp rx=0 tx=0).stop 2024-04-13T05:14:31.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 msgr2=0x7f60d819eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:31.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 0x7f60d819eaa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f60c400cce0 tx=0x7f60c400a620 comp rx=0 tx=0).stop 2024-04-13T05:14:31.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 shutdown_connections 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60bc0a3bb0 0x7f60bc0a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d8109ff0 0x7f60d81a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60d8109630 0x7f60d819eaa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 --2- 172.21.15.57:0/2158515532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60d8108440 0x7f60d819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 >> 172.21.15.57:0/2158515532 conn(0x7f60d8077400 msgr2=0x7f60d810c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 shutdown_connections 2024-04-13T05:14:31.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:31.403+0000 7f60dd600640 1 -- 172.21.15.57:0/2158515532 wait complete. 2024-04-13T05:14:32.198 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:31 smithi189 ceph-mon[35531]: Deploying daemon mds.i on smithi189 2024-04-13T05:14:32.199 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:31 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] up:boot 2024-04-13T05:14:32.199 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:31 smithi189 ceph-mon[35531]: fsmap 8 up:standby 2024-04-13T05:14:32.199 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:31 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-13T05:14:32.199 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:31 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2158515532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:31 smithi083 ceph-mon[35133]: Deploying daemon mds.i on smithi189 2024-04-13T05:14:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:31 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] up:boot 2024-04-13T05:14:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:31 smithi083 ceph-mon[35133]: fsmap 8 up:standby 2024-04-13T05:14:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:31 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-13T05:14:32.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:31 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2158515532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:31 smithi057 ceph-mon[32331]: Deploying daemon mds.i on smithi189 2024-04-13T05:14:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:31 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] up:boot 2024-04-13T05:14:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:31 smithi057 ceph-mon[32331]: fsmap 8 up:standby 2024-04-13T05:14:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:31 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-13T05:14:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:31 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2158515532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:33.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:32 smithi189 ceph-mon[35531]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:33.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:32 smithi083 ceph-mon[35133]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:33.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:32 smithi057 ceph-mon[32331]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:33.426 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:33.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 -- 172.21.15.57:0/2106577221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c077bc0 msgr2=0x7fd04c078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:33.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 --2- 172.21.15.57:0/2106577221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c077bc0 0x7fd04c078020 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd034009900 tx=0x7fd03402bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 -- 172.21.15.57:0/2106577221 shutdown_connections 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 --2- 172.21.15.57:0/2106577221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c078560 0x7fd04c078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 --2- 172.21.15.57:0/2106577221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c077bc0 0x7fd04c078020 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 --2- 172.21.15.57:0/2106577221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd04c076980 0x7fd04c076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 -- 172.21.15.57:0/2106577221 >> 172.21.15.57:0/2106577221 conn(0x7fd04c0ff7b0 msgr2=0x7fd04c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 -- 172.21.15.57:0/2106577221 shutdown_connections 2024-04-13T05:14:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.661+0000 7fd052c00640 1 -- 172.21.15.57:0/2106577221 wait complete. 2024-04-13T05:14:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 Processor -- start 2024-04-13T05:14:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 -- start start 2024-04-13T05:14:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c076980 0x7fd04c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd04c077bc0 0x7fd04c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 0x7fd04c1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:33.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1141c0 con 0x7fd04c078560 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c114040 con 0x7fd04c077bc0 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.662+0000 7fd052c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c114340 con 0x7fd04c076980 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 0x7fd04c1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 0x7fd04c1a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35414/0 (socket says 172.21.15.57:35414) 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 -- 172.21.15.57:0/1217913274 learned_addr learned my addr 172.21.15.57:0/1217913274 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd043400640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd04c077bc0 0x7fd04c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd04be00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c076980 0x7fd04c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 -- 172.21.15.57:0/1217913274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c076980 msgr2=0x7fd04c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c076980 0x7fd04c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 -- 172.21.15.57:0/1217913274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd04c077bc0 msgr2=0x7fd04c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd04c077bc0 0x7fd04c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 -- 172.21.15.57:0/1217913274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1a6460 con 0x7fd04c078560 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd04be00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c076980 0x7fd04c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:33.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.663+0000 7fd050c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 0x7fd04c1a5df0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd04400eaa0 tx=0x7fd04400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:33.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.664+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400cdb0 con 0x7fd04c078560 2024-04-13T05:14:33.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.664+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd044005020 con 0x7fd04c078560 2024-04-13T05:14:33.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.664+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c1a6750 con 0x7fd04c078560 2024-04-13T05:14:33.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.664+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440116d0 con 0x7fd04c078560 2024-04-13T05:14:33.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.664+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a6cf0 con 0x7fd04c078560 2024-04-13T05:14:33.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.665+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c110d90 con 0x7fd04c078560 2024-04-13T05:14:33.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.667+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0440051d0 con 0x7fd04c078560 2024-04-13T05:14:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.668+0000 7fd049600640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd0300a3c30 0x7fd0300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.669+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0440d01f0 con 0x7fd04c078560 2024-04-13T05:14:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.669+0000 7fd04be00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd0300a3c30 0x7fd0300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.669+0000 7fd04be00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd0300a3c30 0x7fd0300a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009b40 tx=0x7fd03c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:33.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.673+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd044016030 con 0x7fd04c078560 2024-04-13T05:14:33.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.920+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c062bc0 con 0x7fd04c078560 2024-04-13T05:14:33.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.921+0000 7fd049600640 1 -- 172.21.15.57:0/1217913274 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7fd04409cfd0 con 0x7fd04c078560 2024-04-13T05:14:33.923 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:33.923 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:33.923 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 9 2024-04-13T05:14:33.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd0300a3c30 msgr2=0x7fd0300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:33.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd0300a3c30 0x7fd0300a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009b40 tx=0x7fd03c005e50 comp rx=0 tx=0).stop 2024-04-13T05:14:33.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 msgr2=0x7fd04c1a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:33.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 0x7fd04c1a5df0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd04400eaa0 tx=0x7fd04400ef60 comp rx=0 tx=0).stop 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 shutdown_connections 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd0300a3c30 0x7fd0300a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd04c077bc0 0x7fd04c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd04c078560 0x7fd04c1a5df0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 --2- 172.21.15.57:0/1217913274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd04c076980 0x7fd04c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 >> 172.21.15.57:0/1217913274 conn(0x7fd04c0ff7b0 msgr2=0x7fd04c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 shutdown_connections 2024-04-13T05:14:33.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:33.924+0000 7fd052c00640 1 -- 172.21.15.57:0/1217913274 wait complete. 2024-04-13T05:14:34.289 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:33 smithi189 systemd[1]: Starting Ceph mds.i for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:34.583 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 podman[66208]: 2024-04-13 05:14:34.191640406 +0000 UTC m=+0.031757062 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:34.583 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 podman[66208]: 2024-04-13 05:14:34.335812237 +0000 UTC m=+0.175928929 container create 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-13T05:14:34.889 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 podman[66208]: 2024-04-13 05:14:34.615001497 +0000 UTC m=+0.455118196 container init 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:14:34.889 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 podman[66208]: 2024-04-13 05:14:34.625643028 +0000 UTC m=+0.465759690 container start 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-13T05:14:34.889 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 ceph-mds[66227]: -- 172.21.15.189:0/2625650528 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a60893a40 con 0x559a60866800 2024-04-13T05:14:34.889 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i[66223]: starting mds.i at 2024-04-13T05:14:34.889 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 bash[66208]: 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf 2024-04-13T05:14:34.889 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:14:34 smithi189 systemd[1]: Started Ceph mds.i for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:34 smithi083 ceph-mon[35133]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:34 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1217913274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:34 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:34 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:35.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:34 smithi057 ceph-mon[32331]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:34 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1217913274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:34 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:35.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:34 smithi189 ceph-mon[35531]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:35.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:34 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1217913274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:35.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:34 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:35.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:34 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:35.948 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:36.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 -- 172.21.15.57:0/2532438354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908077bc0 msgr2=0x7f3908078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 --2- 172.21.15.57:0/2532438354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908077bc0 0x7f3908078020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 -- 172.21.15.57:0/2532438354 shutdown_connections 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 --2- 172.21.15.57:0/2532438354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908078560 0x7f3908078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 --2- 172.21.15.57:0/2532438354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908077bc0 0x7f3908078020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 --2- 172.21.15.57:0/2532438354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908076980 0x7f3908076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 -- 172.21.15.57:0/2532438354 >> 172.21.15.57:0/2532438354 conn(0x7f39080ff7b0 msgr2=0x7f3908101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 -- 172.21.15.57:0/2532438354 shutdown_connections 2024-04-13T05:14:36.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.175+0000 7f390da00640 1 -- 172.21.15.57:0/2532438354 wait complete. 2024-04-13T05:14:36.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.176+0000 7f390da00640 1 Processor -- start 2024-04-13T05:14:36.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.176+0000 7f390da00640 1 -- start start 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.176+0000 7f390da00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 0x7f390819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.176+0000 7f390da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908077bc0 0x7f390819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f390da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908078560 0x7f39081a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f390da00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114180 con 0x7f3908076980 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f390da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114000 con 0x7f3908077bc0 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f390da00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3908114300 con 0x7f3908078560 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 0x7f390819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 0x7f390819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34954/0 (socket says 172.21.15.57:34954) 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 -- 172.21.15.57:0/266271573 learned_addr learned my addr 172.21.15.57:0/266271573 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f38fe000640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908077bc0 0x7f390819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3907400640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908078560 0x7f39081a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 -- 172.21.15.57:0/266271573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908078560 msgr2=0x7f39081a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:36.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908078560 0x7f39081a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 -- 172.21.15.57:0/266271573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908077bc0 msgr2=0x7f390819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908077bc0 0x7f390819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f3906a00640 1 -- 172.21.15.57:0/266271573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39081a63e0 con 0x7f3908076980 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.177+0000 7f38fe000640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908077bc0 0x7f390819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.178+0000 7f3906a00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 0x7f390819e7c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38f800eab0 tx=0x7f38f800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.178+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800ce40 con 0x7f3908076980 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.178+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39081a66d0 con 0x7f3908076980 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.178+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38f8011920 con 0x7f3908076980 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.178+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80109a0 con 0x7f3908076980 2024-04-13T05:14:36.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.178+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a6c20 con 0x7f3908076980 2024-04-13T05:14:36.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.189:6838/1915044018,v1:172.21.15.189:6839/1915044018] up:boot 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: fsmap 9 up:standby 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:36.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:35 smithi057 ceph-mon[32331]: Deploying daemon mds.g on smithi057 2024-04-13T05:14:36.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.180+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3908110d60 con 0x7f3908076980 2024-04-13T05:14:36.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.182+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f38f8011ad0 con 0x7f3908076980 2024-04-13T05:14:36.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.183+0000 7f38ffe00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f38ec0a3bb0 0x7f38ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:36.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.184+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f38f80d06d0 con 0x7f3908076980 2024-04-13T05:14:36.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.184+0000 7f38fe000640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f38ec0a3bb0 0x7f38ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:36.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.185+0000 7f38fe000640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f38ec0a3bb0 0x7f38ec0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38f00096c0 tx=0x7f38f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:36.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.188+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f8016030 con 0x7f3908076980 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.189:6838/1915044018,v1:172.21.15.189:6839/1915044018] up:boot 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: fsmap 9 up:standby 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:35 smithi083 ceph-mon[35133]: Deploying daemon mds.g on smithi057 2024-04-13T05:14:36.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.189:6838/1915044018,v1:172.21.15.189:6839/1915044018] up:boot 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: fsmap 9 up:standby 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:36.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:35 smithi189 ceph-mon[35531]: Deploying daemon mds.g on smithi057 2024-04-13T05:14:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.446+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f390802d5e0 con 0x7f3908076980 2024-04-13T05:14:36.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.447+0000 7f38ffe00640 1 -- 172.21.15.57:0/266271573 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7f38f809d4b0 con 0x7f3908076980 2024-04-13T05:14:36.449 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:36.449 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:36.449 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 10 2024-04-13T05:14:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f38ec0a3bb0 msgr2=0x7f38ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f38ec0a3bb0 0x7f38ec0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38f00096c0 tx=0x7f38f0000bc0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 msgr2=0x7f390819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 0x7f390819e7c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38f800eab0 tx=0x7f38f800ef70 comp rx=0 tx=0).stop 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 shutdown_connections 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f38ec0a3bb0 0x7f38ec0a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3908077bc0 0x7f390819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908078560 0x7f39081a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 --2- 172.21.15.57:0/266271573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3908076980 0x7f390819e7c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 >> 172.21.15.57:0/266271573 conn(0x7f39080ff7b0 msgr2=0x7f3908100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 shutdown_connections 2024-04-13T05:14:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:36.450+0000 7f390da00640 1 -- 172.21.15.57:0/266271573 wait complete. 2024-04-13T05:14:37.004 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:36 smithi057 ceph-mon[32331]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:37.004 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:36 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/266271573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:37.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:36 smithi083 ceph-mon[35133]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:37.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:36 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/266271573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:37.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:36 smithi189 ceph-mon[35531]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:37.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:36 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/266271573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:38.475 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 -- 172.21.15.57:0/3126423235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4102010 msgr2=0x7ff4c4102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 --2- 172.21.15.57:0/3126423235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4102010 0x7ff4c4102430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009ca0 tx=0x7ff4b402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 -- 172.21.15.57:0/3126423235 shutdown_connections 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 --2- 172.21.15.57:0/3126423235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4c4103200 0x7ff4c4103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 --2- 172.21.15.57:0/3126423235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4c4103bc0 0x7ff4c410a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 --2- 172.21.15.57:0/3126423235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4102010 0x7ff4c4102430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 -- 172.21.15.57:0/3126423235 >> 172.21.15.57:0/3126423235 conn(0x7ff4c40fd5a0 msgr2=0x7ff4c40ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 -- 172.21.15.57:0/3126423235 shutdown_connections 2024-04-13T05:14:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.710+0000 7ff4c9800640 1 -- 172.21.15.57:0/3126423235 wait complete. 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.711+0000 7ff4c9800640 1 Processor -- start 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.711+0000 7ff4c9800640 1 -- start start 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4c4102010 0x7ff4c419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 0x7ff4c419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4c4103bc0 0x7ff4c41a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c9800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c410db50 con 0x7ff4c4103200 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c410d9d0 con 0x7ff4c4102010 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c9800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c410dcd0 con 0x7ff4c4103bc0 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4ba000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 0x7ff4c419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:38.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4ba000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 0x7ff4c419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34982/0 (socket says 172.21.15.57:34982) 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4ba000640 1 -- 172.21.15.57:0/2623663616 learned_addr learned my addr 172.21.15.57:0/2623663616 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c2a00640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4c4102010 0x7ff4c419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.712+0000 7ff4c3400640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4c4103bc0 0x7ff4c41a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4ba000640 1 -- 172.21.15.57:0/2623663616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4c4103bc0 msgr2=0x7ff4c41a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4ba000640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4c4103bc0 0x7ff4c41a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4ba000640 1 -- 172.21.15.57:0/2623663616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4c4102010 msgr2=0x7ff4c419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4ba000640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4c4102010 0x7ff4c419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4ba000640 1 -- 172.21.15.57:0/2623663616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c41a2090 con 0x7ff4c4103200 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4ba000640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 0x7ff4c419a9b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00d950 tx=0x7ff4ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac017070 con 0x7ff4c4103200 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4ac009c30 con 0x7ff4c4103200 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00f620 con 0x7ff4c4103200 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c41a2380 con 0x7ff4c4103200 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.713+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a2880 con 0x7ff4c4103200 2024-04-13T05:14:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.715+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c41a2ae0 con 0x7ff4c4103200 2024-04-13T05:14:38.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.717+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4ac0109a0 con 0x7ff4c4103200 2024-04-13T05:14:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.718+0000 7ff4bbe00640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff4a80a3c30 0x7ff4a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:38.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.718+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff4ac0cfc60 con 0x7ff4c4103200 2024-04-13T05:14:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.718+0000 7ff4c2a00640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff4a80a3c30 0x7ff4a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:38.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.719+0000 7ff4c2a00640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff4a80a3c30 0x7ff4a80a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4b402c630 tx=0x7ff4b4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:38.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:38.723+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4ac009de0 con 0x7ff4c4103200 2024-04-13T05:14:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:38 smithi083 ceph-mon[35133]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.010+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4c41a2e70 con 0x7ff4c4103200 2024-04-13T05:14:39.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.012+0000 7ff4bbe00640 1 -- 172.21.15.57:0/2623663616 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8812 (secure 0 0 0) 0x7ff4ac09ca40 con 0x7ff4c4103200 2024-04-13T05:14:39.014 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:39.014 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:39.014 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 11 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff4a80a3c30 msgr2=0x7ff4a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff4a80a3c30 0x7ff4a80a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4b402c630 tx=0x7ff4b4000f40 comp rx=0 tx=0).stop 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 msgr2=0x7ff4c419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 0x7ff4c419a9b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00d950 tx=0x7ff4ac00de10 comp rx=0 tx=0).stop 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 shutdown_connections 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff4a80a3c30 0x7ff4a80a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4c4103bc0 0x7ff4c41a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4c4103200 0x7ff4c419a9b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 --2- 172.21.15.57:0/2623663616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4c4102010 0x7ff4c419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 >> 172.21.15.57:0/2623663616 conn(0x7ff4c40fd5a0 msgr2=0x7ff4c4108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 shutdown_connections 2024-04-13T05:14:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:39.015+0000 7ff4c9800640 1 -- 172.21.15.57:0/2623663616 wait complete. 2024-04-13T05:14:39.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:38 smithi057 ceph-mon[32331]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:39.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:38 smithi189 ceph-mon[35531]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:40.224 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.57:6838/59311703,v1:172.21.15.57:6839/59311703] up:boot 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: fsmap 10 up:standby 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2623663616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:40.225 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:40.226 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:39 smithi057 ceph-mon[32331]: Deploying daemon mds.a on smithi057 2024-04-13T05:14:40.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.57:6838/59311703,v1:172.21.15.57:6839/59311703] up:boot 2024-04-13T05:14:40.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: fsmap 10 up:standby 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2623663616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:40.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:39 smithi083 ceph-mon[35133]: Deploying daemon mds.a on smithi057 2024-04-13T05:14:40.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.57:6838/59311703,v1:172.21.15.57:6839/59311703] up:boot 2024-04-13T05:14:40.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: fsmap 10 up:standby 2024-04-13T05:14:40.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-13T05:14:40.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2623663616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:40.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:40.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:39 smithi189 ceph-mon[35531]: Deploying daemon mds.a on smithi057 2024-04-13T05:14:41.039 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:41.274 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:41 smithi057 ceph-mon[32331]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:41.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.284+0000 7f531fe00640 1 -- 172.21.15.57:0/1688845184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f532010e430 msgr2=0x7f5320110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:41.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.284+0000 7f531fe00640 1 --2- 172.21.15.57:0/1688845184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f532010e430 0x7f5320110820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5318009cd0 tx=0x7f531802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.284+0000 7f531fe00640 1 -- 172.21.15.57:0/1688845184 shutdown_connections 2024-04-13T05:14:41.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.284+0000 7f531fe00640 1 --2- 172.21.15.57:0/1688845184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f532010e430 0x7f5320110820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.285+0000 7f531fe00640 1 --2- 172.21.15.57:0/1688845184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5320074a20 0x7f5320074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.285+0000 7f531fe00640 1 --2- 172.21.15.57:0/1688845184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5320076000 0x7f5320074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.285+0000 7f531fe00640 1 -- 172.21.15.57:0/1688845184 >> 172.21.15.57:0/1688845184 conn(0x7f53200fd3d0 msgr2=0x7f53200ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:41.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.285+0000 7f531fe00640 1 -- 172.21.15.57:0/1688845184 shutdown_connections 2024-04-13T05:14:41.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.285+0000 7f531fe00640 1 -- 172.21.15.57:0/1688845184 wait complete. 2024-04-13T05:14:41.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 Processor -- start 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 -- start start 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5320074a20 0x7f532019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 0x7f532019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f532010e430 0x7f53201a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5320113fa0 con 0x7f5320076000 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5320113e20 con 0x7f5320074a20 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5320114120 con 0x7f532010e430 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.286+0000 7f531e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 0x7f532019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 0x7f532019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35018/0 (socket says 172.21.15.57:35018) 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 -- 172.21.15.57:0/224654542 learned_addr learned my addr 172.21.15.57:0/224654542 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531ea00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5320074a20 0x7f532019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531f400640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f532010e430 0x7f53201a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 -- 172.21.15.57:0/224654542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f532010e430 msgr2=0x7f53201a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:41.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f532010e430 0x7f53201a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 -- 172.21.15.57:0/224654542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5320074a20 msgr2=0x7f532019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:41.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5320074a20 0x7f532019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.287+0000 7f531e000640 1 -- 172.21.15.57:0/224654542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53201a61b0 con 0x7f5320076000 2024-04-13T05:14:41.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.288+0000 7f531e000640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 0x7f532019eb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c920 tx=0x7f530c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:41.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.288+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c007c70 con 0x7f5320076000 2024-04-13T05:14:41.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.288+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f530c007e10 con 0x7f5320076000 2024-04-13T05:14:41.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.288+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201a64a0 con 0x7f5320076000 2024-04-13T05:14:41.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.288+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c018820 con 0x7f5320076000 2024-04-13T05:14:41.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.288+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201a69f0 con 0x7f5320076000 2024-04-13T05:14:41.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.290+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53201044f0 con 0x7f5320076000 2024-04-13T05:14:41.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.294+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f530c0189c0 con 0x7f5320076000 2024-04-13T05:14:41.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.295+0000 7f5317400640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53040a3bb0 0x7f53040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:41.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.295+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f530c0d0120 con 0x7f5320076000 2024-04-13T05:14:41.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.296+0000 7f531ea00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53040a3bb0 0x7f53040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:41.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.296+0000 7f531ea00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53040a3bb0 0x7f53040a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f53100058c0 tx=0x7f531000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:41.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.300+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f530c014030 con 0x7f5320076000 2024-04-13T05:14:41.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:41 smithi189 ceph-mon[35531]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:41.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:41 smithi083 ceph-mon[35133]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:41.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.571+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5320062bc0 con 0x7f5320076000 2024-04-13T05:14:41.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.573+0000 7f5317400640 1 -- 172.21.15.57:0/224654542 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8812 (secure 0 0 0) 0x7f530c09cf00 con 0x7f5320076000 2024-04-13T05:14:41.574 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:41.574 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:41.575 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 11 2024-04-13T05:14:41.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53040a3bb0 msgr2=0x7f53040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53040a3bb0 0x7f53040a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f53100058c0 tx=0x7f531000a2d0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 msgr2=0x7f532019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 0x7f532019eb20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c920 tx=0x7f530c00cde0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 shutdown_connections 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53040a3bb0 0x7f53040a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f532010e430 0x7f53201a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5320076000 0x7f532019eb20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 --2- 172.21.15.57:0/224654542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5320074a20 0x7f532019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:41.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.575+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 >> 172.21.15.57:0/224654542 conn(0x7f53200fd3d0 msgr2=0x7f53201016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:41.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.576+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 shutdown_connections 2024-04-13T05:14:41.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:41.576+0000 7f531fe00640 1 -- 172.21.15.57:0/224654542 wait complete. 2024-04-13T05:14:42.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:42 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/224654542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:42.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:42 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/224654542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:42.498 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 systemd[1]: Starting Ceph mds.a for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:42.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:42 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/224654542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:42.782 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 podman[75301]: 2024-04-13 05:14:42.403128367 +0000 UTC m=+0.038335949 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:42.782 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 podman[75301]: 2024-04-13 05:14:42.556801717 +0000 UTC m=+0.192009336 container create 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0) 2024-04-13T05:14:43.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:43 smithi057 ceph-mon[32331]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:43.064 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 podman[75301]: 2024-04-13 05:14:42.81077265 +0000 UTC m=+0.445980276 container init 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-13T05:14:43.064 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 podman[75301]: 2024-04-13 05:14:42.82308118 +0000 UTC m=+0.458288768 container start 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-13T05:14:43.064 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 ceph-mds[75320]: -- 172.21.15.57:0/943618802 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd49b15a40 con 0x55cd49ae9000 2024-04-13T05:14:43.064 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a[75316]: starting mds.a at 2024-04-13T05:14:43.064 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 bash[75301]: 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d 2024-04-13T05:14:43.064 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:14:42 smithi057 systemd[1]: Started Ceph mds.a for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:43.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:43 smithi189 ceph-mon[35531]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:43.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:43.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:43 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:43.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:43 smithi083 ceph-mon[35133]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:43.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:43.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:43 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:43.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:43.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:43 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:43.597 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 -- 172.21.15.57:0/2261711640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105230 msgr2=0x7feb001056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 --2- 172.21.15.57:0/2261711640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105230 0x7feb001056b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feaec009900 tx=0x7feaec02bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 -- 172.21.15.57:0/2261711640 shutdown_connections 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 --2- 172.21.15.57:0/2261711640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb00105bf0 0x7feb0010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 --2- 172.21.15.57:0/2261711640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105230 0x7feb001056b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 --2- 172.21.15.57:0/2261711640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb00104040 0x7feb00104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 -- 172.21.15.57:0/2261711640 >> 172.21.15.57:0/2261711640 conn(0x7feb000ff5d0 msgr2=0x7feb001019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 -- 172.21.15.57:0/2261711640 shutdown_connections 2024-04-13T05:14:43.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.834+0000 7feb05c00640 1 -- 172.21.15.57:0/2261711640 wait complete. 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.835+0000 7feb05c00640 1 Processor -- start 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.835+0000 7feb05c00640 1 -- start start 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb00104040 0x7feb0019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb00105230 0x7feb0019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 0x7feb001a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7feb0010fbd0 con 0x7feb00105bf0 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb0010fa50 con 0x7feb00104040 2024-04-13T05:14:43.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb0010fd50 con 0x7feb00105230 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 0x7feb001a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 0x7feb001a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35072/0 (socket says 172.21.15.57:35072) 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feb05200640 1 -- 172.21.15.57:0/4124940432 learned_addr learned my addr 172.21.15.57:0/4124940432 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.836+0000 7feaffe00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb00104040 0x7feb0019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05200640 1 -- 172.21.15.57:0/4124940432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb00105230 msgr2=0x7feb0019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05200640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb00105230 0x7feb0019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05200640 1 -- 172.21.15.57:0/4124940432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb00104040 msgr2=0x7feb0019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05200640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb00104040 0x7feb0019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05200640 1 -- 172.21.15.57:0/4124940432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb001a1ed0 con 0x7feb00105bf0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05200640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 0x7feb001a1860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feaf800efd0 tx=0x7feaf800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8003b70 con 0x7feb00105bf0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf8003d10 con 0x7feb00105bf0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8010980 con 0x7feb00105bf0 2024-04-13T05:14:43.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.837+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb001a21c0 con 0x7feb00105bf0 2024-04-13T05:14:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.838+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a26c0 con 0x7feb00105bf0 2024-04-13T05:14:43.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.839+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0010b3e0 con 0x7feb00105bf0 2024-04-13T05:14:43.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.841+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feaf8004020 con 0x7feb00105bf0 2024-04-13T05:14:43.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.842+0000 7feafcc00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feae40a3c30 0x7feae40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:43.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.842+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7feaf80d0430 con 0x7feb00105bf0 2024-04-13T05:14:43.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.843+0000 7feaffe00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feae40a3c30 0x7feae40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:43.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.843+0000 7feaffe00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feae40a3c30 0x7feae40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005fd0 tx=0x7feaf000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:43.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:43.847+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feaf809d210 con 0x7feb00105bf0 2024-04-13T05:14:44.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.102+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb0002d5e0 con 0x7feb00105bf0 2024-04-13T05:14:44.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.104+0000 7feafcc00640 1 -- 172.21.15.57:0/4124940432 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9638 (secure 0 0 0) 0x7feaf809cb10 con 0x7feb00105bf0 2024-04-13T05:14:44.106 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:44.106 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:44.106 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 12 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feae40a3c30 msgr2=0x7feae40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feae40a3c30 0x7feae40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005fd0 tx=0x7feaf000b390 comp rx=0 tx=0).stop 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 msgr2=0x7feb001a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 0x7feb001a1860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feaf800efd0 tx=0x7feaf800c6a0 comp rx=0 tx=0).stop 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 shutdown_connections 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feae40a3c30 0x7feae40a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb00105bf0 0x7feb001a1860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb00105230 0x7feb0019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 --2- 172.21.15.57:0/4124940432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb00104040 0x7feb0019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 >> 172.21.15.57:0/4124940432 conn(0x7feb000ff5d0 msgr2=0x7feb001085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 shutdown_connections 2024-04-13T05:14:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:44.107+0000 7feb05c00640 1 -- 172.21.15.57:0/4124940432 wait complete. 2024-04-13T05:14:44.327 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.57:6840/2975868204,v1:172.21.15.57:6841/2975868204] up:boot 2024-04-13T05:14:44.327 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: fsmap 11 up:standby 2024-04-13T05:14:44.327 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-13T05:14:44.327 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:44.327 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:44.327 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:44.328 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:44.328 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:44 smithi083 ceph-mon[35133]: Deploying daemon mds.b on smithi083 2024-04-13T05:14:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.57:6840/2975868204,v1:172.21.15.57:6841/2975868204] up:boot 2024-04-13T05:14:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: fsmap 11 up:standby 2024-04-13T05:14:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-13T05:14:44.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:44.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:44.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:44.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:44.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:44 smithi189 ceph-mon[35531]: Deploying daemon mds.b on smithi083 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.57:6840/2975868204,v1:172.21.15.57:6841/2975868204] up:boot 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: fsmap 11 up:standby 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:44.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:44 smithi057 ceph-mon[32331]: Deploying daemon mds.b on smithi083 2024-04-13T05:14:45.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:45 smithi083 ceph-mon[35133]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:45 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4124940432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:45.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:45 smithi057 ceph-mon[32331]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:45 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4124940432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:45.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:45 smithi189 ceph-mon[35531]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:45.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:45 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4124940432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:46.131 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 -- 172.21.15.57:0/2208654503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc078560 msgr2=0x7f97cc078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2208654503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc078560 0x7f97cc078e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009ca0 tx=0x7f97c402c100 comp rx=0 tx=0).stop 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 -- 172.21.15.57:0/2208654503 shutdown_connections 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2208654503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc078560 0x7f97cc078e00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2208654503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97cc077bc0 0x7f97cc078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2208654503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc076980 0x7f97cc076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 -- 172.21.15.57:0/2208654503 >> 172.21.15.57:0/2208654503 conn(0x7f97cc0ff7b0 msgr2=0x7f97cc101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:46.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 -- 172.21.15.57:0/2208654503 shutdown_connections 2024-04-13T05:14:46.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.366+0000 7f97d3c00640 1 -- 172.21.15.57:0/2208654503 wait complete. 2024-04-13T05:14:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 Processor -- start 2024-04-13T05:14:46.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 -- start start 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 0x7f97cc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc077bc0 0x7f97cc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97cc078560 0x7f97cc1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc1141c0 con 0x7f97cc076980 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc114040 con 0x7f97cc077bc0 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.367+0000 7f97d3c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc114340 con 0x7f97cc078560 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 0x7f97cc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 0x7f97cc19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39170/0 (socket says 172.21.15.57:39170) 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 -- 172.21.15.57:0/2196330631 learned_addr learned my addr 172.21.15.57:0/2196330631 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97c3e00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc077bc0 0x7f97cc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 -- 172.21.15.57:0/2196330631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97cc078560 msgr2=0x7f97cc1a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97cc078560 0x7f97cc1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 -- 172.21.15.57:0/2196330631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc077bc0 msgr2=0x7f97cc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc077bc0 0x7f97cc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.368+0000 7f97d1200640 1 -- 172.21.15.57:0/2196330631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97cc1a6450 con 0x7f97cc076980 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.369+0000 7f97d1200640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 0x7f97cc19e830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00e9f0 tx=0x7f97bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.369+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00ccc0 con 0x7f97cc076980 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.369+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97bc0108e0 con 0x7f97cc076980 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.369+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0187c0 con 0x7f97cc076980 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.369+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97cc1a6740 con 0x7f97cc076980 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.369+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97cc1a6d60 con 0x7f97cc076980 2024-04-13T05:14:46.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.370+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97cc110d50 con 0x7f97cc076980 2024-04-13T05:14:46.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.372+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f97bc010a90 con 0x7f97cc076980 2024-04-13T05:14:46.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.373+0000 7f97ca000640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f97b00a3c30 0x7f97b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:46.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.374+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f97bc0d03f0 con 0x7f97cc076980 2024-04-13T05:14:46.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.374+0000 7f97c3e00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f97b00a3c30 0x7f97b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:46.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.374+0000 7f97c3e00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f97b00a3c30 0x7f97b00a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97cc19fd40 tx=0x7f97b4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:46.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.378+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97bc09d1d0 con 0x7f97cc076980 2024-04-13T05:14:46.501 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 systemd[1]: Starting Ceph mds.b for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:14:46.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.624+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97cc062bf0 con 0x7f97cc076980 2024-04-13T05:14:46.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.625+0000 7f97ca000640 1 -- 172.21.15.57:0/2196330631 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9638 (secure 0 0 0) 0x7f97bc09cad0 con 0x7f97cc076980 2024-04-13T05:14:46.627 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:46.627 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-13T05:14:46.627 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 12 2024-04-13T05:14:46.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.627+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f97b00a3c30 msgr2=0x7f97b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:46.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.627+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f97b00a3c30 0x7f97b00a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97cc19fd40 tx=0x7f97b4009110 comp rx=0 tx=0).stop 2024-04-13T05:14:46.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.627+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 msgr2=0x7f97cc19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:46.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.627+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 0x7f97cc19e830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00e9f0 tx=0x7f97bc00edf0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 shutdown_connections 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f97b00a3c30 0x7f97b00a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97cc077bc0 0x7f97cc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97cc078560 0x7f97cc1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 --2- 172.21.15.57:0/2196330631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97cc076980 0x7f97cc19e830 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 >> 172.21.15.57:0/2196330631 conn(0x7f97cc0ff7b0 msgr2=0x7f97cc100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 shutdown_connections 2024-04-13T05:14:46.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:46.628+0000 7f97d3c00640 1 -- 172.21.15.57:0/2196330631 wait complete. 2024-04-13T05:14:46.856 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 podman[65824]: 2024-04-13 05:14:46.488497233 +0000 UTC m=+0.033518281 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:14:46.856 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 podman[65824]: 2024-04-13 05:14:46.599793811 +0000 UTC m=+0.144814896 container create 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:14:46.856 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 podman[65824]: 2024-04-13 05:14:46.853948023 +0000 UTC m=+0.398969135 container init 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:47 smithi083 ceph-mon[35133]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:47 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2196330631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:47 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 podman[65824]: 2024-04-13 05:14:46.864130663 +0000 UTC m=+0.409151732 container start 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 ceph-mds[65843]: -- 172.21.15.83:0/1427664227 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbd4d9fa40 con 0x55cbd4d73000 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: starting mds.b at 2024-04-13T05:14:47.251 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 bash[65824]: 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d 2024-04-13T05:14:47.252 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:46 smithi083 systemd[1]: Started Ceph mds.b for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:14:47.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:47 smithi057 ceph-mon[32331]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:47.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:47 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2196330631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:47.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:47 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:47.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:47 smithi189 ceph-mon[35531]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:47.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:47 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2196330631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:47.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:47 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] up:boot 2024-04-13T05:14:48.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: fsmap 12 up:standby 2024-04-13T05:14:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-13T05:14:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:14:48.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] up:boot 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: fsmap 12 up:standby 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:14:48.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] up:boot 2024-04-13T05:14:48.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: fsmap 12 up:standby 2024-04-13T05:14:48.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-13T05:14:48.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:48.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:14:48.651 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:48.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.857+0000 7f78c1a00640 1 -- 172.21.15.57:0/458305533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc105410 msgr2=0x7f78bc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:48.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.857+0000 7f78c1a00640 1 --2- 172.21.15.57:0/458305533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc105410 0x7f78bc105890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009900 tx=0x7f78a402bc40 comp rx=0 tx=0).stop 2024-04-13T05:14:48.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 -- 172.21.15.57:0/458305533 shutdown_connections 2024-04-13T05:14:48.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 --2- 172.21.15.57:0/458305533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 0x7f78bc10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:48.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 --2- 172.21.15.57:0/458305533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc105410 0x7f78bc105890 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:48.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 --2- 172.21.15.57:0/458305533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78bc104220 0x7f78bc104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:48.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 -- 172.21.15.57:0/458305533 >> 172.21.15.57:0/458305533 conn(0x7f78bc0ff7b0 msgr2=0x7f78bc101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:48.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 -- 172.21.15.57:0/458305533 shutdown_connections 2024-04-13T05:14:48.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.858+0000 7f78c1a00640 1 -- 172.21.15.57:0/458305533 wait complete. 2024-04-13T05:14:48.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 Processor -- start 2024-04-13T05:14:48.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 -- start start 2024-04-13T05:14:48.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 0x7f78bc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:48.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78bc105410 0x7f78bc19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 0x7f78bc1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10fdb0 con 0x7f78bc104220 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10fc30 con 0x7f78bc105dd0 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.859+0000 7f78c1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc10ff30 con 0x7f78bc105410 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 0x7f78bc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 0x7f78bc19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39182/0 (socket says 172.21.15.57:39182) 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 -- 172.21.15.57:0/746802318 learned_addr learned my addr 172.21.15.57:0/746802318 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78b2000640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78bc105410 0x7f78bc19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78bb400640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 0x7f78bc1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 -- 172.21.15.57:0/746802318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78bc105410 msgr2=0x7f78bc19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78bc105410 0x7f78bc19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 -- 172.21.15.57:0/746802318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 msgr2=0x7f78bc1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:48.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 0x7f78bc1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78baa00640 1 -- 172.21.15.57:0/746802318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc1a2000 con 0x7f78bc104220 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.860+0000 7f78bb400640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 0x7f78bc1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.861+0000 7f78baa00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 0x7f78bc19a3e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00d950 tx=0x7f78ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.861+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac017070 con 0x7f78bc104220 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.861+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78ac009c30 con 0x7f78bc104220 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.861+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc1a22f0 con 0x7f78bc104220 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.861+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00f620 con 0x7f78bc104220 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.861+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a27c0 con 0x7f78bc104220 2024-04-13T05:14:48.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.862+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc10b5c0 con 0x7f78bc104220 2024-04-13T05:14:48.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.864+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78ac01a070 con 0x7f78bc104220 2024-04-13T05:14:48.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.866+0000 7f78b3e00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f78a00a3bb0 0x7f78a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:48.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.866+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f78ac0cfa80 con 0x7f78bc104220 2024-04-13T05:14:48.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.866+0000 7f78b2000640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f78a00a3bb0 0x7f78a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:48.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.867+0000 7f78b2000640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f78a00a3bb0 0x7f78a00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78a4006010 tx=0x7f78a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:48.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:48.870+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78ac009de0 con 0x7f78bc104220 2024-04-13T05:14:49.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.122+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78bc02d5e0 con 0x7f78bc104220 2024-04-13T05:14:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.123+0000 7f78b3e00640 1 -- 172.21.15.57:0/746802318 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7f78ac09c900 con 0x7f78bc104220 2024-04-13T05:14:49.124 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:49.125 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-13T05:14:49.125 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2024-04-13T05:14:49.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.127+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f78a00a3bb0 msgr2=0x7f78a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.127+0000 7f78c1a00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f78a00a3bb0 0x7f78a00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78a4006010 tx=0x7f78a4000bc0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.127+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 msgr2=0x7f78bc19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.127+0000 7f78c1a00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 0x7f78bc19a3e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00d950 tx=0x7f78ac00de10 comp rx=0 tx=0).stop 2024-04-13T05:14:49.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 shutdown_connections 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f78a00a3bb0 0x7f78a00a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78bc105dd0 0x7f78bc1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78bc105410 0x7f78bc19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 --2- 172.21.15.57:0/746802318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78bc104220 0x7f78bc19a3e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 >> 172.21.15.57:0/746802318 conn(0x7f78bc0ff7b0 msgr2=0x7f78bc108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 shutdown_connections 2024-04-13T05:14:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.128+0000 7f78c1a00640 1 -- 172.21.15.57:0/746802318 wait complete. 2024-04-13T05:14:49.154 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:14:49.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:49 smithi083 ceph-mon[35133]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:49.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:49 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/746802318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:49.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:49 smithi057 ceph-mon[32331]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:49.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:49 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/746802318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:49.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:49 smithi189 ceph-mon[35531]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:49 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/746802318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 -- 172.21.15.57:0/623527892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea5010a1d0 msgr2=0x7fea50110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 --2- 172.21.15.57:0/623527892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea5010a1d0 0x7fea50110a60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea48009ca0 tx=0x7fea4802c100 comp rx=0 tx=0).stop 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 -- 172.21.15.57:0/623527892 shutdown_connections 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 --2- 172.21.15.57:0/623527892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea5010a1d0 0x7fea50110a60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 --2- 172.21.15.57:0/623527892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea50109810 0x7fea50109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 --2- 172.21.15.57:0/623527892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea50108620 0x7fea50108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 -- 172.21.15.57:0/623527892 >> 172.21.15.57:0/623527892 conn(0x7fea500775e0 msgr2=0x7fea50079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:49.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.361+0000 7fea57200640 1 -- 172.21.15.57:0/623527892 shutdown_connections 2024-04-13T05:14:49.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.362+0000 7fea57200640 1 -- 172.21.15.57:0/623527892 wait complete. 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.362+0000 7fea57200640 1 Processor -- start 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 -- start start 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 0x7fea5019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea50109810 0x7fea5019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5010a1d0 0x7fea501a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fea50114130 con 0x7fea50108620 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fea50113fb0 con 0x7fea5010a1d0 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea57200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea501142b0 con 0x7fea50109810 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 0x7fea5019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 0x7fea5019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39190/0 (socket says 172.21.15.57:39190) 2024-04-13T05:14:49.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.363+0000 7fea4fe00640 1 -- 172.21.15.57:0/3146995378 learned_addr learned my addr 172.21.15.57:0/3146995378 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea55200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5010a1d0 0x7fea501a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea47e00640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea50109810 0x7fea5019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4fe00640 1 -- 172.21.15.57:0/3146995378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea50109810 msgr2=0x7fea5019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4fe00640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea50109810 0x7fea5019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4fe00640 1 -- 172.21.15.57:0/3146995378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5010a1d0 msgr2=0x7fea501a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4fe00640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5010a1d0 0x7fea501a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4fe00640 1 -- 172.21.15.57:0/3146995378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea501a6420 con 0x7fea50108620 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea55200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5010a1d0 0x7fea501a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4fe00640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 0x7fea5019e800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d770 tx=0x7fea4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40017070 con 0x7fea50108620 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea40009c30 con 0x7fea50108620 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea501a6710 con 0x7fea50108620 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.364+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40003d00 con 0x7fea50108620 2024-04-13T05:14:49.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.365+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501a6c60 con 0x7fea50108620 2024-04-13T05:14:49.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.366+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5010f9c0 con 0x7fea50108620 2024-04-13T05:14:49.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.368+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea40003ea0 con 0x7fea50108620 2024-04-13T05:14:49.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.369+0000 7fea4d600640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fea340a3bb0 0x7fea340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.369+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fea400cfdc0 con 0x7fea50108620 2024-04-13T05:14:49.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.369+0000 7fea47e00640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fea340a3bb0 0x7fea340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.370+0000 7fea47e00640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fea340a3bb0 0x7fea340a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea5019fd10 tx=0x7fea3800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:49.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.374+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea4009cba0 con 0x7fea50108620 2024-04-13T05:14:49.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.592+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea50062bf0 con 0x7fea50108620 2024-04-13T05:14:49.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.593+0000 7fea4d600640 1 -- 172.21.15.57:0/3146995378 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18382 (secure 0 0 0) 0x7fea40009de0 con 0x7fea50108620 2024-04-13T05:14:49.595 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:49.595 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":82,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:13:46.414789+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:14:49.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.596+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fea340a3bb0 msgr2=0x7fea340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.596+0000 7fea57200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fea340a3bb0 0x7fea340a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea5019fd10 tx=0x7fea3800a4c0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.596+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 msgr2=0x7fea5019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.596+0000 7fea57200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 0x7fea5019e800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d770 tx=0x7fea4000dc30 comp rx=0 tx=0).stop 2024-04-13T05:14:49.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 shutdown_connections 2024-04-13T05:14:49.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fea340a3bb0 0x7fea340a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5010a1d0 0x7fea501a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea50109810 0x7fea5019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 --2- 172.21.15.57:0/3146995378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea50108620 0x7fea5019e800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 >> 172.21.15.57:0/3146995378 conn(0x7fea500775e0 msgr2=0x7fea5010cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:49.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 shutdown_connections 2024-04-13T05:14:49.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.597+0000 7fea57200640 1 -- 172.21.15.57:0/3146995378 wait complete. 2024-04-13T05:14:49.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-13T05:14:49.619 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:14:49.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.844+0000 7f14d7600640 1 -- 172.21.15.57:0/315820614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 msgr2=0x7f14d0105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.844+0000 7f14d7600640 1 --2- 172.21.15.57:0/315820614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d0105870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009900 tx=0x7f14bc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 -- 172.21.15.57:0/315820614 shutdown_connections 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 --2- 172.21.15.57:0/315820614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 0x7f14d010c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 --2- 172.21.15.57:0/315820614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d0105870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 --2- 172.21.15.57:0/315820614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 0x7f14d0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 -- 172.21.15.57:0/315820614 >> 172.21.15.57:0/315820614 conn(0x7f14d00ff770 msgr2=0x7f14d0101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 -- 172.21.15.57:0/315820614 shutdown_connections 2024-04-13T05:14:49.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.845+0000 7f14d7600640 1 -- 172.21.15.57:0/315820614 wait complete. 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.846+0000 7f14d7600640 1 Processor -- start 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.846+0000 7f14d7600640 1 -- start start 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d7600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 0x7f14d019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d7600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d7600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 0x7f14d01a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d7600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010fdd0 con 0x7f14d01053f0 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d7600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010fc50 con 0x7f14d0104200 2024-04-13T05:14:49.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d7600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010ff50 con 0x7f14d0105db0 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14cfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 0x7f14d019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14cfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39214/0 (socket says 172.21.15.57:39214) 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14cfe00640 1 -- 172.21.15.57:0/518080442 learned_addr learned my addr 172.21.15.57:0/518080442 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.847+0000 7f14d5600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 0x7f14d01a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cfe00640 1 -- 172.21.15.57:0/518080442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 msgr2=0x7f14d01a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cfe00640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 0x7f14d01a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cfe00640 1 -- 172.21.15.57:0/518080442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 msgr2=0x7f14d019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cfe00640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 0x7f14d019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cfe00640 1 -- 172.21.15.57:0/518080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d01a2090 con 0x7f14d01053f0 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14d5600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 0x7f14d01a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14d4c00640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 0x7f14d019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cfe00640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d019a9b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14bc000c00 tx=0x7f14bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.848+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc043020 con 0x7f14d01053f0 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.849+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14bc004aa0 con 0x7f14d01053f0 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.849+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d01a2320 con 0x7f14d01053f0 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.849+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc003d00 con 0x7f14d01053f0 2024-04-13T05:14:49.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.849+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01a28f0 con 0x7f14d01053f0 2024-04-13T05:14:49.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.850+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d010b5a0 con 0x7f14d01053f0 2024-04-13T05:14:49.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.852+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14bc046020 con 0x7f14d01053f0 2024-04-13T05:14:49.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.856+0000 7f14cd600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14b80a3bb0 0x7f14b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:49.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.856+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f14bc0f2370 con 0x7f14d01053f0 2024-04-13T05:14:49.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.856+0000 7f14d4c00640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14b80a3bb0 0x7f14b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:49.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.857+0000 7f14d4c00640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14b80a3bb0 0x7f14b80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14d0105250 tx=0x7f14c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:49.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:49.859+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14bc04d020 con 0x7f14d01053f0 2024-04-13T05:14:50.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.072+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14d0062bc0 con 0x7f14d01053f0 2024-04-13T05:14:50.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.072+0000 7f14cd600640 1 -- 172.21.15.57:0/518080442 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f14bc0bf150 con 0x7f14d01053f0 2024-04-13T05:14:50.074 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:14:50.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14b80a3bb0 msgr2=0x7f14b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14b80a3bb0 0x7f14b80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14d0105250 tx=0x7f14c4005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 msgr2=0x7f14d019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d019a9b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14bc000c00 tx=0x7f14bc0042f0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 shutdown_connections 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14b80a3bb0 0x7f14b80a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d0105db0 0x7f14d01a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14d01053f0 0x7f14d019a9b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 --2- 172.21.15.57:0/518080442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0104200 0x7f14d019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 >> 172.21.15.57:0/518080442 conn(0x7f14d00ff770 msgr2=0x7f14d0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 shutdown_connections 2024-04-13T05:14:50.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.075+0000 7f14d7600640 1 -- 172.21.15.57:0/518080442 wait complete. 2024-04-13T05:14:50.096 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-13T05:14:50.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1410910435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4102fb0 msgr2=0x7fe4c4103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1410910435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4102fb0 0x7fe4c4103430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009860 tx=0x7fe4b002bb70 comp rx=0 tx=0).stop 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1410910435 shutdown_connections 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1410910435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4102fb0 0x7fe4c4103430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1410910435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1410910435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4101e60 0x7fe4c4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1410910435 >> 172.21.15.57:0/1410910435 conn(0x7fe4c40fd3d0 msgr2=0x7fe4c40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1410910435 shutdown_connections 2024-04-13T05:14:50.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.294+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1410910435 wait complete. 2024-04-13T05:14:50.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.295+0000 7fe4c3e00640 1 Processor -- start 2024-04-13T05:14:50.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.295+0000 7fe4c3e00640 1 -- start start 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.295+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4101e60 0x7fe4c419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4102fb0 0x7fe4c419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c41a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c410d990 con 0x7fe4c4103970 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c410d810 con 0x7fe4c4101e60 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c410db10 con 0x7fe4c4102fb0 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c41a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c41a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39230/0 (socket says 172.21.15.57:39230) 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 -- 172.21.15.57:0/1201932686 learned_addr learned my addr 172.21.15.57:0/1201932686 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4101e60 0x7fe4c419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c2000640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4102fb0 0x7fe4c419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 -- 172.21.15.57:0/1201932686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4102fb0 msgr2=0x7fe4c419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4102fb0 0x7fe4c419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 -- 172.21.15.57:0/1201932686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4101e60 msgr2=0x7fe4c419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4101e60 0x7fe4c419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c3400640 1 -- 172.21.15.57:0/1201932686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41a1ee0 con 0x7fe4c4103970 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.296+0000 7fe4c2000640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4102fb0 0x7fe4c419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.297+0000 7fe4c3400640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c41a1870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00efc0 tx=0x7fe4bc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.297+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003970 con 0x7fe4c4103970 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.297+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c41a21d0 con 0x7fe4c4103970 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.297+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4bc004020 con 0x7fe4c4103970 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.297+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00b940 con 0x7fe4c4103970 2024-04-13T05:14:50.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.297+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41a26f0 con 0x7fe4c4103970 2024-04-13T05:14:50.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.299+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c41a2950 con 0x7fe4c4103970 2024-04-13T05:14:50.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.301+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe4bc01d020 con 0x7fe4c4103970 2024-04-13T05:14:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.304+0000 7fe4bb400640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4a80a3c30 0x7fe4a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.304+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe4bc0cfec0 con 0x7fe4c4103970 2024-04-13T05:14:50.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.304+0000 7fe4c2a00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4a80a3c30 0x7fe4a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.305+0000 7fe4c2a00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4a80a3c30 0x7fe4a80a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009bd0 tx=0x7fe4b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:50.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.307+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4bc09cca0 con 0x7fe4c4103970 2024-04-13T05:14:50.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:50 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3146995378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:14:50.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:50 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/518080442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:14:50.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.524+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe4c41a2ce0 con 0x7fe4c4103970 2024-04-13T05:14:50.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.527+0000 7fe4bb400640 1 -- 172.21.15.57:0/1201932686 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fe4bc09c5a0 con 0x7fe4c4103970 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:14:50.529 INFO:teuthology.orchestra.run.smithi057.stdout:] 2024-04-13T05:14:50.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:50 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3146995378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:14:50.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:50 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/518080442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:14:50.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4a80a3c30 msgr2=0x7fe4a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4a80a3c30 0x7fe4a80a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009bd0 tx=0x7fe4b4005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 msgr2=0x7fe4c41a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c41a1870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00efc0 tx=0x7fe4bc00c800 comp rx=0 tx=0).stop 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 shutdown_connections 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4a80a3c30 0x7fe4a80a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4c4103970 0x7fe4c41a1870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4c4102fb0 0x7fe4c419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 --2- 172.21.15.57:0/1201932686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c4101e60 0x7fe4c419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 >> 172.21.15.57:0/1201932686 conn(0x7fe4c40fd3d0 msgr2=0x7fe4c40741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 shutdown_connections 2024-04-13T05:14:50.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.531+0000 7fe4c3e00640 1 -- 172.21.15.57:0/1201932686 wait complete. 2024-04-13T05:14:50.557 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-13T05:14:50.557 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-13T05:14:50.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:50 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3146995378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:14:50.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:50 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/518080442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:14:50.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 -- 172.21.15.57:0/1046973811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 msgr2=0x7f2a4c104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 --2- 172.21.15.57:0/1046973811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c104640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009900 tx=0x7f2a3c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:50.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 -- 172.21.15.57:0/1046973811 shutdown_connections 2024-04-13T05:14:50.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 --2- 172.21.15.57:0/1046973811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 0x7f2a4c10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 --2- 172.21.15.57:0/1046973811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a4c105410 0x7f2a4c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 --2- 172.21.15.57:0/1046973811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c104640 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 -- 172.21.15.57:0/1046973811 >> 172.21.15.57:0/1046973811 conn(0x7f2a4c0ff770 msgr2=0x7f2a4c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:50.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 -- 172.21.15.57:0/1046973811 shutdown_connections 2024-04-13T05:14:50.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.747+0000 7f2a53000640 1 -- 172.21.15.57:0/1046973811 wait complete. 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.748+0000 7f2a53000640 1 Processor -- start 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 -- start start 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a4c105410 0x7f2a4c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 0x7f2a4c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c10fd60 con 0x7f2a4c104220 2024-04-13T05:14:50.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c10fbe0 con 0x7f2a4c105dd0 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a53000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c10fee0 con 0x7f2a4c105410 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39244/0 (socket says 172.21.15.57:39244) 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a4be00640 1 -- 172.21.15.57:0/551441355 learned_addr learned my addr 172.21.15.57:0/551441355 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a51000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 0x7f2a4c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.749+0000 7f2a43e00640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a4c105410 0x7f2a4c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a4be00640 1 -- 172.21.15.57:0/551441355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a4c105410 msgr2=0x7f2a4c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a4be00640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a4c105410 0x7f2a4c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a4be00640 1 -- 172.21.15.57:0/551441355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 msgr2=0x7f2a4c1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a4be00640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 0x7f2a4c1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a4be00640 1 -- 172.21.15.57:0/551441355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c1a20a0 con 0x7f2a4c104220 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a51000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 0x7f2a4c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.750+0000 7f2a4be00640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c19a480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000c00 tx=0x7f2a3c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.751+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c043020 con 0x7f2a4c104220 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.751+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a3c004440 con 0x7f2a4c104220 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.751+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c003c90 con 0x7f2a4c104220 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.751+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c1a2330 con 0x7f2a4c104220 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.751+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c1a28d0 con 0x7f2a4c104220 2024-04-13T05:14:50.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.752+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4c10b660 con 0x7f2a4c104220 2024-04-13T05:14:50.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.756+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a3c046020 con 0x7f2a4c104220 2024-04-13T05:14:50.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.757+0000 7f2a49600640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a300a3c30 0x7f2a300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:50.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.757+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a3c0f21b0 con 0x7f2a4c104220 2024-04-13T05:14:50.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.758+0000 7f2a43e00640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a300a3c30 0x7f2a300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:50.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.758+0000 7f2a43e00640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a300a3c30 0x7f2a300a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005fd0 tx=0x7f2a34005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:50.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.762+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a3c04d020 con 0x7f2a4c104220 2024-04-13T05:14:50.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:50.971+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2a4c02d5e0 con 0x7f2a4c104220 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1201932686' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:51 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/551441355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1201932686' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:51 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/551441355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-13T05:14:51.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.557+0000 7f2a49600640 1 -- 172.21.15.57:0/551441355 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f2a3c0a7e20 con 0x7f2a4c104220 2024-04-13T05:14:51.558 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2024-04-13T05:14:51.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a300a3c30 msgr2=0x7f2a300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:51.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a300a3c30 0x7f2a300a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005fd0 tx=0x7f2a34005850 comp rx=0 tx=0).stop 2024-04-13T05:14:51.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 msgr2=0x7f2a4c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:51.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c19a480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000c00 tx=0x7f2a3c004040 comp rx=0 tx=0).stop 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 shutdown_connections 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a300a3c30 0x7f2a300a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a4c105dd0 0x7f2a4c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a4c105410 0x7f2a4c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 --2- 172.21.15.57:0/551441355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a4c104220 0x7f2a4c19a480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 >> 172.21.15.57:0/551441355 conn(0x7f2a4c0ff770 msgr2=0x7f2a4c108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 shutdown_connections 2024-04-13T05:14:51.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.561+0000 7f2a53000640 1 -- 172.21.15.57:0/551441355 wait complete. 2024-04-13T05:14:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1201932686' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-13T05:14:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:51.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:51 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/551441355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-13T05:14:51.595 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-13T05:14:51.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.811+0000 7fe3a7200640 1 -- 172.21.15.57:0/225836411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8102130 msgr2=0x7fe3a81089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:51.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.811+0000 7fe3a7200640 1 --2- 172.21.15.57:0/225836411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8102130 0x7fe3a81089c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009ca0 tx=0x7fe3a002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.811+0000 7fe3a7200640 1 -- 172.21.15.57:0/225836411 shutdown_connections 2024-04-13T05:14:51.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.811+0000 7fe3a7200640 1 --2- 172.21.15.57:0/225836411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8102130 0x7fe3a81089c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.811+0000 7fe3a7200640 1 --2- 172.21.15.57:0/225836411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8101770 0x7fe3a8101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.811+0000 7fe3a7200640 1 --2- 172.21.15.57:0/225836411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a8100580 0x7fe3a81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.812+0000 7fe3a7200640 1 -- 172.21.15.57:0/225836411 >> 172.21.15.57:0/225836411 conn(0x7fe3a80fbaf0 msgr2=0x7fe3a80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:51.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.812+0000 7fe3a7200640 1 -- 172.21.15.57:0/225836411 shutdown_connections 2024-04-13T05:14:51.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.812+0000 7fe3a7200640 1 -- 172.21.15.57:0/225836411 wait complete. 2024-04-13T05:14:51.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 Processor -- start 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 -- start start 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a8100580 0x7fe3a81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 0x7fe3a81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8102130 0x7fe3a819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a810c940 con 0x7fe3a8101770 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a810c7c0 con 0x7fe3a8100580 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a810cac0 con 0x7fe3a8102130 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.813+0000 7fe3a5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 0x7fe3a81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 0x7fe3a81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39266/0 (socket says 172.21.15.57:39266) 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 -- 172.21.15.57:0/2832527362 learned_addr learned my addr 172.21.15.57:0/2832527362 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5e00640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a8100580 0x7fe3a81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a6800640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8102130 0x7fe3a819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:51.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 -- 172.21.15.57:0/2832527362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8102130 msgr2=0x7fe3a819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:51.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8102130 0x7fe3a819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 -- 172.21.15.57:0/2832527362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a8100580 msgr2=0x7fe3a81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:51.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a8100580 0x7fe3a81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:51.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 -- 172.21.15.57:0/2832527362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a819fcd0 con 0x7fe3a8101770 2024-04-13T05:14:51.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a6800640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8102130 0x7fe3a819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:51.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.814+0000 7fe3a5400640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 0x7fe3a81985f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe39400d770 tx=0x7fe39400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:51.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.815+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394017070 con 0x7fe3a8101770 2024-04-13T05:14:51.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.815+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3940054d0 con 0x7fe3a8101770 2024-04-13T05:14:51.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.815+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a819ffc0 con 0x7fe3a8101770 2024-04-13T05:14:51.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.815+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394004cf0 con 0x7fe3a8101770 2024-04-13T05:14:51.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.815+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a81a0480 con 0x7fe3a8101770 2024-04-13T05:14:51.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.816+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a81a06b0 con 0x7fe3a8101770 2024-04-13T05:14:51.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.819+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe394004e90 con 0x7fe3a8101770 2024-04-13T05:14:51.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.821+0000 7fe39ea00640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe38c0a3c30 0x7fe38c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:51.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.822+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fe3940cfa90 con 0x7fe3a8101770 2024-04-13T05:14:51.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.822+0000 7fe3a5e00640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe38c0a3c30 0x7fe38c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:51.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.823+0000 7fe3a5e00640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe38c0a3c30 0x7fe38c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe398005fd0 tx=0x7fe39800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:51.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:51.827+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe39409c700 con 0x7fe3a8101770 2024-04-13T05:14:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.041+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fe3a81a0920 con 0x7fe3a8101770 2024-04-13T05:14:52.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.565+0000 7fe39ea00640 1 -- 172.21.15.57:0/2832527362 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7fe39409c000 con 0x7fe3a8101770 2024-04-13T05:14:52.567 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2024-04-13T05:14:52.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe38c0a3c30 msgr2=0x7fe38c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:52.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe38c0a3c30 0x7fe38c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe398005fd0 tx=0x7fe39800b410 comp rx=0 tx=0).stop 2024-04-13T05:14:52.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 msgr2=0x7fe3a81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:52.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 0x7fe3a81985f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe39400d770 tx=0x7fe39400dc30 comp rx=0 tx=0).stop 2024-04-13T05:14:52.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 shutdown_connections 2024-04-13T05:14:52.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe38c0a3c30 0x7fe38c0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8102130 0x7fe3a819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3a8101770 0x7fe3a81985f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 --2- 172.21.15.57:0/2832527362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3a8100580 0x7fe3a81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 >> 172.21.15.57:0/2832527362 conn(0x7fe3a80fbaf0 msgr2=0x7fe3a80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:52.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 shutdown_connections 2024-04-13T05:14:52.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.569+0000 7fe3a7200640 1 -- 172.21.15.57:0/2832527362 wait complete. 2024-04-13T05:14:52.615 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 -- 172.21.15.57:0/2151428909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4102af0 msgr2=0x7f9cc4104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/2151428909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4102af0 0x7f9cc4104ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009900 tx=0x7f9cb402bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 -- 172.21.15.57:0/2151428909 shutdown_connections 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/2151428909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc4107d50 0x7f9cc410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/2151428909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc4105420 0x7f9cc4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/2151428909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4102af0 0x7f9cc4104ee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 -- 172.21.15.57:0/2151428909 >> 172.21.15.57:0/2151428909 conn(0x7f9cc40fc4a0 msgr2=0x7f9cc40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 -- 172.21.15.57:0/2151428909 shutdown_connections 2024-04-13T05:14:52.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.804+0000 7f9cc9c00640 1 -- 172.21.15.57:0/2151428909 wait complete. 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.805+0000 7f9cc9c00640 1 Processor -- start 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 -- start start 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc4102af0 0x7f9cc4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc4105420 0x7f9cc4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 0x7f9cc419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc410dba0 con 0x7f9cc4107d50 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc410da20 con 0x7f9cc4102af0 2024-04-13T05:14:52.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc9c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc410dd20 con 0x7f9cc4105420 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 0x7f9cc419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 0x7f9cc419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39282/0 (socket says 172.21.15.57:39282) 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc3400640 1 -- 172.21.15.57:0/3170045792 learned_addr learned my addr 172.21.15.57:0/3170045792 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.806+0000 7f9cc2a00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc4102af0 0x7f9cc4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc2000640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc4105420 0x7f9cc4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc3400640 1 -- 172.21.15.57:0/3170045792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc4105420 msgr2=0x7f9cc4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc3400640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc4105420 0x7f9cc4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc3400640 1 -- 172.21.15.57:0/3170045792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc4102af0 msgr2=0x7f9cc4198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc3400640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc4102af0 0x7f9cc4198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:52.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc3400640 1 -- 172.21.15.57:0/3170045792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc419fe30 con 0x7f9cc4107d50 2024-04-13T05:14:52.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cc3400640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 0x7f9cc419f7c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00eb10 tx=0x7f9cbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:52.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.807+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004b20 con 0x7f9cc4107d50 2024-04-13T05:14:52.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.808+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc41a0120 con 0x7f9cc4107d50 2024-04-13T05:14:52.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.808+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cbc004cc0 con 0x7f9cc4107d50 2024-04-13T05:14:52.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.808+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc010690 con 0x7f9cc4107d50 2024-04-13T05:14:52.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.808+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc41a0670 con 0x7f9cc4107d50 2024-04-13T05:14:52.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.809+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc40fdf90 con 0x7f9cc4107d50 2024-04-13T05:14:52.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.811+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9cbc005000 con 0x7f9cc4107d50 2024-04-13T05:14:52.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.813+0000 7f9cbb400640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ca80a3bb0 0x7f9ca80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:52.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.813+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f9cbc0d0b30 con 0x7f9cc4107d50 2024-04-13T05:14:52.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.813+0000 7f9cc2a00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ca80a3bb0 0x7f9ca80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:52.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.814+0000 7f9cc2a00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ca80a3bb0 0x7f9ca80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4000c00 tx=0x7f9cb402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:52.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:52.821+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cbc053ff0 con 0x7f9cc4107d50 2024-04-13T05:14:52.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:52.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/551441355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-13T05:14:52.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: osdmap e83: 12 total, 12 up, 12 in 2024-04-13T05:14:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: pgmap v363: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:14:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2832527362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-13T05:14:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:14:53.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/551441355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: osdmap e83: 12 total, 12 up, 12 in 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: pgmap v363: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2832527362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-13T05:14:53.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:14:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:53.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/551441355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: osdmap e83: 12 total, 12 up, 12 in 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: pgmap v363: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2832527362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-13T05:14:53.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:14:53.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.070+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9cc4062bc0 con 0x7f9cc4107d50 2024-04-13T05:14:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.594+0000 7f9cbb400640 1 -- 172.21.15.57:0/3170045792 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9cbc09d620 con 0x7f9cc4107d50 2024-04-13T05:14:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 2 and data pool 3 2024-04-13T05:14:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.599+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ca80a3bb0 msgr2=0x7f9ca80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:53.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.599+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ca80a3bb0 0x7f9ca80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4000c00 tx=0x7f9cb402c7e0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.599+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 msgr2=0x7f9cc419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.599+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 0x7f9cc419f7c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00eb10 tx=0x7f9cbc00c2d0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.600+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 shutdown_connections 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.600+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ca80a3bb0 0x7f9ca80a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.600+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9cc4107d50 0x7f9cc419f7c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.600+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc4105420 0x7f9cc4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.600+0000 7f9cc9c00640 1 --2- 172.21.15.57:0/3170045792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cc4102af0 0x7f9cc4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.600+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 >> 172.21.15.57:0/3170045792 conn(0x7f9cc40fc4a0 msgr2=0x7f9cc4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.601+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 shutdown_connections 2024-04-13T05:14:53.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.601+0000 7f9cc9c00640 1 -- 172.21.15.57:0/3170045792 wait complete. 2024-04-13T05:14:53.633 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-13T05:14:53.633 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-13T05:14:53.815 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:53 smithi057 ceph-mon[32331]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-13T05:14:53.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:53 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2832527362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-13T05:14:53.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:53 smithi057 ceph-mon[32331]: osdmap e84: 12 total, 12 up, 12 in 2024-04-13T05:14:53.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:53 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3170045792' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-13T05:14:53.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:53 smithi057 ceph-mon[32331]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-13T05:14:53.818 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:53 smithi189 ceph-mon[35531]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-13T05:14:53.818 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:53 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2832527362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-13T05:14:53.818 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:53 smithi189 ceph-mon[35531]: osdmap e84: 12 total, 12 up, 12 in 2024-04-13T05:14:53.819 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:53 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3170045792' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-13T05:14:53.819 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:53 smithi189 ceph-mon[35531]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-13T05:14:53.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 -- 172.21.15.57:0/2080924534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220103a90 msgr2=0x7f622010a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 --2- 172.21.15.57:0/2080924534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220103a90 0x7f622010a3b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6218009ca0 tx=0x7f621802c100 comp rx=0 tx=0).stop 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 -- 172.21.15.57:0/2080924534 shutdown_connections 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 --2- 172.21.15.57:0/2080924534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 0x7f6220103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 --2- 172.21.15.57:0/2080924534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220103a90 0x7f622010a3b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 --2- 172.21.15.57:0/2080924534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220101f00 0x7f6220102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 -- 172.21.15.57:0/2080924534 >> 172.21.15.57:0/2080924534 conn(0x7f62200fd4c0 msgr2=0x7f62200ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 -- 172.21.15.57:0/2080924534 shutdown_connections 2024-04-13T05:14:53.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.822+0000 7f6226200640 1 -- 172.21.15.57:0/2080924534 wait complete. 2024-04-13T05:14:53.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.823+0000 7f6226200640 1 Processor -- start 2024-04-13T05:14:53.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 -- start start 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 0x7f622019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 0x7f622019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220103a90 0x7f62201a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f622010d9e0 con 0x7f6220101f00 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f622010d860 con 0x7f62201030d0 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6226200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f622010db60 con 0x7f6220103a90 2024-04-13T05:14:53.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6224e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 0x7f622019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:53.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6224e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 0x7f622019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39304/0 (socket says 172.21.15.57:39304) 2024-04-13T05:14:53.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.824+0000 7f6224e00640 1 -- 172.21.15.57:0/2245800126 learned_addr learned my addr 172.21.15.57:0/2245800126 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:53.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6225800640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220103a90 0x7f62201a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:53.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f621fe00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 0x7f622019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6224e00640 1 -- 172.21.15.57:0/2245800126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220103a90 msgr2=0x7f62201a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6224e00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220103a90 0x7f62201a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6224e00640 1 -- 172.21.15.57:0/2245800126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 msgr2=0x7f622019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6224e00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 0x7f622019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6224e00640 1 -- 172.21.15.57:0/2245800126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62201a1eb0 con 0x7f6220101f00 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f621fe00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 0x7f622019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:53.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6225800640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220103a90 0x7f62201a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.825+0000 7f6224e00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 0x7f622019a290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f621400e9c0 tx=0x7f621400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.826+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400cd80 con 0x7f6220101f00 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.826+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62140035f0 con 0x7f6220101f00 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.826+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214010690 con 0x7f6220101f00 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.826+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62201a21a0 con 0x7f6220101f00 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.826+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62201a26a0 con 0x7f6220101f00 2024-04-13T05:14:53.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.828+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62201a29c0 con 0x7f6220101f00 2024-04-13T05:14:53.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.830+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f62140037a0 con 0x7f6220101f00 2024-04-13T05:14:53.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.831+0000 7f621d600640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62080a3c30 0x7f62080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:53.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.832+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f62140d06c0 con 0x7f6220101f00 2024-04-13T05:14:53.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.832+0000 7f621fe00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62080a3c30 0x7f62080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:53.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.833+0000 7f621fe00640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62080a3c30 0x7f62080a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f620c0058c0 tx=0x7f620c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:53.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:53.837+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f621409d150 con 0x7f6220101f00 2024-04-13T05:14:54.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:53 smithi083 ceph-mon[35133]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-13T05:14:54.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:53 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2832527362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-13T05:14:54.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:53 smithi083 ceph-mon[35133]: osdmap e84: 12 total, 12 up, 12 in 2024-04-13T05:14:54.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:53 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3170045792' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-13T05:14:54.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:53 smithi083 ceph-mon[35133]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-13T05:14:54.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.049+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f62201a2b90 con 0x7f6220101f00 2024-04-13T05:14:54.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.566+0000 7f621d600640 1 -- 172.21.15.57:0/2245800126 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f621409ca50 con 0x7f6220101f00 2024-04-13T05:14:54.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.569+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62080a3c30 msgr2=0x7f62080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.569+0000 7f6226200640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62080a3c30 0x7f62080a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f620c0058c0 tx=0x7f620c005850 comp rx=0 tx=0).stop 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.569+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 msgr2=0x7f622019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.569+0000 7f6226200640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 0x7f622019a290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f621400e9c0 tx=0x7f621400ee80 comp rx=0 tx=0).stop 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 shutdown_connections 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62080a3c30 0x7f62080a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62201030d0 0x7f622019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6220103a90 0x7f62201a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 --2- 172.21.15.57:0/2245800126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6220101f00 0x7f622019a290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 >> 172.21.15.57:0/2245800126 conn(0x7f62200fd4c0 msgr2=0x7f6220108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:54.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 shutdown_connections 2024-04-13T05:14:54.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.570+0000 7f6226200640 1 -- 172.21.15.57:0/2245800126 wait complete. 2024-04-13T05:14:54.593 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-13T05:14:54.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:54 smithi057 ceph-mon[32331]: osdmap e85: 12 total, 12 up, 12 in 2024-04-13T05:14:54.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:54 smithi057 ceph-mon[32331]: pgmap v366: 97 pgs: 19 creating+peering, 60 unknown, 18 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:54.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:54 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3170045792' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-13T05:14:54.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:54 smithi057 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-13T05:14:54.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:54 smithi057 ceph-mon[32331]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-13T05:14:54.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:54 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2245800126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-13T05:14:54.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 -- 172.21.15.57:0/1117032140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc1051f0 msgr2=0x7fe4cc1075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1117032140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc1075e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009900 tx=0x7fe4b802bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 -- 172.21.15.57:0/1117032140 shutdown_connections 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1117032140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc107b20 0x7fe4cc109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1117032140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc1075e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1117032140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4cc1028c0 0x7fe4cc104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 -- 172.21.15.57:0/1117032140 >> 172.21.15.57:0/1117032140 conn(0x7fe4cc0fc290 msgr2=0x7fe4cc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 -- 172.21.15.57:0/1117032140 shutdown_connections 2024-04-13T05:14:54.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.801+0000 7fe4cb200640 1 -- 172.21.15.57:0/1117032140 wait complete. 2024-04-13T05:14:54.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.802+0000 7fe4cb200640 1 Processor -- start 2024-04-13T05:14:54.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 -- start start 2024-04-13T05:14:54.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4cc1028c0 0x7fe4cc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:54.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:54.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 0x7fe4cc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:54.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10d9d0 con 0x7fe4cc107b20 2024-04-13T05:14:54.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10d850 con 0x7fe4cc1051f0 2024-04-13T05:14:54.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4cb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10db50 con 0x7fe4cc1028c0 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4ca800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 0x7fe4cc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4c9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4cc1028c0 0x7fe4cc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4ca800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 0x7fe4cc1a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39318/0 (socket says 172.21.15.57:39318) 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.803+0000 7fe4ca800640 1 -- 172.21.15.57:0/1004944690 learned_addr learned my addr 172.21.15.57:0/1004944690 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4c9400640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4ca800640 1 -- 172.21.15.57:0/1004944690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4cc1028c0 msgr2=0x7fe4cc19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4ca800640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4cc1028c0 0x7fe4cc19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4ca800640 1 -- 172.21.15.57:0/1004944690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc1051f0 msgr2=0x7fe4cc19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4ca800640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4ca800640 1 -- 172.21.15.57:0/1004944690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc1a1f00 con 0x7fe4cc107b20 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4c9400640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:54.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4ca800640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 0x7fe4cc1a1890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400eb10 tx=0x7fe4c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:54.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.804+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4004b20 con 0x7fe4cc107b20 2024-04-13T05:14:54.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.805+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c4004cc0 con 0x7fe4cc107b20 2024-04-13T05:14:54.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.805+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4010690 con 0x7fe4cc107b20 2024-04-13T05:14:54.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.805+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc1a21f0 con 0x7fe4cc107b20 2024-04-13T05:14:54.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.805+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a26c0 con 0x7fe4cc107b20 2024-04-13T05:14:54.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.806+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc109670 con 0x7fe4cc107b20 2024-04-13T05:14:54.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.809+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe4c4005020 con 0x7fe4cc107b20 2024-04-13T05:14:54.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.811+0000 7fe4c2a00640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4b00a3c30 0x7fe4b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:54.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.811+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fe4c40d0a30 con 0x7fe4cc107b20 2024-04-13T05:14:54.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.811+0000 7fe4c9e00640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4b00a3c30 0x7fe4b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:54.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.812+0000 7fe4c9e00640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4b00a3c30 0x7fe4b00a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009bd0 tx=0x7fe4bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:54.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:54.815+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4c409d380 con 0x7fe4cc107b20 2024-04-13T05:14:54.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:54 smithi189 ceph-mon[35531]: osdmap e85: 12 total, 12 up, 12 in 2024-04-13T05:14:54.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:54 smithi189 ceph-mon[35531]: pgmap v366: 97 pgs: 19 creating+peering, 60 unknown, 18 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:54.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:54 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3170045792' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-13T05:14:54.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:54 smithi189 ceph-mon[35531]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-13T05:14:54.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:54 smithi189 ceph-mon[35531]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-13T05:14:54.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:54 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2245800126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-13T05:14:55.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:54 smithi083 ceph-mon[35133]: osdmap e85: 12 total, 12 up, 12 in 2024-04-13T05:14:55.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:54 smithi083 ceph-mon[35133]: pgmap v366: 97 pgs: 19 creating+peering, 60 unknown, 18 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:14:55.000 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:54 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3170045792' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-13T05:14:55.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:54 smithi083 ceph-mon[35133]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-13T05:14:55.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:54 smithi083 ceph-mon[35133]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-13T05:14:55.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:54 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2245800126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-13T05:14:55.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:55.059+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe4cc062bc0 con 0x7fe4cc107b20 2024-04-13T05:14:55.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: Cluster is now healthy 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2245800126' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: osdmap e86: 12 total, 12 up, 12 in 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] up:active 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1004944690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:14:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: Cluster is now healthy 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2245800126' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: osdmap e86: 12 total, 12 up, 12 in 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] up:active 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:14:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1004944690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:14:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-13T05:14:56.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: Cluster is now healthy 2024-04-13T05:14:56.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2245800126' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-13T05:14:56.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: osdmap e86: 12 total, 12 up, 12 in 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] up:active 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1004944690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:14:56.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:14:56.003 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:14:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.062+0000 7fe4c2a00640 1 -- 172.21.15.57:0/1004944690 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fe4c409cc80 con 0x7fe4cc107b20 2024-04-13T05:14:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data_ec' created 2024-04-13T05:14:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.065+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4b00a3c30 msgr2=0x7fe4b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.065+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4b00a3c30 0x7fe4b00a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009bd0 tx=0x7fe4bc005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.065+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 msgr2=0x7fe4cc1a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.065+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 0x7fe4cc1a1890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400eb10 tx=0x7fe4c400c2d0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 shutdown_connections 2024-04-13T05:14:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4b00a3c30 0x7fe4b00a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4cc1051f0 0x7fe4cc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4cc107b20 0x7fe4cc1a1890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 --2- 172.21.15.57:0/1004944690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4cc1028c0 0x7fe4cc19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 >> 172.21.15.57:0/1004944690 conn(0x7fe4cc0fc290 msgr2=0x7fe4cc102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.066+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 shutdown_connections 2024-04-13T05:14:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.067+0000 7fe4cb200640 1 -- 172.21.15.57:0/1004944690 wait complete. 2024-04-13T05:14:57.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-13T05:14:57.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.300+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1109409404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 msgr2=0x7fd9a01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.300+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1109409404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a01022c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd990009900 tx=0x7fd99002bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1109409404 shutdown_connections 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1109409404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103a40 0x7fd9a010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1109409404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9a0103080 0x7fd9a0103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1109409404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a01022c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1109409404 >> 172.21.15.57:0/1109409404 conn(0x7fd9a00fd460 msgr2=0x7fd9a00ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1109409404 shutdown_connections 2024-04-13T05:14:57.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.301+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1109409404 wait complete. 2024-04-13T05:14:57.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.302+0000 7fd9a4e00640 1 Processor -- start 2024-04-13T05:14:57.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.302+0000 7fd9a4e00640 1 -- start start 2024-04-13T05:14:57.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd9a4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:57.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd9a4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103080 0x7fd9a0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:57.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd9a4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9a0103a40 0x7fd9a019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:57.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd9a4e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a010da40 con 0x7fd9a0101ea0 2024-04-13T05:14:57.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd9a4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a010d8c0 con 0x7fd9a0103080 2024-04-13T05:14:57.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd9a4e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a010dbc0 con 0x7fd9a0103a40 2024-04-13T05:14:57.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:57.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a0198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33500/0 (socket says 172.21.15.57:33500) 2024-04-13T05:14:57.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99f400640 1 -- 172.21.15.57:0/1383858101 learned_addr learned my addr 172.21.15.57:0/1383858101 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:57.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99fe00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9a0103a40 0x7fd9a019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:57.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99ea00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103080 0x7fd9a0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:57.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99f400640 1 -- 172.21.15.57:0/1383858101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9a0103a40 msgr2=0x7fd9a019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:57.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99f400640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9a0103a40 0x7fd9a019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.303+0000 7fd99f400640 1 -- 172.21.15.57:0/1383858101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103080 msgr2=0x7fd9a0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd99f400640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103080 0x7fd9a0198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd99f400640 1 -- 172.21.15.57:0/1383858101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a019fd60 con 0x7fd9a0101ea0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd99ea00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103080 0x7fd9a0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd99f400640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a0198140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd990000c00 tx=0x7fd990004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990043020 con 0x7fd9a0101ea0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd990004440 con 0x7fd9a0101ea0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990003c90 con 0x7fd9a0101ea0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a019fff0 con 0x7fd9a0101ea0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.304+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a01a0450 con 0x7fd9a0101ea0 2024-04-13T05:14:57.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.306+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a0068d30 con 0x7fd9a0101ea0 2024-04-13T05:14:57.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.308+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd990046020 con 0x7fd9a0101ea0 2024-04-13T05:14:57.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.309+0000 7fd997e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd9840a3bb0 0x7fd9840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:57.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.309+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fd9900f2df0 con 0x7fd9a0101ea0 2024-04-13T05:14:57.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.309+0000 7fd99ea00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd9840a3bb0 0x7fd9840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:57.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.310+0000 7fd99ea00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd9840a3bb0 0x7fd9840a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0199650 tx=0x7fd988005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:57.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.313+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9a0068d30 con 0x7fd9a0101ea0 2024-04-13T05:14:57.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:57 smithi189 ceph-mon[35531]: pgmap v368: 97 pgs: 19 creating+peering, 8 unknown, 70 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-04-13T05:14:57.345 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:57 smithi189 ceph-mon[35531]: osdmap e87: 12 total, 12 up, 12 in 2024-04-13T05:14:57.501 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:14:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: 2024-04-13T05:14:57.121+0000 7f9d7b800700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-13T05:14:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:57 smithi083 ceph-mon[35133]: pgmap v368: 97 pgs: 19 creating+peering, 8 unknown, 70 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-04-13T05:14:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:57 smithi083 ceph-mon[35133]: osdmap e87: 12 total, 12 up, 12 in 2024-04-13T05:14:57.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:57.519+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd9a0109360 con 0x7fd9a0101ea0 2024-04-13T05:14:57.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:57 smithi057 ceph-mon[32331]: pgmap v368: 97 pgs: 19 creating+peering, 8 unknown, 70 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-04-13T05:14:57.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:57 smithi057 ceph-mon[32331]: osdmap e87: 12 total, 12 up, 12 in 2024-04-13T05:14:58.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.066+0000 7fd997e00640 1 -- 172.21.15.57:0/1383858101 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fd9a0109360 con 0x7fd9a0101ea0 2024-04-13T05:14:58.069 INFO:teuthology.orchestra.run.smithi057.stderr:set pool 4 allow_ec_overwrites to true 2024-04-13T05:14:58.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.070+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd9840a3bb0 msgr2=0x7fd9840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:58.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.070+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd9840a3bb0 0x7fd9840a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0199650 tx=0x7fd988005f90 comp rx=0 tx=0).stop 2024-04-13T05:14:58.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.070+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 msgr2=0x7fd9a0198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:58.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.070+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a0198140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd990000c00 tx=0x7fd990004040 comp rx=0 tx=0).stop 2024-04-13T05:14:58.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 shutdown_connections 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd9840a3bb0 0x7fd9840a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a0103080 0x7fd9a0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9a0103a40 0x7fd9a019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 --2- 172.21.15.57:0/1383858101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9a0101ea0 0x7fd9a0198140 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 >> 172.21.15.57:0/1383858101 conn(0x7fd9a00fd460 msgr2=0x7fd9a01064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 shutdown_connections 2024-04-13T05:14:58.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.071+0000 7fd9a4e00640 1 -- 172.21.15.57:0/1383858101 wait complete. 2024-04-13T05:14:58.112 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-13T05:14:58.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.303+0000 7fb67a000640 1 -- 172.21.15.57:0/3676489485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 msgr2=0x7fb674102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.303+0000 7fb67a000640 1 --2- 172.21.15.57:0/3676489485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb674102350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb660009900 tx=0x7fb66002bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 -- 172.21.15.57:0/3676489485 shutdown_connections 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 --2- 172.21.15.57:0/3676489485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674102890 0x7fb674109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 --2- 172.21.15.57:0/3676489485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb674102350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 --2- 172.21.15.57:0/3676489485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674100ce0 0x7fb674101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 -- 172.21.15.57:0/3676489485 >> 172.21.15.57:0/3676489485 conn(0x7fb6740fc290 msgr2=0x7fb6740fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 -- 172.21.15.57:0/3676489485 shutdown_connections 2024-04-13T05:14:58.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.304+0000 7fb67a000640 1 -- 172.21.15.57:0/3676489485 wait complete. 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.305+0000 7fb67a000640 1 Processor -- start 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.305+0000 7fb67a000640 1 -- start start 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb67a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674100ce0 0x7fb674104c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb67a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb6741032b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb67a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674102890 0x7fb6741037f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb67a000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb67410c860 con 0x7fb674101ed0 2024-04-13T05:14:58.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb67a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb67410c6e0 con 0x7fb674100ce0 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb67a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb67410c9e0 con 0x7fb674102890 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb673e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb6741032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb673e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb6741032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33526/0 (socket says 172.21.15.57:33526) 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb673e00640 1 -- 172.21.15.57:0/1274017908 learned_addr learned my addr 172.21.15.57:0/1274017908 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb678c00640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674100ce0 0x7fb674104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.306+0000 7fb679600640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674102890 0x7fb6741037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb673e00640 1 -- 172.21.15.57:0/1274017908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674102890 msgr2=0x7fb6741037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb673e00640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674102890 0x7fb6741037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb673e00640 1 -- 172.21.15.57:0/1274017908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674100ce0 msgr2=0x7fb674104c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb673e00640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674100ce0 0x7fb674104c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb673e00640 1 -- 172.21.15.57:0/1274017908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb674104080 con 0x7fb674101ed0 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb679600640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674102890 0x7fb6741037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:58.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.307+0000 7fb673e00640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb6741032b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb660000c00 tx=0x7fb6600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:58.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.308+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660043020 con 0x7fb674101ed0 2024-04-13T05:14:58.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.308+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6600054c0 con 0x7fb674101ed0 2024-04-13T05:14:58.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.308+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660003d70 con 0x7fb674101ed0 2024-04-13T05:14:58.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.308+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6741a2180 con 0x7fb674101ed0 2024-04-13T05:14:58.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.308+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6741a2680 con 0x7fb674101ed0 2024-04-13T05:14:58.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.309+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6741a2880 con 0x7fb674101ed0 2024-04-13T05:14:58.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.312+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb660046020 con 0x7fb674101ed0 2024-04-13T05:14:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.315+0000 7fb671600640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb65c0a3c30 0x7fb65c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.316+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fb6600f2a60 con 0x7fb674101ed0 2024-04-13T05:14:58.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.316+0000 7fb678c00640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb65c0a3c30 0x7fb65c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:58.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.317+0000 7fb678c00640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb65c0a3c30 0x7fb65c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb668005fd0 tx=0x7fb66800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:58.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.319+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb66004d020 con 0x7fb674101ed0 2024-04-13T05:14:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1004944690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-13T05:14:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:58 smithi083 ceph-mon[35133]: osdmap e88: 12 total, 12 up, 12 in 2024-04-13T05:14:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1383858101' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-13T05:14:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1383858101' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-13T05:14:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:58 smithi083 ceph-mon[35133]: osdmap e89: 12 total, 12 up, 12 in 2024-04-13T05:14:58.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1004944690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-13T05:14:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:58 smithi057 ceph-mon[32331]: osdmap e88: 12 total, 12 up, 12 in 2024-04-13T05:14:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1383858101' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-13T05:14:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1383858101' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-13T05:14:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:58 smithi057 ceph-mon[32331]: osdmap e89: 12 total, 12 up, 12 in 2024-04-13T05:14:58.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:58.567+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fb6741a2ab0 con 0x7fb674101ed0 2024-04-13T05:14:58.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1004944690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-13T05:14:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:58 smithi189 ceph-mon[35531]: osdmap e88: 12 total, 12 up, 12 in 2024-04-13T05:14:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1383858101' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-13T05:14:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1383858101' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-13T05:14:58.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:58 smithi189 ceph-mon[35531]: osdmap e89: 12 total, 12 up, 12 in 2024-04-13T05:14:59.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.122+0000 7fb671600640 1 -- 172.21.15.57:0/1274017908 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fb6600bf2e0 con 0x7fb674101ed0 2024-04-13T05:14:59.125 INFO:teuthology.orchestra.run.smithi057.stderr:added data pool 4 to fsmap 2024-04-13T05:14:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.126+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb65c0a3c30 msgr2=0x7fb65c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.126+0000 7fb67a000640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb65c0a3c30 0x7fb65c0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb668005fd0 tx=0x7fb66800b390 comp rx=0 tx=0).stop 2024-04-13T05:14:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.126+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 msgr2=0x7fb6741032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.126+0000 7fb67a000640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb6741032b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb660000c00 tx=0x7fb6600042f0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 shutdown_connections 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb65c0a3c30 0x7fb65c0a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb674102890 0x7fb6741037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb674101ed0 0x7fb6741032b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 --2- 172.21.15.57:0/1274017908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb674100ce0 0x7fb674104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 >> 172.21.15.57:0/1274017908 conn(0x7fb6740fc290 msgr2=0x7fb6740fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 shutdown_connections 2024-04-13T05:14:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.127+0000 7fb67a000640 1 -- 172.21.15.57:0/1274017908 wait complete. 2024-04-13T05:14:59.154 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:14:59.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.360+0000 7f4a63e00640 1 -- 172.21.15.57:0/314682515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64105250 msgr2=0x7f4a641056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.360+0000 7f4a63e00640 1 --2- 172.21.15.57:0/314682515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64105250 0x7f4a641056d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009900 tx=0x7f4a4c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 -- 172.21.15.57:0/314682515 shutdown_connections 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 --2- 172.21.15.57:0/314682515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a64105c10 0x7f4a6410c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 --2- 172.21.15.57:0/314682515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64105250 0x7f4a641056d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 --2- 172.21.15.57:0/314682515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a64104060 0x7f4a64104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 -- 172.21.15.57:0/314682515 >> 172.21.15.57:0/314682515 conn(0x7f4a640ff5d0 msgr2=0x7f4a64101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 -- 172.21.15.57:0/314682515 shutdown_connections 2024-04-13T05:14:59.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.361+0000 7f4a63e00640 1 -- 172.21.15.57:0/314682515 wait complete. 2024-04-13T05:14:59.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 Processor -- start 2024-04-13T05:14:59.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 -- start start 2024-04-13T05:14:59.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 0x7f4a6419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a64105250 0x7f4a6419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a64105c10 0x7f4a641a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410fbf0 con 0x7f4a64104060 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410fa70 con 0x7f4a64105250 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a63e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410fd70 con 0x7f4a64105c10 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.362+0000 7f4a62a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 0x7f4a6419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 0x7f4a6419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33536/0 (socket says 172.21.15.57:33536) 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 -- 172.21.15.57:0/1341120153 learned_addr learned my addr 172.21.15.57:0/1341120153 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a5a000640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a64105250 0x7f4a6419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a63400640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a64105c10 0x7f4a641a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 -- 172.21.15.57:0/1341120153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a64105c10 msgr2=0x7f4a641a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a64105c10 0x7f4a641a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 -- 172.21.15.57:0/1341120153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a64105250 msgr2=0x7f4a6419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a64105250 0x7f4a6419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 -- 172.21.15.57:0/1341120153 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641a1eb0 con 0x7f4a64104060 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a62a00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 0x7f4a6419a290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d950 tx=0x7f4a5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54017070 con 0x7f4a64104060 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.363+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a54009c30 con 0x7f4a64104060 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.364+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a641a21a0 con 0x7f4a64104060 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.364+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400f620 con 0x7f4a64104060 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.364+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641a2670 con 0x7f4a64104060 2024-04-13T05:14:59.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.365+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6410b450 con 0x7f4a64104060 2024-04-13T05:14:59.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.367+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4a5401a070 con 0x7f4a64104060 2024-04-13T05:14:59.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.368+0000 7f4a5be00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a480a3bb0 0x7f4a480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.368+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4a540d0160 con 0x7f4a64104060 2024-04-13T05:14:59.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.368+0000 7f4a5a000640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a480a3bb0 0x7f4a480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.369+0000 7f4a5a000640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a480a3bb0 0x7f4a480a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c006010 tx=0x7f4a4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:59.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.373+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a54009de0 con 0x7f4a64104060 2024-04-13T05:14:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:59 smithi083 ceph-mon[35133]: pgmap v371: 129 pgs: 32 unknown, 19 creating+peering, 78 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-04-13T05:14:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:59 smithi083 ceph-mon[35133]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-13T05:14:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:14:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1274017908' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-13T05:14:59.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:59 smithi057 ceph-mon[32331]: pgmap v371: 129 pgs: 32 unknown, 19 creating+peering, 78 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-04-13T05:14:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:59 smithi057 ceph-mon[32331]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-13T05:14:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:14:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1274017908' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-13T05:14:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:59 smithi189 ceph-mon[35531]: pgmap v371: 129 pgs: 32 unknown, 19 creating+peering, 78 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-04-13T05:14:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:59 smithi189 ceph-mon[35531]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-13T05:14:59.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:14:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1274017908' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-13T05:14:59.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.617+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a6402d5e0 con 0x7f4a64104060 2024-04-13T05:14:59.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.619+0000 7f4a5be00640 1 -- 172.21.15.57:0/1341120153 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11782 (secure 0 0 0) 0x7f4a5409c900 con 0x7f4a64104060 2024-04-13T05:14:59.620 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:14:59.621 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:14:58.570497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34499,"qdb_cluster":[34499]},"id":1}]} 2024-04-13T05:14:59.621 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-04-13T05:14:59.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.621+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a480a3bb0 msgr2=0x7f4a480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.621+0000 7f4a63e00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a480a3bb0 0x7f4a480a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c006010 tx=0x7f4a4c000bc0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.621+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 msgr2=0x7f4a6419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.621+0000 7f4a63e00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 0x7f4a6419a290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d950 tx=0x7f4a5400de10 comp rx=0 tx=0).stop 2024-04-13T05:14:59.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 shutdown_connections 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a480a3bb0 0x7f4a480a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a64105250 0x7f4a6419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a64105c10 0x7f4a641a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 --2- 172.21.15.57:0/1341120153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a64104060 0x7f4a6419a290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 >> 172.21.15.57:0/1341120153 conn(0x7f4a640ff5d0 msgr2=0x7f4a64108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 shutdown_connections 2024-04-13T05:14:59.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.622+0000 7f4a63e00640 1 -- 172.21.15.57:0/1341120153 wait complete. 2024-04-13T05:14:59.644 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:14:59.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 -- 172.21.15.57:0/2259118004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870102fb0 msgr2=0x7f8870103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 --2- 172.21.15.57:0/2259118004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870102fb0 0x7f8870103430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8858009900 tx=0x7f885802bc30 comp rx=0 tx=0).stop 2024-04-13T05:14:59.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 -- 172.21.15.57:0/2259118004 shutdown_connections 2024-04-13T05:14:59.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 --2- 172.21.15.57:0/2259118004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870103970 0x7f887010a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 --2- 172.21.15.57:0/2259118004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870102fb0 0x7f8870103430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 --2- 172.21.15.57:0/2259118004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870101e60 0x7f8870102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 -- 172.21.15.57:0/2259118004 >> 172.21.15.57:0/2259118004 conn(0x7f88700fd3d0 msgr2=0x7f88700ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:14:59.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 -- 172.21.15.57:0/2259118004 shutdown_connections 2024-04-13T05:14:59.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.862+0000 7f8875600640 1 -- 172.21.15.57:0/2259118004 wait complete. 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.863+0000 7f8875600640 1 Processor -- start 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 -- start start 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870101e60 0x7f8870075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870102fb0 0x7f88700743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 0x7f88700748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f887010da00 con 0x7f8870103970 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f887010d880 con 0x7f8870101e60 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8875600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f887010db80 con 0x7f8870102fb0 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8874c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 0x7f88700748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8874c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 0x7f88700748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33564/0 (socket says 172.21.15.57:33564) 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8874c00640 1 -- 172.21.15.57:0/1670333661 learned_addr learned my addr 172.21.15.57:0/1670333661 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f8867400640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870102fb0 0x7f88700743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.864+0000 7f886fe00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870101e60 0x7f8870075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8874c00640 1 -- 172.21.15.57:0/1670333661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870102fb0 msgr2=0x7f88700743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8874c00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870102fb0 0x7f88700743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8874c00640 1 -- 172.21.15.57:0/1670333661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870101e60 msgr2=0x7f8870075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8874c00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870101e60 0x7f8870075d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8874c00640 1 -- 172.21.15.57:0/1670333661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8870075180 con 0x7f8870103970 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8867400640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870102fb0 0x7f88700743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f886fe00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870101e60 0x7f8870075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8874c00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 0x7f88700748f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f886800d750 tx=0x7f886800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868017070 con 0x7f8870103970 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88680054d0 con 0x7f8870103970 2024-04-13T05:14:59.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.865+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88701a21a0 con 0x7f8870103970 2024-04-13T05:14:59.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.866+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868004c60 con 0x7f8870103970 2024-04-13T05:14:59.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.866+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88701a2670 con 0x7f8870103970 2024-04-13T05:14:59.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.867+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88701091f0 con 0x7f8870103970 2024-04-13T05:14:59.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.869+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8868004e00 con 0x7f8870103970 2024-04-13T05:14:59.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.871+0000 7f886d600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88540a3bb0 0x7f88540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:14:59.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.872+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f88680d0520 con 0x7f8870103970 2024-04-13T05:14:59.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.872+0000 7f886fe00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88540a3bb0 0x7f88540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:14:59.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.872+0000 7f886fe00640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88540a3bb0 0x7f88540a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8860005fd0 tx=0x7f8860009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:14:59.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:14:59.874+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f886809cc60 con 0x7f8870103970 2024-04-13T05:15:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.081+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8870062bc0 con 0x7f8870103970 2024-04-13T05:15:00.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.083+0000 7f886d600640 1 -- 172.21.15.57:0/1670333661 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24054 (secure 0 0 0) 0x7f886809c560 con 0x7f8870103970 2024-04-13T05:15:00.085 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:00.086 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:14:58.570092+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88540a3bb0 msgr2=0x7f88540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88540a3bb0 0x7f88540a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8860005fd0 tx=0x7f8860009520 comp rx=0 tx=0).stop 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 msgr2=0x7f88700748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 0x7f88700748f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f886800d750 tx=0x7f886800dc10 comp rx=0 tx=0).stop 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 shutdown_connections 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f88540a3bb0 0x7f88540a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8870103970 0x7f88700748f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8870102fb0 0x7f88700743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.086+0000 7f8875600640 1 --2- 172.21.15.57:0/1670333661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8870101e60 0x7f8870075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.087+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 >> 172.21.15.57:0/1670333661 conn(0x7f88700fd3d0 msgr2=0x7f88701063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:00.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.087+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 shutdown_connections 2024-04-13T05:15:00.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.087+0000 7f8875600640 1 -- 172.21.15.57:0/1670333661 wait complete. 2024-04-13T05:15:00.108 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:00.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 -- 172.21.15.57:0/2563857186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 msgr2=0x7f694810c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 --2- 172.21.15.57:0/2563857186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f694810c1b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6940009900 tx=0x7f694002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:00.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 -- 172.21.15.57:0/2563857186 shutdown_connections 2024-04-13T05:15:00.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 --2- 172.21.15.57:0/2563857186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948074a20 0x7f6948074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 --2- 172.21.15.57:0/2563857186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f694810c1b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 --2- 172.21.15.57:0/2563857186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6948076000 0x7f6948074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 -- 172.21.15.57:0/2563857186 >> 172.21.15.57:0/2563857186 conn(0x7f69480fd3d0 msgr2=0x7f69480ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 -- 172.21.15.57:0/2563857186 shutdown_connections 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.332+0000 7f6947e00640 1 -- 172.21.15.57:0/2563857186 wait complete. 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.333+0000 7f6947e00640 1 Processor -- start 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.333+0000 7f6947e00640 1 -- start start 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6948074a20 0x7f694819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948076000 0x7f694819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f69481a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f694810fbe0 con 0x7f6948109d20 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f694810fa60 con 0x7f6948076000 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f694810fd60 con 0x7f6948074a20 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f69481a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f69481a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33582/0 (socket says 172.21.15.57:33582) 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 -- 172.21.15.57:0/1766874957 learned_addr learned my addr 172.21.15.57:0/1766874957 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6946a00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6948074a20 0x7f694819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6946000640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948076000 0x7f694819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 -- 172.21.15.57:0/1766874957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6948074a20 msgr2=0x7f694819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6948074a20 0x7f694819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 -- 172.21.15.57:0/1766874957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948076000 msgr2=0x7f694819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948076000 0x7f694819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.334+0000 7f6947400640 1 -- 172.21.15.57:0/1766874957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69481a1d70 con 0x7f6948109d20 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.335+0000 7f6947400640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f69481a1700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f694002c170 tx=0x7f6940004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.335+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940043020 con 0x7f6948109d20 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.335+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69400052b0 con 0x7f6948109d20 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.335+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940003cd0 con 0x7f6948109d20 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.335+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69481a2000 con 0x7f6948109d20 2024-04-13T05:15:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.335+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481a2500 con 0x7f6948109d20 2024-04-13T05:15:00.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.336+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6948108a50 con 0x7f6948109d20 2024-04-13T05:15:00.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.338+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6940046020 con 0x7f6948109d20 2024-04-13T05:15:00.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.340+0000 7f694ca00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f692c0a3c30 0x7f692c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.340+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f69400f3410 con 0x7f6948109d20 2024-04-13T05:15:00.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.340+0000 7f6946a00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f692c0a3c30 0x7f692c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.341+0000 7f6946a00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f692c0a3c30 0x7f692c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6938005c40 tx=0x7f693800a780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.342+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f694004d020 con 0x7f6948109d20 2024-04-13T05:15:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:00 smithi083 ceph-mon[35133]: osdmap e90: 12 total, 12 up, 12 in 2024-04-13T05:15:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1274017908' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-13T05:15:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:00 smithi083 ceph-mon[35133]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1341120153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1670333661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:00.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:00 smithi057 ceph-mon[32331]: osdmap e90: 12 total, 12 up, 12 in 2024-04-13T05:15:00.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1274017908' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-13T05:15:00.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:00 smithi057 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:00.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1341120153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:00.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1670333661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:00.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.554+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6948062c60 con 0x7f6948109d20 2024-04-13T05:15:00.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.558+0000 7f694ca00640 1 -- 172.21.15.57:0/1766874957 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24054 (secure 0 0 0) 0x7f69400bfcd0 con 0x7f6948109d20 2024-04-13T05:15:00.561 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:00.561 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:14:58.570092+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:00.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f692c0a3c30 msgr2=0x7f692c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f692c0a3c30 0x7f692c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6938005c40 tx=0x7f693800a780 comp rx=0 tx=0).stop 2024-04-13T05:15:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 msgr2=0x7f69481a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f69481a1700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f694002c170 tx=0x7f6940004270 comp rx=0 tx=0).stop 2024-04-13T05:15:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 shutdown_connections 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f692c0a3c30 0x7f692c0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6948076000 0x7f694819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6948109d20 0x7f69481a1700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 --2- 172.21.15.57:0/1766874957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6948074a20 0x7f694819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.561+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 >> 172.21.15.57:0/1766874957 conn(0x7f69480fd3d0 msgr2=0x7f6948105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.562+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 shutdown_connections 2024-04-13T05:15:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.562+0000 7f6947e00640 1 -- 172.21.15.57:0/1766874957 wait complete. 2024-04-13T05:15:00.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:15:00.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:00 smithi189 ceph-mon[35531]: osdmap e90: 12 total, 12 up, 12 in 2024-04-13T05:15:00.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1274017908' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-13T05:15:00.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:00 smithi189 ceph-mon[35531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:00.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1341120153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:00.601 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1670333661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 -- 172.21.15.57:0/2262939420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 msgr2=0x7f1d20109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 --2- 172.21.15.57:0/2262939420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d20109f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009ca0 tx=0x7f1d1802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 -- 172.21.15.57:0/2262939420 shutdown_connections 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 --2- 172.21.15.57:0/2262939420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d20100420 0x7f1d20107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 --2- 172.21.15.57:0/2262939420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d20109f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 --2- 172.21.15.57:0/2262939420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d200ffa30 0x7f1d200ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 -- 172.21.15.57:0/2262939420 >> 172.21.15.57:0/2262939420 conn(0x7f1d200fb5f0 msgr2=0x7f1d200fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 -- 172.21.15.57:0/2262939420 shutdown_connections 2024-04-13T05:15:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.828+0000 7f1d27000640 1 -- 172.21.15.57:0/2262939420 wait complete. 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.829+0000 7f1d27000640 1 Processor -- start 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.829+0000 7f1d27000640 1 -- start start 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d27000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d200ffa30 0x7f1d2019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d27000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d20100420 0x7f1d2019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d27000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d201a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d27000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2010d940 con 0x7f1d20107b40 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d27000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2010d7c0 con 0x7f1d20100420 2024-04-13T05:15:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d27000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2010dac0 con 0x7f1d200ffa30 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d201a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d201a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33602/0 (socket says 172.21.15.57:33602) 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 -- 172.21.15.57:0/2101242127 learned_addr learned my addr 172.21.15.57:0/2101242127 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d25c00640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d200ffa30 0x7f1d2019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d25200640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d20100420 0x7f1d2019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 -- 172.21.15.57:0/2101242127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d200ffa30 msgr2=0x7f1d2019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d200ffa30 0x7f1d2019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 -- 172.21.15.57:0/2101242127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d20100420 msgr2=0x7f1d2019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.830+0000 7f1d26600640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d20100420 0x7f1d2019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d26600640 1 -- 172.21.15.57:0/2101242127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d201a1e20 con 0x7f1d20107b40 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d26600640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d201a17b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d18032380 tx=0x7f1d18032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18041020 con 0x7f1d20107b40 2024-04-13T05:15:00.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d18003e20 con 0x7f1d20107b40 2024-04-13T05:15:00.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18039040 con 0x7f1d20107b40 2024-04-13T05:15:00.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d201a20b0 con 0x7f1d20107b40 2024-04-13T05:15:00.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.831+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d201a25f0 con 0x7f1d20107b40 2024-04-13T05:15:00.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.832+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d201a2850 con 0x7f1d20107b40 2024-04-13T05:15:00.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.834+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1d18044020 con 0x7f1d20107b40 2024-04-13T05:15:00.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.836+0000 7f1d1e000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1d080a3c30 0x7f1d080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:00.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.836+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1d180f29a0 con 0x7f1d20107b40 2024-04-13T05:15:00.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.836+0000 7f1d25c00640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1d080a3c30 0x7f1d080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:00.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.837+0000 7f1d25c00640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1d080a3c30 0x7f1d080a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005fd0 tx=0x7f1d14005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:00.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:00.840+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d1804b020 con 0x7f1d20107b40 2024-04-13T05:15:01.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.089+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d201a2a20 con 0x7f1d20107b40 2024-04-13T05:15:01.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.091+0000 7f1d1e000640 1 -- 172.21.15.57:0/2101242127 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11782 (secure 0 0 0) 0x7f1d1803b2f0 con 0x7f1d20107b40 2024-04-13T05:15:01.092 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:01.092 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34427,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34457,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:14:58.570497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34499,"qdb_cluster":[34499]},"id":1}]} 2024-04-13T05:15:01.093 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-04-13T05:15:01.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.093+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1d080a3c30 msgr2=0x7f1d080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.093+0000 7f1d27000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1d080a3c30 0x7f1d080a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005fd0 tx=0x7f1d14005850 comp rx=0 tx=0).stop 2024-04-13T05:15:01.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 msgr2=0x7f1d201a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d201a17b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d18032380 tx=0x7f1d18032820 comp rx=0 tx=0).stop 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 shutdown_connections 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1d080a3c30 0x7f1d080a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d20100420 0x7f1d2019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d20107b40 0x7f1d201a17b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 --2- 172.21.15.57:0/2101242127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d200ffa30 0x7f1d2019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 >> 172.21.15.57:0/2101242127 conn(0x7f1d200fb5f0 msgr2=0x7f1d200fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 shutdown_connections 2024-04-13T05:15:01.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.094+0000 7f1d27000640 1 -- 172.21.15.57:0/2101242127 wait complete. 2024-04-13T05:15:01.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:01.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.342+0000 7f3693e00640 1 -- 172.21.15.57:0/976519109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 msgr2=0x7f3694108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.342+0000 7f3693e00640 1 --2- 172.21.15.57:0/976519109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f3694108860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3684009900 tx=0x7f368402bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 -- 172.21.15.57:0/976519109 shutdown_connections 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 --2- 172.21.15.57:0/976519109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3694109630 0x7f3694109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 --2- 172.21.15.57:0/976519109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109ff0 0x7f3694110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 --2- 172.21.15.57:0/976519109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f3694108860 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 -- 172.21.15.57:0/976519109 >> 172.21.15.57:0/976519109 conn(0x7f3694077400 msgr2=0x7f3694079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 -- 172.21.15.57:0/976519109 shutdown_connections 2024-04-13T05:15:01.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.343+0000 7f3693e00640 1 -- 172.21.15.57:0/976519109 wait complete. 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 Processor -- start 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 -- start start 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f369419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109630 0x7f369419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3694109ff0 0x7f36941a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3694113f80 con 0x7f3694108440 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3694113e00 con 0x7f3694109ff0 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.344+0000 7f3693e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3694114100 con 0x7f3694109630 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f369419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f369419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33634/0 (socket says 172.21.15.57:33634) 2024-04-13T05:15:01.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 -- 172.21.15.57:0/2544991195 learned_addr learned my addr 172.21.15.57:0/2544991195 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f368a000640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109630 0x7f369419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3693400640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3694109ff0 0x7f36941a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 -- 172.21.15.57:0/2544991195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109630 msgr2=0x7f369419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109630 0x7f369419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 -- 172.21.15.57:0/2544991195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3694109ff0 msgr2=0x7f36941a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3694109ff0 0x7f36941a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 -- 172.21.15.57:0/2544991195 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36941a62a0 con 0x7f3694108440 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f368a000640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109630 0x7f369419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.345+0000 7f3692a00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f369419e680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3684000c00 tx=0x7f3684004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.346+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684043020 con 0x7f3694108440 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.346+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3684004aa0 con 0x7f3694108440 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.346+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36941a6530 con 0x7f3694108440 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.346+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684003d00 con 0x7f3694108440 2024-04-13T05:15:01.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.346+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36941a6a80 con 0x7f3694108440 2024-04-13T05:15:01.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.347+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369410f800 con 0x7f3694108440 2024-04-13T05:15:01.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.349+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3684046020 con 0x7f3694108440 2024-04-13T05:15:01.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.350+0000 7f368be00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36780a3bb0 0x7f36780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.351+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f36840f2d00 con 0x7f3694108440 2024-04-13T05:15:01.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.351+0000 7f368a000640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36780a3bb0 0x7f36780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.351+0000 7f368a000640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36780a3bb0 0x7f36780a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f369419fb90 tx=0x7f367c0093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.356+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f368404d020 con 0x7f3694108440 2024-04-13T05:15:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:01 smithi083 ceph-mon[35133]: pgmap v374: 129 pgs: 10 unknown, 119 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-13T05:15:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:01 smithi083 ceph-mon[35133]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-13T05:15:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:01 smithi083 ceph-mon[35133]: Cluster is now healthy 2024-04-13T05:15:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1766874957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2101242127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:01 smithi057 ceph-mon[32331]: pgmap v374: 129 pgs: 10 unknown, 119 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-13T05:15:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:01 smithi057 ceph-mon[32331]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-13T05:15:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:01 smithi057 ceph-mon[32331]: Cluster is now healthy 2024-04-13T05:15:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1766874957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2101242127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.566+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3694062bc0 con 0x7f3694108440 2024-04-13T05:15:01.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.570+0000 7f368be00640 1 -- 172.21.15.57:0/2544991195 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24054 (secure 0 0 0) 0x7f36840bf440 con 0x7f3694108440 2024-04-13T05:15:01.572 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:01.572 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:14:58.570092+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:01.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36780a3bb0 msgr2=0x7f36780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36780a3bb0 0x7f36780a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f369419fb90 tx=0x7f367c0093e0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 msgr2=0x7f369419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f369419e680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3684000c00 tx=0x7f3684004270 comp rx=0 tx=0).stop 2024-04-13T05:15:01.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 shutdown_connections 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36780a3bb0 0x7f36780a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3694109ff0 0x7f36941a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3694109630 0x7f369419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 --2- 172.21.15.57:0/2544991195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3694108440 0x7f369419e680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.573+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 >> 172.21.15.57:0/2544991195 conn(0x7f3694077400 msgr2=0x7f369410c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.574+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 shutdown_connections 2024-04-13T05:15:01.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.574+0000 7f3693e00640 1 -- 172.21.15.57:0/2544991195 wait complete. 2024-04-13T05:15:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:01 smithi189 ceph-mon[35531]: pgmap v374: 129 pgs: 10 unknown, 119 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-13T05:15:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:01 smithi189 ceph-mon[35531]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-13T05:15:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:01 smithi189 ceph-mon[35531]: Cluster is now healthy 2024-04-13T05:15:01.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1766874957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2101242127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:01.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-13T05:14:53.072497+0000', 'modified': '2024-04-13T05:14:58.570497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34499': {'gid': 34499, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/3095811120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3095811120}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3095811120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34499, 'qdb_cluster': [34499]} 2024-04-13T05:15:01.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-13T05:15:01.596 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-13T05:15:01.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 -- 172.21.15.57:0/1292566662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 msgr2=0x7f3658108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 --2- 172.21.15.57:0/1292566662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f3658108a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f364c009900 tx=0x7f364c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 -- 172.21.15.57:0/1292566662 shutdown_connections 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 --2- 172.21.15.57:0/1292566662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 0x7f3658110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 --2- 172.21.15.57:0/1292566662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 0x7f3658109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 --2- 172.21.15.57:0/1292566662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f3658108a40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 -- 172.21.15.57:0/1292566662 >> 172.21.15.57:0/1292566662 conn(0x7f36580775c0 msgr2=0x7f3658079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 -- 172.21.15.57:0/1292566662 shutdown_connections 2024-04-13T05:15:01.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.811+0000 7f365fe00640 1 -- 172.21.15.57:0/1292566662 wait complete. 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 Processor -- start 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 -- start start 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f365819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 0x7f365819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 0x7f36581a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f36581141a0 con 0x7f3658108620 2024-04-13T05:15:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3658114020 con 0x7f365810a1d0 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.812+0000 7f365fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3658114320 con 0x7f3658109810 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f365819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f365819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33642/0 (socket says 172.21.15.57:33642) 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 -- 172.21.15.57:0/1262415381 learned_addr learned my addr 172.21.15.57:0/1262415381 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 -- 172.21.15.57:0/1262415381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 msgr2=0x7f365819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365de00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 0x7f36581a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365ca00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 0x7f365819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 0x7f365819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 -- 172.21.15.57:0/1262415381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 msgr2=0x7f36581a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 0x7f36581a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 -- 172.21.15.57:0/1262415381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36581a64f0 con 0x7f3658108620 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365de00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 0x7f36581a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365ca00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 0x7f365819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.813+0000 7f365d400640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f365819e8d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f364c005ab0 tx=0x7f364c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.814+0000 7f3656000640 1 -- 172.21.15.57:0/1262415381 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c043020 con 0x7f3658108620 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.814+0000 7f3656000640 1 -- 172.21.15.57:0/1262415381 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364c0054c0 con 0x7f3658108620 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.814+0000 7f3656000640 1 -- 172.21.15.57:0/1262415381 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c003d10 con 0x7f3658108620 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.814+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36581a6780 con 0x7f3658108620 2024-04-13T05:15:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.814+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36581a6c80 con 0x7f3658108620 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.817+0000 7f3656000640 1 -- 172.21.15.57:0/1262415381 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f364c046020 con 0x7f3658108620 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.817+0000 7f3662bec740 1 Processor -- start 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.817+0000 7f3662bec740 1 -- start start 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.817+0000 7f3662bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e117f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f3662bec740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1184a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f3662bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e1187560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f3662bec740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b8e10e5990 con 0x55b8e117ff60 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f3662bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8e108c530 con 0x55b8e11850d0 2024-04-13T05:15:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f3662bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b8e10e1950 con 0x55b8e117f570 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1184a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1184a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33654/0 (socket says 172.21.15.57:33654) 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 -- 172.21.15.57:0/156450354 learned_addr learned my addr 172.21.15.57:0/156450354 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365de00640 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e117f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365ca00640 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e1187560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 -- 172.21.15.57:0/156450354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 msgr2=0x55b8e117f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e117f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 -- 172.21.15.57:0/156450354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 msgr2=0x55b8e1187560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.818+0000 7f365d400640 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e1187560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f365d400640 1 -- 172.21.15.57:0/156450354 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e0df82d0 con 0x55b8e117ff60 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f365de00640 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e117f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f365d400640 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1184a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f364c0f8a90 tx=0x7f364c0f8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4acec2d56ddba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f3656000640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36400a3c30 0x7f36400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f364a600640 1 -- 172.21.15.57:0/156450354 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c101070 con 0x55b8e117ff60 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f365de00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36400a3c30 0x7f36400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f3656000640 1 -- 172.21.15.57:0/1262415381 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f364c0f29f0 con 0x7f3658108620 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f364a600640 1 -- 172.21.15.57:0/156450354 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364c0bf130 con 0x55b8e117ff60 2024-04-13T05:15:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.819+0000 7f364a600640 1 -- 172.21.15.57:0/156450354 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0bacd0 con 0x55b8e117ff60 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 -- 172.21.15.57:0/156450354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 msgr2=0x55b8e1184a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f365de00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36400a3c30 0x7f36400a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55b8e11807c0 tx=0x7f36500095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1184a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f364c0f8a90 tx=0x7f364c0f8b80 comp rx=0 tx=0).stop 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 -- 172.21.15.57:0/156450354 shutdown_connections 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e1187560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1184a60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 --2- 172.21.15.57:0/156450354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e117f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.820+0000 7f3662bec740 1 -- 172.21.15.57:0/156450354 >> 172.21.15.57:0/156450354 conn(0x55b8e117aae0 msgr2=0x55b8e117aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.821+0000 7f3662bec740 1 -- 172.21.15.57:0/156450354 shutdown_connections 2024-04-13T05:15:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.821+0000 7f3662bec740 1 -- 172.21.15.57:0/156450354 wait complete. 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.821+0000 7f3662bec740 1 -- start start 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.821+0000 7f3662bec740 1 Processor -- start 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e11897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1189d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e1190d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1187e40 con 0x55b8e117ff60 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8e109f8f0 con 0x55b8e117f570 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1187fc0 con 0x55b8e11850d0 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f3662bec740 10 client.0 _ll_register_callbacks cb 0x55b8e109f2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f365ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1189d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f365ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1189d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33668/0 (socket says 172.21.15.57:33668) 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f365ca00640 1 -- 172.21.15.57:0/2880310524 learned_addr learned my addr 172.21.15.57:0/2880310524 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f365de00640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e1190d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.822+0000 7f365d400640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e11897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f365ca00640 1 -- 172.21.15.57:0/2880310524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 msgr2=0x55b8e1190d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f365ca00640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e1190d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f365ca00640 1 -- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 msgr2=0x55b8e11897f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f365ca00640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e11897f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f365ca00640 1 -- 172.21.15.57:0/2880310524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e0ec26b0 con 0x55b8e117ff60 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f365ca00640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1189d30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3644010670 tx=0x7f3644010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dc9dcd0a8640db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f363ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f363ea00640 1 -- 172.21.15.57:0/2880310524 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364400ea70 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f363ea00640 1 -- 172.21.15.57:0/2880310524 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f364400ec10 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.823+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c0bf130 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f363ea00640 1 -- 172.21.15.57:0/2880310524 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f364400ee10 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55b8e11941b0 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f363ea00640 1 -- 172.21.15.57:0/2880310524 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364c0bf130 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f363ea00640 1 -- 172.21.15.57:0/2880310524 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f364400f040 con 0x55b8e117ff60 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 20 client.14757 populate_metadata read hostname 'smithi057' 2024-04-13T05:15:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e1196880 0x55b8e1197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 4 client.14757 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:15:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.824+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8e108bb40 con 0x55b8e1196880 2024-04-13T05:15:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.825+0000 7f365d400640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e1196880 0x55b8e1197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.826+0000 7f365d400640 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e1196880 0x55b8e1197cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555edc54382c4c18 server_cookie=410e6039296644ce in_seq=0 out_seq=0 2024-04-13T05:15:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.826+0000 7f363ea00640 10 client.14757 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:15:01.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.828+0000 7f363ea00640 1 -- 172.21.15.57:0/2880310524 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8e108bb40 con 0x55b8e1196880 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.828+0000 7f363ea00640 10 client.14757 handle_command_reply: tid=0 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.828+0000 7f3662bec740 1 client.14757 shutdown 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 msgr2=0x55b8e1189d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1189d30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3644010670 tx=0x7f3644010b30 comp rx=0 tx=0).stop 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 shutdown_connections 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e1196880 0x55b8e1197cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e1190d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1189d30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f363ea00640 0 client.14757 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:15:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 --2- 172.21.15.57:0/2880310524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e11897f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 >> 172.21.15.57:0/2880310524 conn(0x55b8e117aae0 msgr2=0x55b8e1179d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 shutdown_connections 2024-04-13T05:15:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 1 -- 172.21.15.57:0/2880310524 wait complete. 2024-04-13T05:15:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.829+0000 7f3662bec740 20 client.14757 trim_cache size 0 max 16384 2024-04-13T05:15:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 Processor -- start 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 -- start start 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e1188320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1190d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e117aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1196030 con 0x55b8e117ff60 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1196700 con 0x55b8e11850d0 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.849+0000 7f3662bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1196dd0 con 0x55b8e117f570 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1190d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1190d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33670/0 (socket says 172.21.15.57:33670) 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 -- 172.21.15.57:0/2667065222 learned_addr learned my addr 172.21.15.57:0/2667065222 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365d400640 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e1188320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 -- 172.21.15.57:0/2667065222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 msgr2=0x55b8e1188320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e1188320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 -- 172.21.15.57:0/2667065222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 msgr2=0x55b8e117aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365de00640 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e117aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e117aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365ca00640 1 -- 172.21.15.57:0/2667065222 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e1184740 con 0x55b8e117ff60 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.850+0000 7f365d400640 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e1188320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.851+0000 7f365ca00640 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1190d30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3644001eb0 tx=0x7f364401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cbf54333788741 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.851+0000 7f3649c00640 1 -- 172.21.15.57:0/2667065222 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364401cb60 con 0x55b8e117ff60 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.851+0000 7f3649c00640 1 -- 172.21.15.57:0/2667065222 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364400eb70 con 0x55b8e117ff60 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.851+0000 7f3649c00640 1 -- 172.21.15.57:0/2667065222 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644005460 con 0x55b8e117ff60 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 -- 172.21.15.57:0/2667065222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 msgr2=0x55b8e1190d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1190d30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3644001eb0 tx=0x7f364401ca90 comp rx=0 tx=0).stop 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 -- 172.21.15.57:0/2667065222 shutdown_connections 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e11850d0 0x55b8e117aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e1190d30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 --2- 172.21.15.57:0/2667065222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e117f570 0x55b8e1188320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 -- 172.21.15.57:0/2667065222 >> 172.21.15.57:0/2667065222 conn(0x55b8e117aae0 msgr2=0x55b8e1175ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 -- 172.21.15.57:0/2667065222 shutdown_connections 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 -- 172.21.15.57:0/2667065222 wait complete. 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 -- start start 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.852+0000 7f3662bec740 1 Processor -- start 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e118ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e11f8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e11fad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1197b70 con 0x55b8e117ff60 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b8e11974a0 con 0x55b8e117f570 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b8e1198240 con 0x55b8e11850d0 2024-04-13T05:15:01.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f3662bec740 10 client.0 _ll_register_callbacks cb 0x7f3644009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:15:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f365ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e11f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f365ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e11f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33684/0 (socket says 172.21.15.57:33684) 2024-04-13T05:15:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f365ca00640 1 -- 172.21.15.57:0/2255712510 learned_addr learned my addr 172.21.15.57:0/2255712510 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:01.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f365d400640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e118ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.853+0000 7f365de00640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e11fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f365ca00640 1 -- 172.21.15.57:0/2255712510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 msgr2=0x55b8e11fad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f365ca00640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e11fad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f365ca00640 1 -- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 msgr2=0x55b8e118ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f365ca00640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e118ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f365ca00640 1 -- 172.21.15.57:0/2255712510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e11fb570 con 0x55b8e117ff60 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f365ca00640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e11f8810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3644005910 tx=0x7f3644009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9898f0e0df7b2c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f363ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f363ea00640 1 -- 172.21.15.57:0/2255712510 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364401c760 con 0x55b8e117ff60 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f363ea00640 1 -- 172.21.15.57:0/2255712510 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f364401cd10 con 0x55b8e117ff60 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.854+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364400eb70 con 0x55b8e117ff60 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f363ea00640 1 -- 172.21.15.57:0/2255712510 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f36440051e0 con 0x55b8e117ff60 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 10 client.14763 fetch_fsmap learned FSMap version 17 2024-04-13T05:15:01.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55b8e11fc7e0 con 0x55b8e117ff60 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f363ea00640 1 -- 172.21.15.57:0/2255712510 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f364400eb70 con 0x55b8e117ff60 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f363ea00640 1 -- 172.21.15.57:0/2255712510 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f3644004d00 con 0x55b8e117ff60 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 10 client.14763 fetch_fsmap finished waiting for FSMap version 17 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 20 client.14763 populate_metadata read hostname 'smithi057' 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e12011b0 0x55b8e11fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 4 client.14763 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.855+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "status"}) -- 0x55b8e10f33b0 con 0x55b8e12011b0 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.856+0000 7f365d400640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e12011b0 0x55b8e11fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f365d400640 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e12011b0 0x55b8e11fd5f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef68b425c8b2063 server_cookie=e92c588f9826c94b in_seq=0 out_seq=0 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f363ea00640 10 client.14763 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f363ea00640 1 -- 172.21.15.57:0/2255712510 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b8e10f33b0 con 0x55b8e12011b0 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f363ea00640 10 client.14763 handle_command_reply: tid=0 2024-04-13T05:15:01.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 client.14763 shutdown 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 msgr2=0x55b8e11f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e11f8810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3644005910 tx=0x7f3644009d40 comp rx=0 tx=0).stop 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 shutdown_connections 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b8e12011b0 0x55b8e11fd5f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8e11850d0 0x55b8e11fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b8e117ff60 0x55b8e11f8810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 --2- 172.21.15.57:0/2255712510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b8e117f570 0x55b8e118ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f363ea00640 0 client.14763 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.857+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 >> 172.21.15.57:0/2255712510 conn(0x55b8e117aae0 msgr2=0x55b8e117aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 shutdown_connections 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f3662bec740 1 -- 172.21.15.57:0/2255712510 wait complete. 2024-04-13T05:15:01.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f3662bec740 20 client.14763 trim_cache size 0 max 16384 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34499, 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:15:01.863 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:15:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 8.2595936170000002, 2024-04-13T05:15:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 17, 2024-04-13T05:15:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 90, 2024-04-13T05:15:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 90, 2024-04-13T05:15:01.864 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 14.940814519 2024-04-13T05:15:01.864 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36400a3c30 msgr2=0x7f36400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f365fe00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36400a3c30 0x7f36400a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55b8e11807c0 tx=0x7f36500095b0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 msgr2=0x7f365819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.858+0000 7f365fe00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f365819e8d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f364c005ab0 tx=0x7f364c0042f0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 shutdown_connections 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f36400a3c30 0x7f36400a60f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f365810a1d0 0x7f36581a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3658109810 0x7f365819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 --2- 172.21.15.57:0/1262415381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3658108620 0x7f365819e8d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:01.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 >> 172.21.15.57:0/1262415381 conn(0x7f36580775c0 msgr2=0x7f365810cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:01.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 shutdown_connections 2024-04-13T05:15:01.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:01.859+0000 7f365fe00640 1 -- 172.21.15.57:0/1262415381 wait complete. 2024-04-13T05:15:01.880 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T05:15:01.880 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:01.881 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph fs ls 2024-04-13T05:15:02.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2544991195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:02.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:02 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2544991195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:02.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:02 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2544991195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:03 smithi083 ceph-mon[35133]: pgmap v375: 129 pgs: 2 unknown, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 555 B/s wr, 5 op/s 2024-04-13T05:15:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:03 smithi057 ceph-mon[32331]: pgmap v375: 129 pgs: 2 unknown, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 555 B/s wr, 5 op/s 2024-04-13T05:15:03.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:03 smithi189 ceph-mon[35531]: pgmap v375: 129 pgs: 2 unknown, 127 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 555 B/s wr, 5 op/s 2024-04-13T05:15:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 -- 172.21.15.57:0/328247703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 msgr2=0x7f601410dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 --2- 172.21.15.57:0/328247703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601410dc40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0099c0 tx=0x7f5ffc009cd0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 -- 172.21.15.57:0/328247703 shutdown_connections 2024-04-13T05:15:04.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 --2- 172.21.15.57:0/328247703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 0x7f6014115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 --2- 172.21.15.57:0/328247703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 0x7f6014109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 --2- 172.21.15.57:0/328247703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601410dc40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 -- 172.21.15.57:0/328247703 >> 172.21.15.57:0/328247703 conn(0x7f60140788b0 msgr2=0x7f60141078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 -- 172.21.15.57:0/328247703 shutdown_connections 2024-04-13T05:15:04.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.614+0000 7f6018e00700 1 -- 172.21.15.57:0/328247703 wait complete. 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 Processor -- start 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 -- start start 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 0x7f601419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 0x7f60141a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6014119210 con 0x7f6014109c00 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6014119090 con 0x7f6014109250 2024-04-13T05:15:04.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f6018e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6014119390 con 0x7f601410e320 2024-04-13T05:15:04.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.616+0000 7f600be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:04.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33690/0 (socket says 172.21.15.57:33690) 2024-04-13T05:15:04.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 -- 172.21.15.57:0/3975985749 learned_addr learned my addr 172.21.15.57:0/3975985749 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:04.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f6013400700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 0x7f601419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f6013e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 0x7f60141a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 -- 172.21.15.57:0/3975985749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 msgr2=0x7f60141a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 0x7f60141a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 -- 172.21.15.57:0/3975985749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 msgr2=0x7f601419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 0x7f601419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 -- 172.21.15.57:0/3975985749 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60141a4630 con 0x7f6014109c00 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f6013e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 0x7f60141a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f6013400700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 0x7f601419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.617+0000 7f600be00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601419fb60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009fd0 tx=0x7f5ffc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.618+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc004830 con 0x7f6014109c00 2024-04-13T05:15:04.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.618+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60141a48c0 con 0x7f6014109c00 2024-04-13T05:15:04.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.618+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ffc003ba0 con 0x7f6014109c00 2024-04-13T05:15:04.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.618+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc005380 con 0x7f6014109c00 2024-04-13T05:15:04.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.618+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141ac1a0 con 0x7f6014109c00 2024-04-13T05:15:04.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.619+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60140697a0 con 0x7f6014109c00 2024-04-13T05:15:04.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.621+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ffc005540 con 0x7f6014109c00 2024-04-13T05:15:04.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.622+0000 7f6010c00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5ff80a3c40 0x7f5ff80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:04.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.623+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5ffc0d1bb0 con 0x7f6014109c00 2024-04-13T05:15:04.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.623+0000 7f6013400700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5ff80a3c40 0x7f5ff80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:04.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.624+0000 7f6013400700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5ff80a3c40 0x7f5ff80a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6004005fd0 tx=0x7f6004005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:04.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.626+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ffc013070 con 0x7f6014109c00 2024-04-13T05:15:04.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.908+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f601402dfa0 con 0x7f6014109c00 2024-04-13T05:15:04.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.909+0000 7f6010c00700 1 -- 172.21.15.57:0/3975985749 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f5ffc09e380 con 0x7f6014109c00 2024-04-13T05:15:04.911 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.912+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5ff80a3c40 msgr2=0x7f5ff80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.912+0000 7f6018e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5ff80a3c40 0x7f5ff80a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6004005fd0 tx=0x7f6004005e30 comp rx=0 tx=0).stop 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.912+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 msgr2=0x7f601419fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.912+0000 7f6018e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601419fb60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009fd0 tx=0x7f5ffc005f10 comp rx=0 tx=0).stop 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 shutdown_connections 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601410e320 0x7f60141a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5ff80a3c40 0x7f5ff80a6100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6014109250 0x7f601419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 --2- 172.21.15.57:0/3975985749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6014109c00 0x7f601419fb60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:04.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 >> 172.21.15.57:0/3975985749 conn(0x7f60140788b0 msgr2=0x7f6014078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:04.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 shutdown_connections 2024-04-13T05:15:04.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:04.913+0000 7f6018e00700 1 -- 172.21.15.57:0/3975985749 wait complete. 2024-04-13T05:15:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:05 smithi083 ceph-mon[35133]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 470 B/s wr, 4 op/s 2024-04-13T05:15:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:05 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3975985749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-13T05:15:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:05 smithi057 ceph-mon[32331]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 470 B/s wr, 4 op/s 2024-04-13T05:15:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:05 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3975985749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-13T05:15:05.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:05 smithi189 ceph-mon[35531]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 470 B/s wr, 4 op/s 2024-04-13T05:15:05.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:05 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3975985749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-13T05:15:05.742 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T05:15:05.743 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-13T05:15:05.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:05.743 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T05:15:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41868sec preferred_lft 41868sec 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link noprefixroute 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:15:05.806 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2024-04-13T05:15:05.808 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-13T05:15:05.808 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:15:05.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:15:06.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:06.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:06.365 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T05:15:06.449 INFO:teuthology.orchestra.run.smithi057.stdout:1 2024-04-13T05:15:06.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:06.453 DEBUG:teuthology.orchestra.run.smithi057:> route 2024-04-13T05:15:06.512 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2024-04-13T05:15:06.512 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T05:15:06.512 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-13T05:15:06.512 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-13T05:15:06.512 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T05:15:06.513 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:15:06.513 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:15:06.513 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-13T05:15:06.513 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-13T05:15:06.513 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-13T05:15:06.514 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:15:06.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:15:07.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:07 smithi057 ceph-mon[32331]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2024-04-13T05:15:07.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:07.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:07.297 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-04-13T05:15:07.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:07.341 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2024-04-13T05:15:07.430 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:15:07.430 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:15:07.430 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:07.430 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-13T05:15:07.430 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:15:07.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:07 smithi189 ceph-mon[35531]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2024-04-13T05:15:07.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:07 smithi083 ceph-mon[35133]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2024-04-13T05:15:07.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:15:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:07.628 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-13T05:15:07.628 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-13T05:15:07.629 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:15:07.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:15:07.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:07.965 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:15:07.965 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:15:07.965 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2024-04-13T05:15:07.965 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2024-04-13T05:15:07.965 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:15:08.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:15:08.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:08.198 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-13T05:15:08.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T05:15:08.198 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:08.244 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-13T05:15:08.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T05:15:08.244 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:08.338 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-04-13T05:15:08.420 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T05:15:08.471 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-04-13T05:15:08.472 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6851f3d5cad808da345862ffed2203ee2c439f121f2ac3db87bb2c5457b1ce80/merged 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/eee32290b21af5b97799ed3b97942259138b04d6a361a6949b56fbc33ff47406/merged 2024-04-13T05:15:08.473 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ba38b59613f94e78d8047d50e6da3db99e177f5d78bb8531a90381148c82eb45/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/86ab784d697c2d6a710126b875258c1c5a46e596ffb190580fdb9bf6e15cbc46/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bc0adc14b06059c566e30aae0619eb2c6bb7c1cf05d5b6007ea150b67749e6a0/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/88f5e5177d007b1027576a24101334ffb000d2df9af20c9f436e68dc9282c4b0/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/b7bd757dedb3c03736477ef5fb575a52b3e9f6193e0d3927122420700d88449f/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e95c76b032ca6ef0528bd5d53e3d1a2bbef7697ac98e87aeff39bd3e73bc4da9/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/5fc78578520fb786ab26f5309544797a8902e2ed889908f6fc3ba1a61097a768/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/7897aeadb114fe2014e7c81b3367e0bceac12315f7fdebc7f217e8f1da1b05fa/merged 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:08.474 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:08.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:08.475 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-04-13T05:15:08.533 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-13T05:15:08.533 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-13T05:15:08.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-04-13T05:15:08.622 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T05:15:08.668 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-04-13T05:15:08.669 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6851f3d5cad808da345862ffed2203ee2c439f121f2ac3db87bb2c5457b1ce80/merged 2024-04-13T05:15:08.670 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/eee32290b21af5b97799ed3b97942259138b04d6a361a6949b56fbc33ff47406/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ba38b59613f94e78d8047d50e6da3db99e177f5d78bb8531a90381148c82eb45/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/86ab784d697c2d6a710126b875258c1c5a46e596ffb190580fdb9bf6e15cbc46/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bc0adc14b06059c566e30aae0619eb2c6bb7c1cf05d5b6007ea150b67749e6a0/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/88f5e5177d007b1027576a24101334ffb000d2df9af20c9f436e68dc9282c4b0/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/b7bd757dedb3c03736477ef5fb575a52b3e9f6193e0d3927122420700d88449f/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e95c76b032ca6ef0528bd5d53e3d1a2bbef7697ac98e87aeff39bd3e73bc4da9/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/5fc78578520fb786ab26f5309544797a8902e2ed889908f6fc3ba1a61097a768/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/7897aeadb114fe2014e7c81b3367e0bceac12315f7fdebc7f217e8f1da1b05fa/merged 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:08.671 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:08.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:08.675 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-04-13T05:15:08.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:2024-04-13T05:15:08.760+0000 7fe60f636480 -1 init, newargv = 0x55a3b3a46f00 newargc=15 2024-04-13T05:15:08.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[77589]: starting ceph client 2024-04-13T05:15:08.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[77589]: starting fuse 2024-04-13T05:15:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:09 smithi083 ceph-mon[35133]: pgmap v378: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 322 B/s wr, 2 op/s 2024-04-13T05:15:09.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:09 smithi057 ceph-mon[32331]: pgmap v378: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 322 B/s wr, 2 op/s 2024-04-13T05:15:09.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:09 smithi189 ceph-mon[35531]: pgmap v378: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 322 B/s wr, 2 op/s 2024-04-13T05:15:09.866 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-04-13T05:15:09.979 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-04-13T05:15:10.034 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-04-13T05:15:10.035 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6851f3d5cad808da345862ffed2203ee2c439f121f2ac3db87bb2c5457b1ce80/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/eee32290b21af5b97799ed3b97942259138b04d6a361a6949b56fbc33ff47406/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ba38b59613f94e78d8047d50e6da3db99e177f5d78bb8531a90381148c82eb45/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/86ab784d697c2d6a710126b875258c1c5a46e596ffb190580fdb9bf6e15cbc46/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bc0adc14b06059c566e30aae0619eb2c6bb7c1cf05d5b6007ea150b67749e6a0/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/88f5e5177d007b1027576a24101334ffb000d2df9af20c9f436e68dc9282c4b0/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/b7bd757dedb3c03736477ef5fb575a52b3e9f6193e0d3927122420700d88449f/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e95c76b032ca6ef0528bd5d53e3d1a2bbef7697ac98e87aeff39bd3e73bc4da9/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/5fc78578520fb786ab26f5309544797a8902e2ed889908f6fc3ba1a61097a768/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/7897aeadb114fe2014e7c81b3367e0bceac12315f7fdebc7f217e8f1da1b05fa/merged 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-04-13T05:15:10.036 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:10.037 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:15:10.037 INFO:teuthology.orchestra.run.smithi057.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:10.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:10.038 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-04-13T05:15:10.089 INFO:teuthology.orchestra.run.smithi057.stdout:119 2024-04-13T05:15:10.089 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> sudo stdin-killer -- python3 -c ' 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> import re 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> import os 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2024-04-13T05:15:10.090 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> return f 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.admin")) 2024-04-13T05:15:10.091 DEBUG:teuthology.orchestra.run.smithi057:> ' 2024-04-13T05:15:10.282 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.admin.77589.asok 2024-04-13T05:15:10.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:10.301 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.77589.asok 2024-04-13T05:15:10.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:10.301 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.77589.asok status 2024-04-13T05:15:10.553 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "218a691417ef57c5d22a8d5cf3c5f41f7b542838", 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev)", 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "admin", 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "77589", 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2024-04-13T05:15:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 14775, 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 14775 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1210660832 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-04-13T05:15:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1210660832 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.14775 192.168.0.1:0/1210660832", 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/1210660832", 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 17, 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 90, 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2024-04-13T05:15:10.556 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:15:10.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:10.570 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2024-04-13T05:15:10.570 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:10.630 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2024-04-13T05:15:10.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:10.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:10.630 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:10.735 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-13T05:15:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-13T05:15:10.901 INFO:teuthology.orchestra.run.smithi057.stdout:# file: . 2024-04-13T05:15:10.901 INFO:teuthology.orchestra.run.smithi057.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-13T05:15:10.902 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:10.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:15:10.913 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:15:10.913 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T05:15:10.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:11.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.203+0000 7feae3600640 1 -- 172.21.15.57:0/664387181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 msgr2=0x7feadc10c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.203+0000 7feae3600640 1 --2- 172.21.15.57:0/664387181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc10c680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fead4009ca0 tx=0x7fead402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 -- 172.21.15.57:0/664387181 shutdown_connections 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 --2- 172.21.15.57:0/664387181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc10c680 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 --2- 172.21.15.57:0/664387181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 0x7feadc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 --2- 172.21.15.57:0/664387181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feadc104240 0x7feadc104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 -- 172.21.15.57:0/664387181 >> 172.21.15.57:0/664387181 conn(0x7feadc0ff7b0 msgr2=0x7feadc101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 -- 172.21.15.57:0/664387181 shutdown_connections 2024-04-13T05:15:11.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.204+0000 7feae3600640 1 -- 172.21.15.57:0/664387181 wait complete. 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 Processor -- start 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 -- start start 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feadc104240 0x7feadc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 0x7feadc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7feadc10fd50 con 0x7feadc105df0 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feadc10fbd0 con 0x7feadc104240 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.205+0000 7feae3600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feadc10fed0 con 0x7feadc105430 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49572/0 (socket says 172.21.15.57:49572) 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 -- 172.21.15.57:0/442702763 learned_addr learned my addr 172.21.15.57:0/442702763 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae0c00640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feadc104240 0x7feadc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feadbe00640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 0x7feadc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 -- 172.21.15.57:0/442702763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 msgr2=0x7feadc19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 0x7feadc19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 -- 172.21.15.57:0/442702763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feadc104240 msgr2=0x7feadc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feadc104240 0x7feadc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 -- 172.21.15.57:0/442702763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1a2040 con 0x7feadc105df0 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feadbe00640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 0x7feadc19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:11.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.206+0000 7feae1600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc1a19d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fead4032380 tx=0x7fead4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.207+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4041020 con 0x7feadc105df0 2024-04-13T05:15:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.207+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead4003e20 con 0x7feadc105df0 2024-04-13T05:15:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.207+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1a22d0 con 0x7feadc105df0 2024-04-13T05:15:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.207+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4039040 con 0x7feadc105df0 2024-04-13T05:15:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.207+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a27c0 con 0x7feadc105df0 2024-04-13T05:15:11.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.208+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc10b680 con 0x7feadc105df0 2024-04-13T05:15:11.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.211+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fead4044020 con 0x7feadc105df0 2024-04-13T05:15:11.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.212+0000 7fead9600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feac40a3bb0 0x7feac40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.212+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fead40f2da0 con 0x7feadc105df0 2024-04-13T05:15:11.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.212+0000 7feae0c00640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feac40a3bb0 0x7feac40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.213+0000 7feae0c00640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feac40a3bb0 0x7feac40a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fead0005fd0 tx=0x7fead0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:11.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.215+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fead404b020 con 0x7feadc105df0 2024-04-13T05:15:11.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.422+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feadc02d7b0 con 0x7feadc105df0 2024-04-13T05:15:11.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.426+0000 7fead9600640 1 -- 172.21.15.57:0/442702763 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24054 (secure 0 0 0) 0x7fead40333a0 con 0x7feadc105df0 2024-04-13T05:15:11.427 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:11.427 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:14:58.570092+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:11.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feac40a3bb0 msgr2=0x7feac40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feac40a3bb0 0x7feac40a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fead0005fd0 tx=0x7fead0005850 comp rx=0 tx=0).stop 2024-04-13T05:15:11.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 msgr2=0x7feadc1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc1a19d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fead4032380 tx=0x7fead4032820 comp rx=0 tx=0).stop 2024-04-13T05:15:11.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 shutdown_connections 2024-04-13T05:15:11.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feac40a3bb0 0x7feac40a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feadc105df0 0x7feadc1a19d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feadc105430 0x7feadc19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 --2- 172.21.15.57:0/442702763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feadc104240 0x7feadc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 >> 172.21.15.57:0/442702763 conn(0x7feadc0ff7b0 msgr2=0x7feadc108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:11.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 shutdown_connections 2024-04-13T05:15:11.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.429+0000 7feae3600640 1 -- 172.21.15.57:0/442702763 wait complete. 2024-04-13T05:15:11.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:11 smithi057 ceph-mon[32331]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 195 B/s wr, 0 op/s 2024-04-13T05:15:11.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:15:11.453 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:15:11.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:11 smithi083 ceph-mon[35133]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 195 B/s wr, 0 op/s 2024-04-13T05:15:11.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:11 smithi189 ceph-mon[35531]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 195 B/s wr, 0 op/s 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 -- 172.21.15.57:0/2647667503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc077bc0 msgr2=0x7f4bcc078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 --2- 172.21.15.57:0/2647667503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc077bc0 0x7f4bcc078020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009900 tx=0x7f4bb402bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 -- 172.21.15.57:0/2647667503 shutdown_connections 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 --2- 172.21.15.57:0/2647667503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bcc078560 0x7f4bcc078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 --2- 172.21.15.57:0/2647667503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc077bc0 0x7f4bcc078020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 --2- 172.21.15.57:0/2647667503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bcc076980 0x7f4bcc076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.684+0000 7f4bcb200640 1 -- 172.21.15.57:0/2647667503 >> 172.21.15.57:0/2647667503 conn(0x7f4bcc0ff6d0 msgr2=0x7f4bcc101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.685+0000 7f4bcb200640 1 -- 172.21.15.57:0/2647667503 shutdown_connections 2024-04-13T05:15:11.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.685+0000 7f4bcb200640 1 -- 172.21.15.57:0/2647667503 wait complete. 2024-04-13T05:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 Processor -- start 2024-04-13T05:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 -- start start 2024-04-13T05:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 0x7f4bcc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bcc077bc0 0x7f4bcc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bcc078560 0x7f4bcc1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc114080 con 0x7f4bcc076980 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc113f00 con 0x7f4bcc078560 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bcb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc114200 con 0x7f4bcc077bc0 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bc9e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 0x7f4bcc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bc9e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 0x7f4bcc19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49594/0 (socket says 172.21.15.57:49594) 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.686+0000 7f4bc9e00640 1 -- 172.21.15.57:0/768746670 learned_addr learned my addr 172.21.15.57:0/768746670 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9400640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bcc077bc0 0x7f4bcc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bca800640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bcc078560 0x7f4bcc1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9e00640 1 -- 172.21.15.57:0/768746670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bcc077bc0 msgr2=0x7f4bcc19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9e00640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bcc077bc0 0x7f4bcc19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9e00640 1 -- 172.21.15.57:0/768746670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bcc078560 msgr2=0x7f4bcc1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9e00640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bcc078560 0x7f4bcc1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9e00640 1 -- 172.21.15.57:0/768746670 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc1a6350 con 0x7f4bcc076980 2024-04-13T05:15:11.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.687+0000 7f4bc9e00640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 0x7f4bcc19e730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d950 tx=0x7f4bbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:11.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.688+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc017070 con 0x7f4bcc076980 2024-04-13T05:15:11.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.688+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bbc009c30 con 0x7f4bcc076980 2024-04-13T05:15:11.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.688+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00f620 con 0x7f4bcc076980 2024-04-13T05:15:11.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.688+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc1a6640 con 0x7f4bcc076980 2024-04-13T05:15:11.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.688+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a6be0 con 0x7f4bcc076980 2024-04-13T05:15:11.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.689+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc110ab0 con 0x7f4bcc076980 2024-04-13T05:15:11.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.691+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4bbc0109a0 con 0x7f4bcc076980 2024-04-13T05:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.693+0000 7f4bc2a00640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb00a3c30 0x7f4bb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.693+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4bbc0d0300 con 0x7f4bcc076980 2024-04-13T05:15:11.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.693+0000 7f4bc9400640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb00a3c30 0x7f4bb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:11.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.694+0000 7f4bc9400640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb00a3c30 0x7f4bb00a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc19fc40 tx=0x7f4bb4005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:11.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.697+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bbc009de0 con 0x7f4bcc076980 2024-04-13T05:15:11.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.910+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4bcc062bc0 con 0x7f4bcc076980 2024-04-13T05:15:11.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.911+0000 7f4bc2a00640 1 -- 172.21.15.57:0/768746670 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f4bbc09ca40 con 0x7f4bcc076980 2024-04-13T05:15:11.912 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb00a3c30 msgr2=0x7f4bb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb00a3c30 0x7f4bb00a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc19fc40 tx=0x7f4bb4005be0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 msgr2=0x7f4bcc19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 0x7f4bcc19e730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d950 tx=0x7f4bbc00de10 comp rx=0 tx=0).stop 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 shutdown_connections 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb00a3c30 0x7f4bb00a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bcc078560 0x7f4bcc1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bcc077bc0 0x7f4bcc19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 --2- 172.21.15.57:0/768746670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bcc076980 0x7f4bcc19e730 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 >> 172.21.15.57:0/768746670 conn(0x7f4bcc0ff6d0 msgr2=0x7f4bcc102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 shutdown_connections 2024-04-13T05:15:11.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:11.914+0000 7f4bcb200640 1 -- 172.21.15.57:0/768746670 wait complete. 2024-04-13T05:15:11.933 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:15:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 -- 172.21.15.57:0/346287092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc10a040 msgr2=0x7f14bc10e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 --2- 172.21.15.57:0/346287092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc10a040 0x7f14bc10e900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009be0 tx=0x7f14b402bef0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 -- 172.21.15.57:0/346287092 shutdown_connections 2024-04-13T05:15:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 --2- 172.21.15.57:0/346287092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc10a040 0x7f14bc10e900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 --2- 172.21.15.57:0/346287092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14bc1096a0 0x7f14bc109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 --2- 172.21.15.57:0/346287092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc073400 0x7f14bc073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.174+0000 7f14c2e00640 1 -- 172.21.15.57:0/346287092 >> 172.21.15.57:0/346287092 conn(0x7f14bc06eed0 msgr2=0x7f14bc071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:12.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.175+0000 7f14c2e00640 1 -- 172.21.15.57:0/346287092 shutdown_connections 2024-04-13T05:15:12.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.175+0000 7f14c2e00640 1 -- 172.21.15.57:0/346287092 wait complete. 2024-04-13T05:15:12.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.176+0000 7f14c2e00640 1 Processor -- start 2024-04-13T05:15:12.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.176+0000 7f14c2e00640 1 -- start start 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.176+0000 7f14c2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14bc073400 0x7f14bc1a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.176+0000 7f14c2e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 0x7f14bc1a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc10a040 0x7f14bc1ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c2e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc11c840 con 0x7f14bc1096a0 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc11c6c0 con 0x7f14bc10a040 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc11c9c0 con 0x7f14bc073400 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 0x7f14bc1a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 0x7f14bc1a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49604/0 (socket says 172.21.15.57:49604) 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 -- 172.21.15.57:0/454043484 learned_addr learned my addr 172.21.15.57:0/454043484 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1a00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14bc073400 0x7f14bc1a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c2400640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc10a040 0x7f14bc1ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 -- 172.21.15.57:0/454043484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14bc073400 msgr2=0x7f14bc1a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14bc073400 0x7f14bc1a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 -- 172.21.15.57:0/454043484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc10a040 msgr2=0x7f14bc1ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc10a040 0x7f14bc1ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c1000640 1 -- 172.21.15.57:0/454043484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc1aeb20 con 0x7f14bc1096a0 2024-04-13T05:15:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.177+0000 7f14c2400640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc10a040 0x7f14bc1ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.178+0000 7f14c1000640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 0x7f14bc1a7440 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b4a0 tx=0x7f14a800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.178+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8014070 con 0x7f14bc1096a0 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.178+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a8010ac0 con 0x7f14bc1096a0 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.178+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8017c30 con 0x7f14bc1096a0 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.178+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc1aee10 con 0x7f14bc1096a0 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.178+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc1af350 con 0x7f14bc1096a0 2024-04-13T05:15:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.180+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc10e000 con 0x7f14bc1096a0 2024-04-13T05:15:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.182+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14a8017070 con 0x7f14bc1096a0 2024-04-13T05:15:12.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.184+0000 7f14ba000640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14a40a3c00 0x7f14a40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.184+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f14a80cff10 con 0x7f14bc1096a0 2024-04-13T05:15:12.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.184+0000 7f14c1a00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14a40a3c00 0x7f14a40a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.185+0000 7f14c1a00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14a40a3c00 0x7f14a40a60c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005fd0 tx=0x7f14b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.188+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14a809c530 con 0x7f14bc1096a0 2024-04-13T05:15:12.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/442702763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:12.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-13T05:15:12.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-13T05:15:12.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-13T05:15:12.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-13T05:15:12.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-04-13T05:15:12.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-13T05:15:12.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-13T05:15:12.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-13T05:15:12.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 10]}]: dispatch 2024-04-13T05:15:12.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-13T05:15:12.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:12 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/768746670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:15:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.394+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14bc062bc0 con 0x7f14bc1096a0 2024-04-13T05:15:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.395+0000 7f14ba000640 1 -- 172.21.15.57:0/454043484 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f14a809be30 con 0x7f14bc1096a0 2024-04-13T05:15:12.395 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:15:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.397+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14a40a3c00 msgr2=0x7f14a40a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.397+0000 7f14c2e00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14a40a3c00 0x7f14a40a60c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005fd0 tx=0x7f14b0005850 comp rx=0 tx=0).stop 2024-04-13T05:15:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.397+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 msgr2=0x7f14bc1a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.397+0000 7f14c2e00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 0x7f14bc1a7440 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b4a0 tx=0x7f14a800b960 comp rx=0 tx=0).stop 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 shutdown_connections 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14a40a3c00 0x7f14a40a60c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14bc10a040 0x7f14bc1ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14bc1096a0 0x7f14bc1a7440 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 --2- 172.21.15.57:0/454043484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14bc073400 0x7f14bc1a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 >> 172.21.15.57:0/454043484 conn(0x7f14bc06eed0 msgr2=0x7f14bc070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 shutdown_connections 2024-04-13T05:15:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.398+0000 7f14c2e00640 1 -- 172.21.15.57:0/454043484 wait complete. 2024-04-13T05:15:12.417 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/442702763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-04-13T05:15:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-13T05:15:12.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-13T05:15:12.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-13T05:15:12.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 10]}]: dispatch 2024-04-13T05:15:12.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-13T05:15:12.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:12 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/768746670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/442702763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 10]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-13T05:15:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:12 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/768746670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 -- 172.21.15.57:0/572816445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 msgr2=0x7f5bac109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 --2- 172.21.15.57:0/572816445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac109c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009900 tx=0x7f5b9802bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 -- 172.21.15.57:0/572816445 shutdown_connections 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 --2- 172.21.15.57:0/572816445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac10a1d0 0x7f5bac110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 --2- 172.21.15.57:0/572816445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac109c90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 --2- 172.21.15.57:0/572816445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac108620 0x7f5bac108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 -- 172.21.15.57:0/572816445 >> 172.21.15.57:0/572816445 conn(0x7f5bac0775c0 msgr2=0x7f5bac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 -- 172.21.15.57:0/572816445 shutdown_connections 2024-04-13T05:15:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.642+0000 7f5bb4600640 1 -- 172.21.15.57:0/572816445 wait complete. 2024-04-13T05:15:12.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.643+0000 7f5bb4600640 1 Processor -- start 2024-04-13T05:15:12.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 -- start start 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac108620 0x7f5bac19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac10a1d0 0x7f5bac1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1141b0 con 0x7f5bac109810 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac114030 con 0x7f5bac108620 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb4600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac114330 con 0x7f5bac10a1d0 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49614/0 (socket says 172.21.15.57:49614) 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.644+0000 7f5bb1200640 1 -- 172.21.15.57:0/3709205083 learned_addr learned my addr 172.21.15.57:0/3709205083 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1c00640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac108620 0x7f5bac19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1200640 1 -- 172.21.15.57:0/3709205083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac10a1d0 msgr2=0x7f5bac1a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1200640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac10a1d0 0x7f5bac1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1200640 1 -- 172.21.15.57:0/3709205083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac108620 msgr2=0x7f5bac19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1200640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac108620 0x7f5bac19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1200640 1 -- 172.21.15.57:0/3709205083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac1a64a0 con 0x7f5bac109810 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb1200640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac19edc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000c00 tx=0x7f5b980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98043020 con 0x7f5bac109810 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b980054c0 con 0x7f5bac109810 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98003d40 con 0x7f5bac109810 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.645+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac1a6730 con 0x7f5bac109810 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.646+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac1a6cd0 con 0x7f5bac109810 2024-04-13T05:15:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.647+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac10f9c0 con 0x7f5bac109810 2024-04-13T05:15:12.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.649+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b98046020 con 0x7f5bac109810 2024-04-13T05:15:12.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.651+0000 7f5baa000640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b940a3c30 0x7f5b940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:12.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.651+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5b980f2cc0 con 0x7f5bac109810 2024-04-13T05:15:12.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.651+0000 7f5bb1c00640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b940a3c30 0x7f5b940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.652+0000 7f5bb1c00640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b940a3c30 0x7f5b940a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0005fd0 tx=0x7f5ba0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:12.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.655+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b980f9080 con 0x7f5bac109810 2024-04-13T05:15:12.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.861+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5bac062bc0 con 0x7f5bac109810 2024-04-13T05:15:12.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.862+0000 7f5baa000640 1 -- 172.21.15.57:0/3709205083 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f5b9804d020 con 0x7f5bac109810 2024-04-13T05:15:12.863 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b940a3c30 msgr2=0x7f5b940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b940a3c30 0x7f5b940a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0005fd0 tx=0x7f5ba0005850 comp rx=0 tx=0).stop 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 msgr2=0x7f5bac19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac19edc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000c00 tx=0x7f5b980042f0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 shutdown_connections 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b940a3c30 0x7f5b940a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac10a1d0 0x7f5bac1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bac109810 0x7f5bac19edc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 --2- 172.21.15.57:0/3709205083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac108620 0x7f5bac19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 >> 172.21.15.57:0/3709205083 conn(0x7f5bac0775c0 msgr2=0x7f5bac10cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.865+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 shutdown_connections 2024-04-13T05:15:12.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:12.866+0000 7f5bb4600640 1 -- 172.21.15.57:0/3709205083 wait complete. 2024-04-13T05:15:12.884 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 -- 172.21.15.57:0/2665948176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c108620 msgr2=0x7f5b1c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 --2- 172.21.15.57:0/2665948176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c108620 0x7f5b1c108a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009900 tx=0x7f5b0c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 -- 172.21.15.57:0/2665948176 shutdown_connections 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 --2- 172.21.15.57:0/2665948176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c109810 0x7f5b1c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 --2- 172.21.15.57:0/2665948176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b1c10a1d0 0x7f5b1c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 --2- 172.21.15.57:0/2665948176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c108620 0x7f5b1c108a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 -- 172.21.15.57:0/2665948176 >> 172.21.15.57:0/2665948176 conn(0x7f5b1c0775c0 msgr2=0x7f5b1c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 -- 172.21.15.57:0/2665948176 shutdown_connections 2024-04-13T05:15:13.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.100+0000 7f5b21e00640 1 -- 172.21.15.57:0/2665948176 wait complete. 2024-04-13T05:15:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.101+0000 7f5b21e00640 1 Processor -- start 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.101+0000 7f5b21e00640 1 -- start start 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b1c108620 0x7f5b1c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 0x7f5b1c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c10a1d0 0x7f5b1c1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b21e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1141a0 con 0x7f5b1c109810 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b21e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c114020 con 0x7f5b1c10a1d0 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b21e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c114320 con 0x7f5b1c108620 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 0x7f5b1c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 0x7f5b1c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49624/0 (socket says 172.21.15.57:49624) 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 -- 172.21.15.57:0/413398964 learned_addr learned my addr 172.21.15.57:0/413398964 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1aa00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b1c108620 0x7f5b1c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1b400640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c10a1d0 0x7f5b1c1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 -- 172.21.15.57:0/413398964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b1c108620 msgr2=0x7f5b1c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b1c108620 0x7f5b1c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 -- 172.21.15.57:0/413398964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c10a1d0 msgr2=0x7f5b1c1a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c10a1d0 0x7f5b1c1a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1a000640 1 -- 172.21.15.57:0/413398964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c1a6460 con 0x7f5b1c109810 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.102+0000 7f5b1b400640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c10a1d0 0x7f5b1c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.103+0000 7f5b1a000640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 0x7f5b1c19edd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c920 tx=0x7f5b0800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.103+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08007c70 con 0x7f5b1c109810 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.103+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b08007e10 con 0x7f5b1c109810 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.103+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c1a6750 con 0x7f5b1c109810 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.103+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08018820 con 0x7f5b1c109810 2024-04-13T05:15:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.103+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1a6ca0 con 0x7f5b1c109810 2024-04-13T05:15:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.105+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c10fa10 con 0x7f5b1c109810 2024-04-13T05:15:13.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.107+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b080189c0 con 0x7f5b1c109810 2024-04-13T05:15:13.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.109+0000 7f5b20a00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b000a3bb0 0x7f5b000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.109+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5b080d09a0 con 0x7f5b1c109810 2024-04-13T05:15:13.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.109+0000 7f5b1aa00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b000a3bb0 0x7f5b000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.110+0000 7f5b1aa00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b000a3bb0 0x7f5b000a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c02c240 tx=0x7f5b0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:13.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.113+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b08014030 con 0x7f5b1c109810 2024-04-13T05:15:13.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.336+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5b1c062bf0 con 0x7f5b1c109810 2024-04-13T05:15:13.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.337+0000 7f5b20a00640 1 -- 172.21.15.57:0/413398964 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f5b0809cfc0 con 0x7f5b1c109810 2024-04-13T05:15:13.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:15:13.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b000a3bb0 msgr2=0x7f5b000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b000a3bb0 0x7f5b000a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c02c240 tx=0x7f5b0c000bc0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 msgr2=0x7f5b1c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 0x7f5b1c19edd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c920 tx=0x7f5b0800cde0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 shutdown_connections 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5b000a3bb0 0x7f5b000a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1c10a1d0 0x7f5b1c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b1c109810 0x7f5b1c19edd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 --2- 172.21.15.57:0/413398964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b1c108620 0x7f5b1c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 >> 172.21.15.57:0/413398964 conn(0x7f5b1c0775c0 msgr2=0x7f5b1c10cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 shutdown_connections 2024-04-13T05:15:13.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.340+0000 7f5b21e00640 1 -- 172.21.15.57:0/413398964 wait complete. 2024-04-13T05:15:13.361 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-13T05:15:13.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 10]}]': finished 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: osdmap e91: 12 total, 12 up, 12 in 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/454043484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3709205083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:15:13.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:13 smithi057 ceph-mon[32331]: osdmap e92: 12 total, 12 up, 12 in 2024-04-13T05:15:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-13T05:15:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-13T05:15:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-13T05:15:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 10]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: osdmap e91: 12 total, 12 up, 12 in 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/454043484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3709205083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:15:13.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:13 smithi083 ceph-mon[35133]: osdmap e92: 12 total, 12 up, 12 in 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 10]}]': finished 2024-04-13T05:15:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-13T05:15:13.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: osdmap e91: 12 total, 12 up, 12 in 2024-04-13T05:15:13.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/454043484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:15:13.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3709205083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:15:13.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:13 smithi189 ceph-mon[35531]: osdmap e92: 12 total, 12 up, 12 in 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.588+0000 7f1f4aa00640 1 -- 172.21.15.57:0/205426031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f44105cb0 msgr2=0x7f1f4410c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.588+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/205426031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f44105cb0 0x7f1f4410c540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009900 tx=0x7f1f3c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 -- 172.21.15.57:0/205426031 shutdown_connections 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/205426031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f441052f0 0x7f1f44105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/205426031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f44105cb0 0x7f1f4410c540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/205426031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44104100 0x7f1f44104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 -- 172.21.15.57:0/205426031 >> 172.21.15.57:0/205426031 conn(0x7f1f440ff670 msgr2=0x7f1f44101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 -- 172.21.15.57:0/205426031 shutdown_connections 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.589+0000 7f1f4aa00640 1 -- 172.21.15.57:0/205426031 wait complete. 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 Processor -- start 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 -- start start 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44104100 0x7f1f4419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 0x7f1f4419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44105cb0 0x7f1f441a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410fc90 con 0x7f1f441052f0 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410fb10 con 0x7f1f44104100 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f4aa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410fe10 con 0x7f1f44105cb0 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f48c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 0x7f1f4419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.590+0000 7f1f48c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 0x7f1f4419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49632/0 (socket says 172.21.15.57:49632) 2024-04-13T05:15:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 -- 172.21.15.57:0/2930414207 learned_addr learned my addr 172.21.15.57:0/2930414207 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f49600640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44104100 0x7f1f4419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f4a000640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44105cb0 0x7f1f441a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 -- 172.21.15.57:0/2930414207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44105cb0 msgr2=0x7f1f441a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44105cb0 0x7f1f441a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 -- 172.21.15.57:0/2930414207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44104100 msgr2=0x7f1f4419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44104100 0x7f1f4419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 -- 172.21.15.57:0/2930414207 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f441a1ee0 con 0x7f1f441052f0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f49600640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44104100 0x7f1f4419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.591+0000 7f1f48c00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 0x7f1f4419a800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000d950 tx=0x7f1f3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.592+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30017070 con 0x7f1f441052f0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.592+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f30009c30 con 0x7f1f441052f0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.592+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f441a21d0 con 0x7f1f441052f0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.592+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000f620 con 0x7f1f441052f0 2024-04-13T05:15:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.592+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441a2770 con 0x7f1f441052f0 2024-04-13T05:15:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.593+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4410b420 con 0x7f1f441052f0 2024-04-13T05:15:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.595+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f300109a0 con 0x7f1f441052f0 2024-04-13T05:15:13.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.597+0000 7f1f42000640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:13.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.597+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1f300d0420 con 0x7f1f441052f0 2024-04-13T05:15:13.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.597+0000 7f1f49600640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:13.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.598+0000 7f1f49600640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f44105150 tx=0x7f1f38009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:13.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.602+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f30009de0 con 0x7f1f441052f0 2024-04-13T05:15:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.824+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f44062bc0 con 0x7f1f441052f0 2024-04-13T05:15:13.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.825+0000 7f1f42000640 1 -- 172.21.15.57:0/2930414207 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+588 (secure 0 0 0) 0x7f1f3009ca40 con 0x7f1f441052f0 2024-04-13T05:15:13.826 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2270359290 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:13.826 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1023620189 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:13.826 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:13.826 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/863867962 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:13.826 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1502699862 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:13.827 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:13.827 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:13.827 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1172825336 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:13.827 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:13.827 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/980869677 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:13.827 INFO:teuthology.orchestra.run.smithi057.stderr:listed 10 entries 2024-04-13T05:15:13.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f2c0a3c30 msgr2=0x7f1f2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f44105150 tx=0x7f1f38009540 comp rx=0 tx=0).stop 2024-04-13T05:15:13.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 msgr2=0x7f1f4419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 0x7f1f4419a800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000d950 tx=0x7f1f3000de10 comp rx=0 tx=0).stop 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 shutdown_connections 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f2c0a3c30 0x7f1f2c0a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44105cb0 0x7f1f441a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f441052f0 0x7f1f4419a800 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 --2- 172.21.15.57:0/2930414207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44104100 0x7f1f4419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 >> 172.21.15.57:0/2930414207 conn(0x7f1f440ff670 msgr2=0x7f1f441085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.828+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 shutdown_connections 2024-04-13T05:15:13.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:13.829+0000 7f1f4aa00640 1 -- 172.21.15.57:0/2930414207 wait complete. 2024-04-13T05:15:13.849 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:15:13.849 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T05:15:13.904 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:14.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.136+0000 7fbec4a00640 1 -- 172.21.15.57:0/2469536147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 msgr2=0x7fbec0102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.136+0000 7fbec4a00640 1 --2- 172.21.15.57:0/2469536147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec0102250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009900 tx=0x7fbeb002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 -- 172.21.15.57:0/2469536147 shutdown_connections 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 --2- 172.21.15.57:0/2469536147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec0103940 0x7fbec010a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 --2- 172.21.15.57:0/2469536147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 0x7fbec0103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 --2- 172.21.15.57:0/2469536147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec0102250 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 -- 172.21.15.57:0/2469536147 >> 172.21.15.57:0/2469536147 conn(0x7fbec00fd3a0 msgr2=0x7fbec00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 -- 172.21.15.57:0/2469536147 shutdown_connections 2024-04-13T05:15:14.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.137+0000 7fbec4a00640 1 -- 172.21.15.57:0/2469536147 wait complete. 2024-04-13T05:15:14.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 Processor -- start 2024-04-13T05:15:14.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 -- start start 2024-04-13T05:15:14.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 0x7fbec019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec0103940 0x7fbec01a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbec010da80 con 0x7fbec0101e30 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbec010d900 con 0x7fbec0103940 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbec4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbec010dc00 con 0x7fbec0102f80 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.138+0000 7fbebe800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49650/0 (socket says 172.21.15.57:49650) 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 -- 172.21.15.57:0/3109001047 learned_addr learned my addr 172.21.15.57:0/3109001047 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebf200640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec0103940 0x7fbec01a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 -- 172.21.15.57:0/3109001047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 msgr2=0x7fbec019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebde00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 0x7fbec019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 0x7fbec019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 -- 172.21.15.57:0/3109001047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec0103940 msgr2=0x7fbec01a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec0103940 0x7fbec01a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 -- 172.21.15.57:0/3109001047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec01a2090 con 0x7fbec0101e30 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebde00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 0x7fbec019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:14.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.139+0000 7fbebe800640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec019a470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0036040 tx=0x7fbeb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:14.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.140+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0043020 con 0x7fbec0101e30 2024-04-13T05:15:14.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.140+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeb00054c0 con 0x7fbec0101e30 2024-04-13T05:15:14.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.140+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec01a2320 con 0x7fbec0101e30 2024-04-13T05:15:14.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.140+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003d70 con 0x7fbec0101e30 2024-04-13T05:15:14.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.140+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec01a28f0 con 0x7fbec0101e30 2024-04-13T05:15:14.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.141+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec01a2c10 con 0x7fbec0101e30 2024-04-13T05:15:14.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.143+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbeb0046020 con 0x7fbec0101e30 2024-04-13T05:15:14.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.145+0000 7fbeb7400640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbea40a3bb0 0x7fbea40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.145+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbeb00f2d00 con 0x7fbec0101e30 2024-04-13T05:15:14.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.146+0000 7fbebde00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbea40a3bb0 0x7fbea40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.146+0000 7fbebde00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbea40a3bb0 0x7fbea40a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbeac006fd0 tx=0x7fbeac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:14.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.150+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeb004d020 con 0x7fbec0101e30 2024-04-13T05:15:14.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.360+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbec01a2de0 con 0x7fbec0101e30 2024-04-13T05:15:14.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.364+0000 7fbeb7400640 1 -- 172.21.15.57:0/3109001047 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24583 (secure 0 0 0) 0x7fbeb00a9020 con 0x7fbec0101e30 2024-04-13T05:15:14.365 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:14.365 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:15:13.150673+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:14.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbea40a3bb0 msgr2=0x7fbea40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbea40a3bb0 0x7fbea40a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbeac006fd0 tx=0x7fbeac008040 comp rx=0 tx=0).stop 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 msgr2=0x7fbec019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec019a470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0036040 tx=0x7fbeb00042f0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 shutdown_connections 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbea40a3bb0 0x7fbea40a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec0103940 0x7fbec01a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec0102f80 0x7fbec019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 --2- 172.21.15.57:0/3109001047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbec0101e30 0x7fbec019a470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 >> 172.21.15.57:0/3109001047 conn(0x7fbec00fd3a0 msgr2=0x7fbec00741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 shutdown_connections 2024-04-13T05:15:14.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.367+0000 7fbec4a00640 1 -- 172.21.15.57:0/3109001047 wait complete. 2024-04-13T05:15:14.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:15:14.387 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:15:14.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:14 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/413398964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:15:14.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:14 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2930414207' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:15:14.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:14 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/413398964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:15:14.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:14 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2930414207' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:15:14.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:14 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/413398964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:15:14.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:14 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2930414207' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 -- 172.21.15.57:0/3890300704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810101ff0 msgr2=0x7f4810102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 --2- 172.21.15.57:0/3890300704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810101ff0 0x7f4810102410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4800009900 tx=0x7f480002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 -- 172.21.15.57:0/3890300704 shutdown_connections 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 --2- 172.21.15.57:0/3890300704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48101031e0 0x7f4810103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 --2- 172.21.15.57:0/3890300704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4810103ba0 0x7f481010a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 --2- 172.21.15.57:0/3890300704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810101ff0 0x7f4810102410 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 -- 172.21.15.57:0/3890300704 >> 172.21.15.57:0/3890300704 conn(0x7f48100fd580 msgr2=0x7f48100ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 -- 172.21.15.57:0/3890300704 shutdown_connections 2024-04-13T05:15:14.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.613+0000 7f4816a00640 1 -- 172.21.15.57:0/3890300704 wait complete. 2024-04-13T05:15:14.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.614+0000 7f4816a00640 1 Processor -- start 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.614+0000 7f4816a00640 1 -- start start 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4816a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4810101ff0 0x7f4810198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4816a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48101031e0 0x7f4810198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4816a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 0x7f481019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4816a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f481010db30 con 0x7f4810103ba0 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4816a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f481010d9b0 con 0x7f4810101ff0 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4816a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f481010dcb0 con 0x7f48101031e0 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4814a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 0x7f481019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4814a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 0x7f481019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49668/0 (socket says 172.21.15.57:49668) 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f4814a00640 1 -- 172.21.15.57:0/4106541608 learned_addr learned my addr 172.21.15.57:0/4106541608 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f480fe00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4810101ff0 0x7f4810198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.615+0000 7f480f400640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48101031e0 0x7f4810198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4814a00640 1 -- 172.21.15.57:0/4106541608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48101031e0 msgr2=0x7f4810198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4814a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48101031e0 0x7f4810198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4814a00640 1 -- 172.21.15.57:0/4106541608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4810101ff0 msgr2=0x7f4810198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4814a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4810101ff0 0x7f4810198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4814a00640 1 -- 172.21.15.57:0/4106541608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481019fe60 con 0x7f4810103ba0 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f480f400640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48101031e0 0x7f4810198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4814a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 0x7f481019f7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f480800eaa0 tx=0x7f480800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480800cdb0 con 0x7f4810103ba0 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4808005020 con 0x7f4810103ba0 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48101a0150 con 0x7f4810103ba0 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080116d0 con 0x7f4810103ba0 2024-04-13T05:15:14.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.616+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101a06a0 con 0x7f4810103ba0 2024-04-13T05:15:14.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.618+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4810109330 con 0x7f4810103ba0 2024-04-13T05:15:14.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.620+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48080051d0 con 0x7f4810103ba0 2024-04-13T05:15:14.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.622+0000 7f480cc00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47f40a3bb0 0x7f47f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:14.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.622+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f48080d0930 con 0x7f4810103ba0 2024-04-13T05:15:14.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.623+0000 7f480fe00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47f40a3bb0 0x7f47f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:14.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.623+0000 7f480fe00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47f40a3bb0 0x7f47f40a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4800005ab0 tx=0x7f4800005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:14.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.627+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4808016030 con 0x7f4810103ba0 2024-04-13T05:15:14.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.832+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4810062bc0 con 0x7f4810103ba0 2024-04-13T05:15:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.833+0000 7f480cc00640 1 -- 172.21.15.57:0/4106541608 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f480809cf50 con 0x7f4810103ba0 2024-04-13T05:15:14.833 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:15:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.835+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47f40a3bb0 msgr2=0x7f47f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.835+0000 7f4816a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47f40a3bb0 0x7f47f40a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4800005ab0 tx=0x7f4800005fd0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.835+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 msgr2=0x7f481019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.835+0000 7f4816a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 0x7f481019f7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f480800eaa0 tx=0x7f480800ef60 comp rx=0 tx=0).stop 2024-04-13T05:15:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 shutdown_connections 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47f40a3bb0 0x7f47f40a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4810103ba0 0x7f481019f7f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48101031e0 0x7f4810198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 --2- 172.21.15.57:0/4106541608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4810101ff0 0x7f4810198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 >> 172.21.15.57:0/4106541608 conn(0x7f48100fd580 msgr2=0x7f4810106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 shutdown_connections 2024-04-13T05:15:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:14.836+0000 7f4816a00640 1 -- 172.21.15.57:0/4106541608 wait complete. 2024-04-13T05:15:14.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:15:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 -- 172.21.15.57:0/2094198659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec104100 msgr2=0x7f51ec104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 --2- 172.21.15.57:0/2094198659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec104100 0x7f51ec104520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009900 tx=0x7f51dc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 -- 172.21.15.57:0/2094198659 shutdown_connections 2024-04-13T05:15:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 --2- 172.21.15.57:0/2094198659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 0x7f51ec105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 --2- 172.21.15.57:0/2094198659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec105cb0 0x7f51ec10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 --2- 172.21.15.57:0/2094198659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec104100 0x7f51ec104520 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 -- 172.21.15.57:0/2094198659 >> 172.21.15.57:0/2094198659 conn(0x7f51ec0ff670 msgr2=0x7f51ec101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.083+0000 7f51ebe00640 1 -- 172.21.15.57:0/2094198659 shutdown_connections 2024-04-13T05:15:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.084+0000 7f51ebe00640 1 -- 172.21.15.57:0/2094198659 wait complete. 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.084+0000 7f51ebe00640 1 Processor -- start 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 -- start start 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec104100 0x7f51ec19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 0x7f51ec19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 0x7f51ec1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10fc90 con 0x7f51ec105cb0 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10fb10 con 0x7f51ec1052f0 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51ebe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec10fe10 con 0x7f51ec104100 2024-04-13T05:15:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51eb400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 0x7f51ec1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51eb400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 0x7f51ec1a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34646/0 (socket says 172.21.15.57:34646) 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51eb400640 1 -- 172.21.15.57:0/1923344372 learned_addr learned my addr 172.21.15.57:0/1923344372 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.085+0000 7f51eaa00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec104100 0x7f51ec19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51ea000640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 0x7f51ec19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eb400640 1 -- 172.21.15.57:0/1923344372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec104100 msgr2=0x7f51ec19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eb400640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec104100 0x7f51ec19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eb400640 1 -- 172.21.15.57:0/1923344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 msgr2=0x7f51ec19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eb400640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 0x7f51ec19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eb400640 1 -- 172.21.15.57:0/1923344372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec1a1f40 con 0x7f51ec105cb0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51ea000640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 0x7f51ec19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eaa00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec104100 0x7f51ec19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51eb400640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 0x7f51ec1a18d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51e400d750 tx=0x7f51e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4017070 con 0x7f51ec105cb0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51e40054d0 con 0x7f51ec105cb0 2024-04-13T05:15:15.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4004c60 con 0x7f51ec105cb0 2024-04-13T05:15:15.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.086+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec1a2230 con 0x7f51ec105cb0 2024-04-13T05:15:15.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.087+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1a2730 con 0x7f51ec105cb0 2024-04-13T05:15:15.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.088+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec10b420 con 0x7f51ec105cb0 2024-04-13T05:15:15.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.090+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f51e4004e00 con 0x7f51ec105cb0 2024-04-13T05:15:15.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.091+0000 7f51e3400640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51d00a3bb0 0x7f51d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.091+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f51e40d01f0 con 0x7f51ec105cb0 2024-04-13T05:15:15.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.092+0000 7f51eaa00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51d00a3bb0 0x7f51d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.092+0000 7f51eaa00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51d00a3bb0 0x7f51d00a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51dc02c240 tx=0x7f51dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.096+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51e409c810 con 0x7f51ec105cb0 2024-04-13T05:15:15.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.304+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f51ec062bc0 con 0x7f51ec105cb0 2024-04-13T05:15:15.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.305+0000 7f51e3400640 1 -- 172.21.15.57:0/1923344372 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f51e409c110 con 0x7f51ec105cb0 2024-04-13T05:15:15.306 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51d00a3bb0 msgr2=0x7f51d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51d00a3bb0 0x7f51d00a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51dc02c240 tx=0x7f51dc005fd0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 msgr2=0x7f51ec1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 0x7f51ec1a18d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51e400d750 tx=0x7f51e400dc10 comp rx=0 tx=0).stop 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 shutdown_connections 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51d00a3bb0 0x7f51d00a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51ec1052f0 0x7f51ec19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51ec105cb0 0x7f51ec1a18d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 --2- 172.21.15.57:0/1923344372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51ec104100 0x7f51ec19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 >> 172.21.15.57:0/1923344372 conn(0x7f51ec0ff670 msgr2=0x7f51ec1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:15.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.308+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 shutdown_connections 2024-04-13T05:15:15.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.309+0000 7f51ebe00640 1 -- 172.21.15.57:0/1923344372 wait complete. 2024-04-13T05:15:15.330 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:15:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:15 smithi083 ceph-mon[35133]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-13T05:15:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:15 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3109001047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:15 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4106541608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:15:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:15 smithi057 ceph-mon[32331]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-13T05:15:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:15 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3109001047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:15 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4106541608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:15:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 -- 172.21.15.57:0/3452724584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208078560 msgr2=0x7f2208078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 --2- 172.21.15.57:0/3452724584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208078560 0x7f2208078e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2200009ca0 tx=0x7f220002c100 comp rx=0 tx=0).stop 2024-04-13T05:15:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 -- 172.21.15.57:0/3452724584 shutdown_connections 2024-04-13T05:15:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 --2- 172.21.15.57:0/3452724584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208078560 0x7f2208078e00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 --2- 172.21.15.57:0/3452724584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2208077bc0 0x7f2208078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 --2- 172.21.15.57:0/3452724584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2208076980 0x7f2208076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 -- 172.21.15.57:0/3452724584 >> 172.21.15.57:0/3452724584 conn(0x7f22080ff770 msgr2=0x7f2208101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:15.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 -- 172.21.15.57:0/3452724584 shutdown_connections 2024-04-13T05:15:15.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.560+0000 7f220cc00640 1 -- 172.21.15.57:0/3452724584 wait complete. 2024-04-13T05:15:15.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 Processor -- start 2024-04-13T05:15:15.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 -- start start 2024-04-13T05:15:15.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 0x7f220819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2208077bc0 0x7f220819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2208078560 0x7f22081a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2208114160 con 0x7f2208076980 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2208113fe0 con 0x7f2208078560 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f220cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22081142e0 con 0x7f2208077bc0 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f2205e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 0x7f220819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f2205e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 0x7f220819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34672/0 (socket says 172.21.15.57:34672) 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.562+0000 7f2205e00640 1 -- 172.21.15.57:0/976664430 learned_addr learned my addr 172.21.15.57:0/976664430 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2206800640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2208078560 0x7f22081a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205400640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2208077bc0 0x7f220819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205e00640 1 -- 172.21.15.57:0/976664430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2208077bc0 msgr2=0x7f220819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205e00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2208077bc0 0x7f220819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205e00640 1 -- 172.21.15.57:0/976664430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2208078560 msgr2=0x7f22081a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205e00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2208078560 0x7f22081a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205e00640 1 -- 172.21.15.57:0/976664430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081a6420 con 0x7f2208076980 2024-04-13T05:15:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.563+0000 7f2205e00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 0x7f220819e800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d950 tx=0x7f21f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.564+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8017070 con 0x7f2208076980 2024-04-13T05:15:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.564+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21f8009c30 con 0x7f2208076980 2024-04-13T05:15:15.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.564+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f800f620 con 0x7f2208076980 2024-04-13T05:15:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.564+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081a6710 con 0x7f2208076980 2024-04-13T05:15:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.564+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081a6c10 con 0x7f2208076980 2024-04-13T05:15:15.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.565+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208110dd0 con 0x7f2208076980 2024-04-13T05:15:15.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.567+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f21f80109a0 con 0x7f2208076980 2024-04-13T05:15:15.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.568+0000 7f21fea00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:15.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.569+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f21f80d03d0 con 0x7f2208076980 2024-04-13T05:15:15.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.569+0000 7f2205400640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:15.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.570+0000 7f2205400640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21f4005fd0 tx=0x7f21f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:15.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.572+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21f8009de0 con 0x7f2208076980 2024-04-13T05:15:15.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:15 smithi189 ceph-mon[35531]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-13T05:15:15.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:15 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3109001047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:15.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:15 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4106541608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:15:15.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.780+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f220802d870 con 0x7f2208076980 2024-04-13T05:15:15.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.781+0000 7f21fea00640 1 -- 172.21.15.57:0/976664430 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f21f809ca70 con 0x7f2208076980 2024-04-13T05:15:15.781 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:15:15.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.783+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f21ec0a3c30 msgr2=0x7f21ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.783+0000 7f220cc00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21f4005fd0 tx=0x7f21f4005850 comp rx=0 tx=0).stop 2024-04-13T05:15:15.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.783+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 msgr2=0x7f220819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:15.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 0x7f220819e800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d950 tx=0x7f21f800de10 comp rx=0 tx=0).stop 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 shutdown_connections 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f21ec0a3c30 0x7f21ec0a60f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2208078560 0x7f22081a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2208077bc0 0x7f220819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 --2- 172.21.15.57:0/976664430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2208076980 0x7f220819e800 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 >> 172.21.15.57:0/976664430 conn(0x7f22080ff770 msgr2=0x7f2208100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 shutdown_connections 2024-04-13T05:15:15.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:15.784+0000 7f220cc00640 1 -- 172.21.15.57:0/976664430 wait complete. 2024-04-13T05:15:15.805 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:15:16.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.040+0000 7fdfe2800640 1 -- 172.21.15.57:0/2966325111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 msgr2=0x7fdfdc10a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.040+0000 7fdfe2800640 1 --2- 172.21.15.57:0/2966325111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc10a000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009ca0 tx=0x7fdfd402c100 comp rx=0 tx=0).stop 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 -- 172.21.15.57:0/2966325111 shutdown_connections 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 --2- 172.21.15.57:0/2966325111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc10a000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 --2- 172.21.15.57:0/2966325111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfdc105280 0x7fdfdc107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 --2- 172.21.15.57:0/2966325111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc102950 0x7fdfdc104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 -- 172.21.15.57:0/2966325111 >> 172.21.15.57:0/2966325111 conn(0x7fdfdc0fc360 msgr2=0x7fdfdc0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 -- 172.21.15.57:0/2966325111 shutdown_connections 2024-04-13T05:15:16.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.041+0000 7fdfe2800640 1 -- 172.21.15.57:0/2966325111 wait complete. 2024-04-13T05:15:16.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 Processor -- start 2024-04-13T05:15:16.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 -- start start 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfdc102950 0x7fdfdc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc105280 0x7fdfdc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10d9f0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10d870 con 0x7fdfdc105280 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.042+0000 7fdfe2800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10db70 con 0x7fdfdc102950 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34694/0 (socket says 172.21.15.57:34694) 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 -- 172.21.15.57:0/3551719321 learned_addr learned my addr 172.21.15.57:0/3551719321 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe0a00640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc105280 0x7fdfdc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1400640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfdc102950 0x7fdfdc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 -- 172.21.15.57:0/3551719321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfdc102950 msgr2=0x7fdfdc19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfdc102950 0x7fdfdc19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 -- 172.21.15.57:0/3551719321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc105280 msgr2=0x7fdfdc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc105280 0x7fdfdc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.043+0000 7fdfe1e00640 1 -- 172.21.15.57:0/3551719321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc1a1e40 con 0x7fdfdc107bb0 2024-04-13T05:15:16.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.044+0000 7fdfe1e00640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc1a17d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4032520 tx=0x7fdfd4034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:16.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.044+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4039040 con 0x7fdfdc107bb0 2024-04-13T05:15:16.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.044+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfd4044070 con 0x7fdfdc107bb0 2024-04-13T05:15:16.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.044+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4004460 con 0x7fdfdc107bb0 2024-04-13T05:15:16.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.044+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc1a20d0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.044+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1a2610 con 0x7fdfdc107bb0 2024-04-13T05:15:16.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.045+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfdc1096a0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.047+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfd4003be0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.049+0000 7fdfda000640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.049+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdfd40f35a0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.049+0000 7fdfe1400640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.050+0000 7fdfe1400640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0005fd0 tx=0x7fdfd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:16.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.052+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfd40bfbc0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.258+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdfdc02d5e0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.258+0000 7fdfda000640 1 -- 172.21.15.57:0/3551719321 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fdfd40bf4c0 con 0x7fdfdc107bb0 2024-04-13T05:15:16.259 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:15:16.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfc40a3c30 msgr2=0x7fdfc40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0005fd0 tx=0x7fdfd0005850 comp rx=0 tx=0).stop 2024-04-13T05:15:16.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 msgr2=0x7fdfdc1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc1a17d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4032520 tx=0x7fdfd4034880 comp rx=0 tx=0).stop 2024-04-13T05:15:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 shutdown_connections 2024-04-13T05:15:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfc40a3c30 0x7fdfc40a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfdc105280 0x7fdfdc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfdc107bb0 0x7fdfdc1a17d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 --2- 172.21.15.57:0/3551719321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfdc102950 0x7fdfdc19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 >> 172.21.15.57:0/3551719321 conn(0x7fdfdc0fc360 msgr2=0x7fdfdc101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:16.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.261+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 shutdown_connections 2024-04-13T05:15:16.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.262+0000 7fdfe2800640 1 -- 172.21.15.57:0/3551719321 wait complete. 2024-04-13T05:15:16.281 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T05:15:16.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:16 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1923344372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:15:16.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:16 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/976664430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:15:16.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.507+0000 7fc7f7200640 1 -- 172.21.15.57:0/3751807973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8104100 msgr2=0x7fc7f8104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.507+0000 7fc7f7200640 1 --2- 172.21.15.57:0/3751807973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8104100 0x7fc7f8104520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009ca0 tx=0x7fc7e802c100 comp rx=0 tx=0).stop 2024-04-13T05:15:16.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.507+0000 7fc7f7200640 1 -- 172.21.15.57:0/3751807973 shutdown_connections 2024-04-13T05:15:16.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.507+0000 7fc7f7200640 1 --2- 172.21.15.57:0/3751807973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f81052f0 0x7fc7f8105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.508+0000 7fc7f7200640 1 --2- 172.21.15.57:0/3751807973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8105cb0 0x7fc7f810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.508+0000 7fc7f7200640 1 --2- 172.21.15.57:0/3751807973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8104100 0x7fc7f8104520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.508+0000 7fc7f7200640 1 -- 172.21.15.57:0/3751807973 >> 172.21.15.57:0/3751807973 conn(0x7fc7f80ff670 msgr2=0x7fc7f8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.508+0000 7fc7f7200640 1 -- 172.21.15.57:0/3751807973 shutdown_connections 2024-04-13T05:15:16.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.508+0000 7fc7f7200640 1 -- 172.21.15.57:0/3751807973 wait complete. 2024-04-13T05:15:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 Processor -- start 2024-04-13T05:15:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 -- start start 2024-04-13T05:15:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8104100 0x7fc7f819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 0x7fc7f819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f8105cb0 0x7fc7f81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f810fc40 con 0x7fc7f81052f0 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f810fac0 con 0x7fc7f8105cb0 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7f7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f810fdc0 con 0x7fc7f8104100 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7ed400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 0x7fc7f819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.509+0000 7fc7ed400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 0x7fc7f819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34722/0 (socket says 172.21.15.57:34722) 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 -- 172.21.15.57:0/2593552405 learned_addr learned my addr 172.21.15.57:0/2593552405 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7f5e00640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8104100 0x7fc7f819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7f6800640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f8105cb0 0x7fc7f81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 -- 172.21.15.57:0/2593552405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8104100 msgr2=0x7fc7f819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8104100 0x7fc7f819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 -- 172.21.15.57:0/2593552405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f8105cb0 msgr2=0x7fc7f81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f8105cb0 0x7fc7f81a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 -- 172.21.15.57:0/2593552405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81a1e60 con 0x7fc7f81052f0 2024-04-13T05:15:16.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7f6800640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f8105cb0 0x7fc7f81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:16.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7f5e00640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8104100 0x7fc7f819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ed400640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 0x7fc7f819a780 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000c920 tx=0x7fc7e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.510+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0007c70 con 0x7fc7f81052f0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.511+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7e0007e10 con 0x7fc7f81052f0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.511+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0018820 con 0x7fc7f81052f0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.511+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81a2150 con 0x7fc7f81052f0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.511+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81a2650 con 0x7fc7f81052f0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.512+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f810b420 con 0x7fc7f81052f0 2024-04-13T05:15:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.514+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc7e00189c0 con 0x7fc7f81052f0 2024-04-13T05:15:16.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.515+0000 7fc7ef400640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7dc0a3c30 0x7fc7dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:16.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.515+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc7e00d0960 con 0x7fc7f81052f0 2024-04-13T05:15:16.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.516+0000 7fc7f5e00640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7dc0a3c30 0x7fc7dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:16.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.516+0000 7fc7f5e00640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7dc0a3c30 0x7fc7dc0a60f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8037040 tx=0x7fc7e8005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:16.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.519+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7e0014030 con 0x7fc7f81052f0 2024-04-13T05:15:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:16 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1923344372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:15:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:16 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/976664430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:15:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:16 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1923344372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:15:16.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:16 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/976664430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:15:16.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.746+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7f802d5e0 con 0x7fc7f81052f0 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.747+0000 7fc7ef400640 1 -- 172.21.15.57:0/2593552405 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+588 (secure 0 0 0) 0x7fc7e009cf80 con 0x7fc7f81052f0 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2270359290 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1023620189 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/863867962 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1502699862 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1172825336 2024-04-14T05:04:34.865891+0000 2024-04-13T05:15:16.748 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:16.749 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/980869677 2024-04-14T05:05:11.407697+0000 2024-04-13T05:15:16.749 INFO:teuthology.orchestra.run.smithi057.stderr:listed 10 entries 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7dc0a3c30 msgr2=0x7fc7dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7dc0a3c30 0x7fc7dc0a60f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8037040 tx=0x7fc7e8005ef0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 msgr2=0x7fc7f819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 0x7fc7f819a780 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000c920 tx=0x7fc7e000cde0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 shutdown_connections 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7dc0a3c30 0x7fc7dc0a60f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7f8105cb0 0x7fc7f81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f81052f0 0x7fc7f819a780 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 --2- 172.21.15.57:0/2593552405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f8104100 0x7fc7f819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 >> 172.21.15.57:0/2593552405 conn(0x7fc7f80ff670 msgr2=0x7fc7f81085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 shutdown_connections 2024-04-13T05:15:16.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:16.750+0000 7fc7f7200640 1 -- 172.21.15.57:0/2593552405 wait complete. 2024-04-13T05:15:16.770 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2024-04-13T05:15:16.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:16.770 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-13T05:15:16.867 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[77589]: fuse finished with error 0 and tester_r 0 2024-04-13T05:15:16.872 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T05:15:16.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T05:15:16.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:15:16.951 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-13T05:15:17.017 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:17.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 -- 172.21.15.57:0/1685208312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 msgr2=0x7f948c1109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 --2- 172.21.15.57:0/1685208312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1109d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9484009cd0 tx=0x7f948402c0e0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 -- 172.21.15.57:0/1685208312 shutdown_connections 2024-04-13T05:15:17.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 --2- 172.21.15.57:0/1685208312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1109d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 --2- 172.21.15.57:0/1685208312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f948c074a20 0x7f948c074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 --2- 172.21.15.57:0/1685208312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948c076000 0x7f948c074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.246+0000 7f9491600640 1 -- 172.21.15.57:0/1685208312 >> 172.21.15.57:0/1685208312 conn(0x7f948c0fd5c0 msgr2=0x7f948c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:17.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.247+0000 7f9491600640 1 -- 172.21.15.57:0/1685208312 shutdown_connections 2024-04-13T05:15:17.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.247+0000 7f9491600640 1 -- 172.21.15.57:0/1685208312 wait complete. 2024-04-13T05:15:17.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 Processor -- start 2024-04-13T05:15:17.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 -- start start 2024-04-13T05:15:17.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f948c074a20 0x7f948c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948c076000 0x7f948c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f948c114110 con 0x7f948c10e5e0 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f948c113f90 con 0x7f948c076000 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f9491600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f948c114290 con 0x7f948c074a20 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f948b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f948b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34734/0 (socket says 172.21.15.57:34734) 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.248+0000 7f948b400640 1 -- 172.21.15.57:0/617814110 learned_addr learned my addr 172.21.15.57:0/617814110 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948aa00640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f948c074a20 0x7f948c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f9482000640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948c076000 0x7f948c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948b400640 1 -- 172.21.15.57:0/617814110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f948c074a20 msgr2=0x7f948c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948b400640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f948c074a20 0x7f948c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948b400640 1 -- 172.21.15.57:0/617814110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948c076000 msgr2=0x7f948c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948b400640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948c076000 0x7f948c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948b400640 1 -- 172.21.15.57:0/617814110 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c1a62e0 con 0x7f948c10e5e0 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.249+0000 7f948b400640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1a5c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9484032380 tx=0x7f9484035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.250+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484039040 con 0x7f948c10e5e0 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.250+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c1a6570 con 0x7f948c10e5e0 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.250+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9484003970 con 0x7f948c10e5e0 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.250+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840045e0 con 0x7f948c10e5e0 2024-04-13T05:15:17.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.250+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c1a6ab0 con 0x7f948c10e5e0 2024-04-13T05:15:17.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.251+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948c1047b0 con 0x7f948c10e5e0 2024-04-13T05:15:17.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.253+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9484046030 con 0x7f948c10e5e0 2024-04-13T05:15:17.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.255+0000 7f9483e00640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94700a3bb0 0x7f94700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.255+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f94840f2720 con 0x7f948c10e5e0 2024-04-13T05:15:17.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.255+0000 7f948aa00640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94700a3bb0 0x7f94700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.256+0000 7f948aa00640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94700a3bb0 0x7f94700a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f947c005e10 tx=0x7f947c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:17.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.260+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f948404d060 con 0x7f948c10e5e0 2024-04-13T05:15:17.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.469+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f948c062bc0 con 0x7f948c10e5e0 2024-04-13T05:15:17.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.473+0000 7f9483e00640 1 -- 172.21.15.57:0/617814110 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24583 (secure 0 0 0) 0x7f94840098b0 con 0x7f948c10e5e0 2024-04-13T05:15:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:17.475 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:15:13.150673+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.476+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94700a3bb0 msgr2=0x7f94700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.476+0000 7f9491600640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94700a3bb0 0x7f94700a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f947c005e10 tx=0x7f947c005da0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 msgr2=0x7f948c1a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1a5c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9484032380 tx=0x7f9484035e20 comp rx=0 tx=0).stop 2024-04-13T05:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 shutdown_connections 2024-04-13T05:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94700a3bb0 0x7f94700a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f948c076000 0x7f948c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f948c10e5e0 0x7f948c1a5c70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 --2- 172.21.15.57:0/617814110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f948c074a20 0x7f948c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 >> 172.21.15.57:0/617814110 conn(0x7f948c0fd5c0 msgr2=0x7f948c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:17.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 shutdown_connections 2024-04-13T05:15:17.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.477+0000 7f9491600640 1 -- 172.21.15.57:0/617814110 wait complete. 2024-04-13T05:15:17.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:17 smithi057 ceph-mon[32331]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:17.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:17 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3551719321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:15:17.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:17 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2593552405' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:15:17.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:17.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:17 smithi083 ceph-mon[35133]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:17.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:17 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3551719321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:15:17.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:17 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2593552405' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:15:17.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:17 smithi189 ceph-mon[35531]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:17.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:17 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3551719321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:15:17.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:17 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2593552405' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 -- 172.21.15.57:0/2706022459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4077bc0 msgr2=0x7f2ff4078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/2706022459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4077bc0 0x7f2ff4078020 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009900 tx=0x7f2fe002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 -- 172.21.15.57:0/2706022459 shutdown_connections 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/2706022459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ff4078560 0x7f2ff4078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/2706022459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4077bc0 0x7f2ff4078020 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/2706022459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4076980 0x7f2ff4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 -- 172.21.15.57:0/2706022459 >> 172.21.15.57:0/2706022459 conn(0x7f2ff40ff5d0 msgr2=0x7f2ff41019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 -- 172.21.15.57:0/2706022459 shutdown_connections 2024-04-13T05:15:17.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.729+0000 7f2ff8a00640 1 -- 172.21.15.57:0/2706022459 wait complete. 2024-04-13T05:15:17.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.730+0000 7f2ff8a00640 1 Processor -- start 2024-04-13T05:15:17.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.730+0000 7f2ff8a00640 1 -- start start 2024-04-13T05:15:17.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 0x7f2ff419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4077bc0 0x7f2ff419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ff4078560 0x7f2ff41a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff8a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4113f80 con 0x7f2ff4076980 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4113e00 con 0x7f2ff4077bc0 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4114100 con 0x7f2ff4078560 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 0x7f2ff419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 0x7f2ff419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34752/0 (socket says 172.21.15.57:34752) 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 -- 172.21.15.57:0/3673171643 learned_addr learned my addr 172.21.15.57:0/3673171643 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff1e00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4077bc0 0x7f2ff419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 -- 172.21.15.57:0/3673171643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ff4078560 msgr2=0x7f2ff41a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ff4078560 0x7f2ff41a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 -- 172.21.15.57:0/3673171643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4077bc0 msgr2=0x7f2ff419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4077bc0 0x7f2ff419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff2800640 1 -- 172.21.15.57:0/3673171643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff41a6210 con 0x7f2ff4076980 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.731+0000 7f2ff1e00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4077bc0 0x7f2ff419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.732+0000 7f2ff2800640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 0x7f2ff419e5f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d950 tx=0x7f2fe400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.732+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4017070 con 0x7f2ff4076980 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.732+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe4009c30 con 0x7f2ff4076980 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.732+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff41a6500 con 0x7f2ff4076980 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.732+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400f620 con 0x7f2ff4076980 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.732+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff41a6aa0 con 0x7f2ff4076980 2024-04-13T05:15:17.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.733+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff4110b40 con 0x7f2ff4076980 2024-04-13T05:15:17.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.737+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2fe40109a0 con 0x7f2ff4076980 2024-04-13T05:15:17.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.738+0000 7f2feb400640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2fd80a3c30 0x7f2fd80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:17.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.738+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2fe40d03e0 con 0x7f2ff4076980 2024-04-13T05:15:17.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.739+0000 7f2ff1e00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2fd80a3c30 0x7f2fd80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:17.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.739+0000 7f2ff1e00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2fd80a3c30 0x7f2fd80a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0036040 tx=0x7f2fe0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:17.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.741+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fe4009de0 con 0x7f2ff4076980 2024-04-13T05:15:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.949+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ff4062c60 con 0x7f2ff4076980 2024-04-13T05:15:17.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.952+0000 7f2feb400640 1 -- 172.21.15.57:0/3673171643 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24583 (secure 0 0 0) 0x7f2fe409ca80 con 0x7f2ff4076980 2024-04-13T05:15:17.953 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:17.953 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:15:13.150673+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:17.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2fd80a3c30 msgr2=0x7f2fd80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2fd80a3c30 0x7f2fd80a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0036040 tx=0x7f2fe0005fd0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 msgr2=0x7f2ff419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:17.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 0x7f2ff419e5f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d950 tx=0x7f2fe400de10 comp rx=0 tx=0).stop 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 shutdown_connections 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2fd80a3c30 0x7f2fd80a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff4077bc0 0x7f2ff419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ff4078560 0x7f2ff41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 --2- 172.21.15.57:0/3673171643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff4076980 0x7f2ff419e5f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 >> 172.21.15.57:0/3673171643 conn(0x7f2ff40ff5d0 msgr2=0x7f2ff4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 shutdown_connections 2024-04-13T05:15:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:17.955+0000 7f2ff8a00640 1 -- 172.21.15.57:0/3673171643 wait complete. 2024-04-13T05:15:17.978 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 -- 172.21.15.57:0/299343835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103a40 msgr2=0x7f6a5810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 --2- 172.21.15.57:0/299343835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103a40 0x7f6a5810a360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009900 tx=0x7f6a5002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 -- 172.21.15.57:0/299343835 shutdown_connections 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 --2- 172.21.15.57:0/299343835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103a40 0x7f6a5810a360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 --2- 172.21.15.57:0/299343835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58103080 0x7f6a58103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 --2- 172.21.15.57:0/299343835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 0x7f6a581022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:18.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 -- 172.21.15.57:0/299343835 >> 172.21.15.57:0/299343835 conn(0x7f6a580fd460 msgr2=0x7f6a580ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:18.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 -- 172.21.15.57:0/299343835 shutdown_connections 2024-04-13T05:15:18.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.198+0000 7f6a57200640 1 -- 172.21.15.57:0/299343835 wait complete. 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.199+0000 7f6a57200640 1 Processor -- start 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.199+0000 7f6a57200640 1 -- start start 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a57200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 0x7f6a58075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a57200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 0x7f6a580743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a57200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58103a40 0x7f6a580748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a57200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810da50 con 0x7f6a58103080 2024-04-13T05:15:18.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a57200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810d8d0 con 0x7f6a58101ea0 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a57200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810dbd0 con 0x7f6a58103a40 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a55400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 0x7f6a580743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a55400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 0x7f6a580743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34768/0 (socket says 172.21.15.57:34768) 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a55400640 1 -- 172.21.15.57:0/2296524773 learned_addr learned my addr 172.21.15.57:0/2296524773 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.200+0000 7f6a56800640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58103a40 0x7f6a580748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55e00640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 0x7f6a58075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55400640 1 -- 172.21.15.57:0/2296524773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58103a40 msgr2=0x7f6a580748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55400640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58103a40 0x7f6a580748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55400640 1 -- 172.21.15.57:0/2296524773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 msgr2=0x7f6a58075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55400640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 0x7f6a58075d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55400640 1 -- 172.21.15.57:0/2296524773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a58075180 con 0x7f6a58103080 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55e00640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 0x7f6a58075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a55400640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 0x7f6a580743b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000eb90 tx=0x7f6a4000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:18.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40004d20 con 0x7f6a58103080 2024-04-13T05:15:18.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a4000ee90 con 0x7f6a58103080 2024-04-13T05:15:18.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581a2220 con 0x7f6a58103080 2024-04-13T05:15:18.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40010670 con 0x7f6a58103080 2024-04-13T05:15:18.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.201+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581a2720 con 0x7f6a58103080 2024-04-13T05:15:18.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.203+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a581091e0 con 0x7f6a58103080 2024-04-13T05:15:18.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.205+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a40005000 con 0x7f6a58103080 2024-04-13T05:15:18.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.206+0000 7f6a4ea00640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6a380a3bb0 0x7f6a380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:18.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.206+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6a400d10f0 con 0x7f6a58103080 2024-04-13T05:15:18.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.206+0000 7f6a55e00640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6a380a3bb0 0x7f6a380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:18.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.207+0000 7f6a55e00640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6a380a3bb0 0x7f6a380a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a58102bf0 tx=0x7f6a48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:18.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.209+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a40053ff0 con 0x7f6a58103080 2024-04-13T05:15:18.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:18.463+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6a5802d850 con 0x7f6a58103080 2024-04-13T05:15:18.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:18 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/617814110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:18.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:18 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3673171643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:18 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/617814110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:18 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3673171643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:18.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:18 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/617814110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:18.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:18 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3673171643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:19.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.235+0000 7f6a4ea00640 1 -- 172.21.15.57:0/2296524773 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f6a4009d770 con 0x7f6a58103080 2024-04-13T05:15:19.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6a380a3bb0 msgr2=0x7f6a380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:19.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6a380a3bb0 0x7f6a380a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a58102bf0 tx=0x7f6a48005f90 comp rx=0 tx=0).stop 2024-04-13T05:15:19.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 msgr2=0x7f6a580743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:19.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 0x7f6a580743b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000eb90 tx=0x7f6a4000c6a0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 shutdown_connections 2024-04-13T05:15:19.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6a380a3bb0 0x7f6a380a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58103a40 0x7f6a580748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a58103080 0x7f6a580743b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 --2- 172.21.15.57:0/2296524773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101ea0 0x7f6a58075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.239+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 >> 172.21.15.57:0/2296524773 conn(0x7f6a580fd460 msgr2=0x7f6a581063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.240+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 shutdown_connections 2024-04-13T05:15:19.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.240+0000 7f6a57200640 1 -- 172.21.15.57:0/2296524773 wait complete. 2024-04-13T05:15:19.263 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-13T05:15:19.263 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-13T05:15:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 -- 172.21.15.57:0/2279838592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 msgr2=0x7f594410c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 --2- 172.21.15.57:0/2279838592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f594410c540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f593c009ca0 tx=0x7f593c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 -- 172.21.15.57:0/2279838592 shutdown_connections 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 --2- 172.21.15.57:0/2279838592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59441052f0 0x7f5944105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 --2- 172.21.15.57:0/2279838592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f594410c540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 --2- 172.21.15.57:0/2279838592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 0x7f5944104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.495+0000 7f5949000640 1 -- 172.21.15.57:0/2279838592 >> 172.21.15.57:0/2279838592 conn(0x7f59440ff670 msgr2=0x7f5944101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.496+0000 7f5949000640 1 -- 172.21.15.57:0/2279838592 shutdown_connections 2024-04-13T05:15:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.496+0000 7f5949000640 1 -- 172.21.15.57:0/2279838592 wait complete. 2024-04-13T05:15:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 Processor -- start 2024-04-13T05:15:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 -- start start 2024-04-13T05:15:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 0x7f594419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59441052f0 0x7f594419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f59441a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f594410fc40 con 0x7f5944105cb0 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f594410fac0 con 0x7f59441052f0 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5949000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f594410fdc0 con 0x7f5944104100 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.497+0000 7f5943e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f59441a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f59441a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34784/0 (socket says 172.21.15.57:34784) 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 -- 172.21.15.57:0/2898287439 learned_addr learned my addr 172.21.15.57:0/2898287439 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5942a00640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59441052f0 0x7f594419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943400640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 0x7f594419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 -- 172.21.15.57:0/2898287439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 msgr2=0x7f594419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 0x7f594419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 -- 172.21.15.57:0/2898287439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59441052f0 msgr2=0x7f594419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59441052f0 0x7f594419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 -- 172.21.15.57:0/2898287439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59441a1f10 con 0x7f5944105cb0 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943400640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 0x7f594419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f5943e00640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f59441a18a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f593c032380 tx=0x7f593c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.498+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c041020 con 0x7f5944105cb0 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.499+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f593c003e20 con 0x7f5944105cb0 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.499+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59441a21a0 con 0x7f5944105cb0 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.499+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c039040 con 0x7f5944105cb0 2024-04-13T05:15:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.499+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59441a2690 con 0x7f5944105cb0 2024-04-13T05:15:19.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:19 smithi083 ceph-mon[35133]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:19.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:19 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2296524773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-13T05:15:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.500+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594410b470 con 0x7f5944105cb0 2024-04-13T05:15:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.502+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f593c044020 con 0x7f5944105cb0 2024-04-13T05:15:19.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.504+0000 7f593be00640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f59280a3bb0 0x7f59280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:19.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.504+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f593c0f2bc0 con 0x7f5944105cb0 2024-04-13T05:15:19.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.504+0000 7f5943400640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f59280a3bb0 0x7f59280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:19.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.505+0000 7f5943400640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f59280a3bb0 0x7f59280a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5944105150 tx=0x7f5934005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:19.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.508+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f593c04b020 con 0x7f5944105cb0 2024-04-13T05:15:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:19 smithi057 ceph-mon[32331]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:19 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2296524773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-13T05:15:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:19 smithi189 ceph-mon[35531]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:19 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2296524773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-13T05:15:19.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:19.758+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f5944062bc0 con 0x7f5944105cb0 2024-04-13T05:15:20.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.257+0000 7f593be00640 1 -- 172.21.15.57:0/2898287439 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f593c0bf1e0 con 0x7f5944105cb0 2024-04-13T05:15:20.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.260+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f59280a3bb0 msgr2=0x7f59280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:20.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.260+0000 7f5949000640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f59280a3bb0 0x7f59280a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5944105150 tx=0x7f5934005e70 comp rx=0 tx=0).stop 2024-04-13T05:15:20.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.260+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 msgr2=0x7f59441a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:20.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.260+0000 7f5949000640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f59441a18a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f593c032380 tx=0x7f593c032820 comp rx=0 tx=0).stop 2024-04-13T05:15:20.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 shutdown_connections 2024-04-13T05:15:20.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f59280a3bb0 0x7f59280a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59441052f0 0x7f594419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5944105cb0 0x7f59441a18a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 --2- 172.21.15.57:0/2898287439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5944104100 0x7f594419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 >> 172.21.15.57:0/2898287439 conn(0x7f59440ff670 msgr2=0x7f5944108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:20.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 shutdown_connections 2024-04-13T05:15:20.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.261+0000 7f5949000640 1 -- 172.21.15.57:0/2898287439 wait complete. 2024-04-13T05:15:20.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-13T05:15:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 -- 172.21.15.57:0/4208505113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe010a0b0 msgr2=0x7febe0110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 --2- 172.21.15.57:0/4208505113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe010a0b0 0x7febe0110940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febd8009ca0 tx=0x7febd802c100 comp rx=0 tx=0).stop 2024-04-13T05:15:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 -- 172.21.15.57:0/4208505113 shutdown_connections 2024-04-13T05:15:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 --2- 172.21.15.57:0/4208505113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe010a0b0 0x7febe0110940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 --2- 172.21.15.57:0/4208505113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febe01096f0 0x7febe0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 --2- 172.21.15.57:0/4208505113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febe0108500 0x7febe0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.504+0000 7febdf200640 1 -- 172.21.15.57:0/4208505113 >> 172.21.15.57:0/4208505113 conn(0x7febe0077490 msgr2=0x7febe00798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.505+0000 7febdf200640 1 -- 172.21.15.57:0/4208505113 shutdown_connections 2024-04-13T05:15:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.505+0000 7febdf200640 1 -- 172.21.15.57:0/4208505113 wait complete. 2024-04-13T05:15:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 Processor -- start 2024-04-13T05:15:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 -- start start 2024-04-13T05:15:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febe0108500 0x7febe019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 0x7febe019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febe010a0b0 0x7febe01a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7febe0114040 con 0x7febe01096f0 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7febe0113ec0 con 0x7febe0108500 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.506+0000 7febdf200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febe01141c0 con 0x7febe010a0b0 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 0x7febe019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 0x7febe019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34798/0 (socket says 172.21.15.57:34798) 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 -- 172.21.15.57:0/1194325560 learned_addr learned my addr 172.21.15.57:0/1194325560 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdde00640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febe0108500 0x7febe019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febde800640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febe010a0b0 0x7febe01a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 -- 172.21.15.57:0/1194325560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febe010a0b0 msgr2=0x7febe01a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febe010a0b0 0x7febe01a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 -- 172.21.15.57:0/1194325560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febe0108500 msgr2=0x7febe019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febe0108500 0x7febe019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 -- 172.21.15.57:0/1194325560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01a62a0 con 0x7febe01096f0 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.507+0000 7febdd400640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 0x7febe019ec10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febc800c890 tx=0x7febc800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.508+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8004220 con 0x7febe01096f0 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.508+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01a6590 con 0x7febe01096f0 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.508+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc80043c0 con 0x7febe01096f0 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.508+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8004a00 con 0x7febe01096f0 2024-04-13T05:15:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.508+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01a6ae0 con 0x7febe01096f0 2024-04-13T05:15:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.509+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe010f820 con 0x7febe01096f0 2024-04-13T05:15:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.511+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7febc8004ba0 con 0x7febe01096f0 2024-04-13T05:15:20.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.512+0000 7febd6a00640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7febc40a3bb0 0x7febc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:20.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.513+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7febc80d0500 con 0x7febe01096f0 2024-04-13T05:15:20.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.513+0000 7febdde00640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7febc40a3bb0 0x7febc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:20.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.514+0000 7febdde00640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7febc40a3bb0 0x7febc40a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7febd0005fd0 tx=0x7febd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:20.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.517+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febc8014030 con 0x7febe01096f0 2024-04-13T05:15:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:20 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2296524773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-13T05:15:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:20 smithi057 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:20 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2898287439' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-13T05:15:20.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:20 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2296524773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-13T05:15:20.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:20 smithi189 ceph-mon[35531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:20.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:20 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2898287439' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-13T05:15:20.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:20 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2296524773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-13T05:15:20.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:20 smithi083 ceph-mon[35133]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:20.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:20 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2898287439' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-13T05:15:20.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:20.768+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7febe0062bc0 con 0x7febe01096f0 2024-04-13T05:15:21.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.278+0000 7febd6a00640 1 -- 172.21.15.57:0/1194325560 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7febc809cb20 con 0x7febe01096f0 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7febc40a3bb0 msgr2=0x7febc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7febc40a3bb0 0x7febc40a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7febd0005fd0 tx=0x7febd0005850 comp rx=0 tx=0).stop 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 msgr2=0x7febe019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 0x7febe019ec10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febc800c890 tx=0x7febc800cd50 comp rx=0 tx=0).stop 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 shutdown_connections 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7febc40a3bb0 0x7febc40a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febe010a0b0 0x7febe01a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febe01096f0 0x7febe019ec10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 --2- 172.21.15.57:0/1194325560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febe0108500 0x7febe019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 >> 172.21.15.57:0/1194325560 conn(0x7febe0077490 msgr2=0x7febe010c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:21.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.282+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 shutdown_connections 2024-04-13T05:15:21.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.283+0000 7febdf200640 1 -- 172.21.15.57:0/1194325560 wait complete. 2024-04-13T05:15:21.311 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-13T05:15:21.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 -- 172.21.15.57:0/1304245998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47410a1d0 msgr2=0x7fb474110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 --2- 172.21.15.57:0/1304245998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47410a1d0 0x7fb474110a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009ca0 tx=0x7fb46c02c100 comp rx=0 tx=0).stop 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 -- 172.21.15.57:0/1304245998 shutdown_connections 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 --2- 172.21.15.57:0/1304245998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb474109810 0x7fb474109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 --2- 172.21.15.57:0/1304245998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47410a1d0 0x7fb474110a60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 --2- 172.21.15.57:0/1304245998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb474108620 0x7fb474108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 -- 172.21.15.57:0/1304245998 >> 172.21.15.57:0/1304245998 conn(0x7fb4740775c0 msgr2=0x7fb474079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 -- 172.21.15.57:0/1304245998 shutdown_connections 2024-04-13T05:15:21.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.514+0000 7fb479400640 1 -- 172.21.15.57:0/1304245998 wait complete. 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 Processor -- start 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 -- start start 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb474108620 0x7fb47419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 0x7fb47419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47410a1d0 0x7fb4741a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb474114150 con 0x7fb474109810 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb474113fd0 con 0x7fb47410a1d0 2024-04-13T05:15:21.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.515+0000 7fb479400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741142d0 con 0x7fb474108620 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 0x7fb47419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 0x7fb47419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34818/0 (socket says 172.21.15.57:34818) 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 -- 172.21.15.57:0/1540826987 learned_addr learned my addr 172.21.15.57:0/1540826987 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb472800640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb474108620 0x7fb47419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb473200640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47410a1d0 0x7fb4741a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 -- 172.21.15.57:0/1540826987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb474108620 msgr2=0x7fb47419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb474108620 0x7fb47419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 -- 172.21.15.57:0/1540826987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47410a1d0 msgr2=0x7fb4741a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47410a1d0 0x7fb4741a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 -- 172.21.15.57:0/1540826987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4741a6440 con 0x7fb474109810 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.516+0000 7fb46be00640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 0x7fb47419ed60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d950 tx=0x7fb45c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.517+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c017070 con 0x7fb474109810 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.517+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45c009c30 con 0x7fb474109810 2024-04-13T05:15:21.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.517+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4741a6730 con 0x7fb474109810 2024-04-13T05:15:21.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.517+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00f620 con 0x7fb474109810 2024-04-13T05:15:21.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.517+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4741a6c80 con 0x7fb474109810 2024-04-13T05:15:21.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.518+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47410f990 con 0x7fb474109810 2024-04-13T05:15:21.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.520+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb45c0109a0 con 0x7fb474109810 2024-04-13T05:15:21.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.521+0000 7fb46b400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4580a3bb0 0x7fb4580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:21.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.521+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb45c0d04a0 con 0x7fb474109810 2024-04-13T05:15:21.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.522+0000 7fb472800640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4580a3bb0 0x7fb4580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:21.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.523+0000 7fb472800640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4580a3bb0 0x7fb4580a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb464005fd0 tx=0x7fb464005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:21.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.527+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb45c009de0 con 0x7fb474109810 2024-04-13T05:15:21.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2898287439' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-13T05:15:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:21 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1194325560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-13T05:15:21.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2898287439' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-13T05:15:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:21 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1194325560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2898287439' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-13T05:15:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:21 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1194325560' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-13T05:15:21.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:21.777+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb47402d5e0 con 0x7fb474109810 2024-04-13T05:15:22.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.304+0000 7fb46b400640 1 -- 172.21.15.57:0/1540826987 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fb45c09cac0 con 0x7fb474109810 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4580a3bb0 msgr2=0x7fb4580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4580a3bb0 0x7fb4580a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb464005fd0 tx=0x7fb464005850 comp rx=0 tx=0).stop 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 msgr2=0x7fb47419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 0x7fb47419ed60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d950 tx=0x7fb45c00de10 comp rx=0 tx=0).stop 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 shutdown_connections 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4580a3bb0 0x7fb4580a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47410a1d0 0x7fb4741a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb474109810 0x7fb47419ed60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 --2- 172.21.15.57:0/1540826987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb474108620 0x7fb47419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.308+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 >> 172.21.15.57:0/1540826987 conn(0x7fb4740775c0 msgr2=0x7fb47410cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:22.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.309+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 shutdown_connections 2024-04-13T05:15:22.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.309+0000 7fb479400640 1 -- 172.21.15.57:0/1540826987 wait complete. 2024-04-13T05:15:22.334 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-13T05:15:22.335 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-13T05:15:22.501 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: 2024-04-13T05:15:22.123+0000 7f9d7b800700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-13T05:15:22.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1194325560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-13T05:15:22.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] up:active 2024-04-13T05:15:22.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-13T05:15:22.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-13T05:15:22.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: Cluster is now healthy 2024-04-13T05:15:22.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-13T05:15:22.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-13T05:15:22.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-13T05:15:22.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:22 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1540826987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 -- 172.21.15.57:0/3680121113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac104240 msgr2=0x7f60ac104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 --2- 172.21.15.57:0/3680121113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac104240 0x7f60ac104660 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f609c009ca0 tx=0x7f609c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 -- 172.21.15.57:0/3680121113 shutdown_connections 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 --2- 172.21.15.57:0/3680121113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac105df0 0x7f60ac10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 --2- 172.21.15.57:0/3680121113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60ac105430 0x7f60ac1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 --2- 172.21.15.57:0/3680121113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac104240 0x7f60ac104660 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 -- 172.21.15.57:0/3680121113 >> 172.21.15.57:0/3680121113 conn(0x7f60ac0ff7b0 msgr2=0x7f60ac101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:22.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 -- 172.21.15.57:0/3680121113 shutdown_connections 2024-04-13T05:15:22.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.567+0000 7f60b3800640 1 -- 172.21.15.57:0/3680121113 wait complete. 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.568+0000 7f60b3800640 1 Processor -- start 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.568+0000 7f60b3800640 1 -- start start 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b3800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60ac104240 0x7f60ac19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b3800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac105430 0x7f60ac19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b3800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 0x7f60ac1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b3800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10fd80 con 0x7f60ac105df0 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b3800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10fc00 con 0x7f60ac105430 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b3800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10ff00 con 0x7f60ac104240 2024-04-13T05:15:22.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 0x7f60ac1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:22.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 0x7f60ac1a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34834/0 (socket says 172.21.15.57:34834) 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b1800640 1 -- 172.21.15.57:0/2085269871 learned_addr learned my addr 172.21.15.57:0/2085269871 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60abe00640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac105430 0x7f60ac19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.569+0000 7f60b0e00640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60ac104240 0x7f60ac19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b1800640 1 -- 172.21.15.57:0/2085269871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60ac104240 msgr2=0x7f60ac19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b1800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60ac104240 0x7f60ac19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b1800640 1 -- 172.21.15.57:0/2085269871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac105430 msgr2=0x7f60ac19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b1800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac105430 0x7f60ac19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b1800640 1 -- 172.21.15.57:0/2085269871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac1a2020 con 0x7f60ac105df0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b1800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 0x7f60ac1a19b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f60a400d750 tx=0x7f60a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4017070 con 0x7f60ac105df0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60a40054d0 con 0x7f60ac105df0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1a2310 con 0x7f60ac105df0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4004c60 con 0x7f60ac105df0 2024-04-13T05:15:22.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.570+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a2810 con 0x7f60ac105df0 2024-04-13T05:15:22.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.572+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac10b630 con 0x7f60ac105df0 2024-04-13T05:15:22.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.574+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60a4004e00 con 0x7f60ac105df0 2024-04-13T05:15:22.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.575+0000 7f60a9600640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60900a3bb0 0x7f60900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:22.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.575+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f60a40d0200 con 0x7f60ac105df0 2024-04-13T05:15:22.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.576+0000 7f60b0e00640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60900a3bb0 0x7f60900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:22.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.577+0000 7f60b0e00640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60900a3bb0 0x7f60900a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f609c02c630 tx=0x7f609c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:22.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.580+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60a409c880 con 0x7f60ac105df0 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1194325560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] up:active 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: Cluster is now healthy 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-13T05:15:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:22 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1540826987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-13T05:15:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1194325560' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-13T05:15:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] up:active 2024-04-13T05:15:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-13T05:15:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-13T05:15:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: Cluster is now healthy 2024-04-13T05:15:22.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-13T05:15:22.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-13T05:15:22.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-13T05:15:22.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1540826987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-13T05:15:22.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.816+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f60ac062400 con 0x7f60900a3bb0 2024-04-13T05:15:22.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.882+0000 7f60a9600640 1 -- 172.21.15.57:0/2085269871 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60ac062400 con 0x7f60900a3bb0 2024-04-13T05:15:22.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60900a3bb0 msgr2=0x7f60900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60900a3bb0 0x7f60900a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f609c02c630 tx=0x7f609c000f40 comp rx=0 tx=0).stop 2024-04-13T05:15:22.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 msgr2=0x7f60ac1a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:22.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 0x7f60ac1a19b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f60a400d750 tx=0x7f60a400dc10 comp rx=0 tx=0).stop 2024-04-13T05:15:22.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 shutdown_connections 2024-04-13T05:15:22.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f60900a3bb0 0x7f60900a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac105430 0x7f60ac19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60ac105df0 0x7f60ac1a19b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 --2- 172.21.15.57:0/2085269871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60ac104240 0x7f60ac19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:22.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.885+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 >> 172.21.15.57:0/2085269871 conn(0x7f60ac0ff7b0 msgr2=0x7f60ac108800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:22.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.886+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 shutdown_connections 2024-04-13T05:15:22.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:22.886+0000 7f60b3800640 1 -- 172.21.15.57:0/2085269871 wait complete. 2024-04-13T05:15:22.908 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-13T05:15:23.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.124+0000 7f698a800640 1 -- 172.21.15.57:0/1130846288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103a20 msgr2=0x7f698410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.124+0000 7f698a800640 1 --2- 172.21.15.57:0/1130846288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103a20 0x7f698410a340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 -- 172.21.15.57:0/1130846288 shutdown_connections 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 --2- 172.21.15.57:0/1130846288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6984103060 0x7f69841034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 --2- 172.21.15.57:0/1130846288 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103a20 0x7f698410a340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 --2- 172.21.15.57:0/1130846288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6984101e80 0x7f69841022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 -- 172.21.15.57:0/1130846288 >> 172.21.15.57:0/1130846288 conn(0x7f69840fd460 msgr2=0x7f69840ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 -- 172.21.15.57:0/1130846288 shutdown_connections 2024-04-13T05:15:23.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.125+0000 7f698a800640 1 -- 172.21.15.57:0/1130846288 wait complete. 2024-04-13T05:15:23.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 Processor -- start 2024-04-13T05:15:23.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 -- start start 2024-04-13T05:15:23.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6984101e80 0x7f6984198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 0x7f69841985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6984103a20 0x7f698419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f698410da00 con 0x7f6984103060 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698410d880 con 0x7f6984103a20 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f698a800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f698410db80 con 0x7f6984101e80 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.126+0000 7f6988a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 0x7f69841985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 0x7f69841985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34878/0 (socket says 172.21.15.57:34878) 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 -- 172.21.15.57:0/566217145 learned_addr learned my addr 172.21.15.57:0/566217145 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6989e00640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6984103a20 0x7f698419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6989400640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6984101e80 0x7f6984198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 -- 172.21.15.57:0/566217145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6984101e80 msgr2=0x7f6984198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6984101e80 0x7f6984198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 -- 172.21.15.57:0/566217145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6984103a20 msgr2=0x7f698419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6984103a20 0x7f698419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 -- 172.21.15.57:0/566217145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698419fc90 con 0x7f6984103060 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6988a00640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 0x7f69841985b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9f0 tx=0x7f697000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.127+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697000cd80 con 0x7f6984103060 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.128+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69700108e0 con 0x7f6984103060 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.128+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698419ff80 con 0x7f6984103060 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.128+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970018920 con 0x7f6984103060 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.128+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a0490 con 0x7f6984103060 2024-04-13T05:15:23.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.129+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6984109220 con 0x7f6984103060 2024-04-13T05:15:23.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.132+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6970010a90 con 0x7f6984103060 2024-04-13T05:15:23.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.133+0000 7f6982000640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f696c0a3c30 0x7f696c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.133+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69700d0b40 con 0x7f6984103060 2024-04-13T05:15:23.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.133+0000 7f6989400640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f696c0a3c30 0x7f696c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.134+0000 7f6989400640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f696c0a3c30 0x7f696c0a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6984102bd0 tx=0x7f6978005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:23.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.138+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f697009d160 con 0x7f6984103060 2024-04-13T05:15:23.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:15:22.829+0000 7fb163a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-13T05:15:23.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:15:22.829+0000 7fb163a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-13T05:15:23.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:15:22.829+0000 7fb163a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-13T05:15:23.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:15:22.829+0000 7fb163a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-13T05:15:23.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:15:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:15:22.829+0000 7fb163a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-13T05:15:23.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.378+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6984062490 con 0x7f696c0a3c30 2024-04-13T05:15:23.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.441+0000 7f6982000640 1 -- 172.21.15.57:0/566217145 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6984062490 con 0x7f696c0a3c30 2024-04-13T05:15:23.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f696c0a3c30 msgr2=0x7f696c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f696c0a3c30 0x7f696c0a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6984102bd0 tx=0x7f6978005f90 comp rx=0 tx=0).stop 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 msgr2=0x7f69841985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 0x7f69841985b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9f0 tx=0x7f697000eeb0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 shutdown_connections 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f696c0a3c30 0x7f696c0a60f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6984103a20 0x7f698419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6984103060 0x7f69841985b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 --2- 172.21.15.57:0/566217145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6984101e80 0x7f6984198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.444+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 >> 172.21.15.57:0/566217145 conn(0x7f69840fd460 msgr2=0x7f69841063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.445+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 shutdown_connections 2024-04-13T05:15:23.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.445+0000 7f698a800640 1 -- 172.21.15.57:0/566217145 wait complete. 2024-04-13T05:15:23.466 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-13T05:15:23.543 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:23 smithi057 ceph-mon[32331]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 325 B/s wr, 0 op/s; 61 B/s, 0 objects/s recovering 2024-04-13T05:15:23.544 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:23 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1540826987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-13T05:15:23.544 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:23 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] up:active 2024-04-13T05:15:23.544 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:23 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-13T05:15:23.544 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:23 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:15:23.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:23 smithi189 ceph-mon[35531]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 325 B/s wr, 0 op/s; 61 B/s, 0 objects/s recovering 2024-04-13T05:15:23.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:23 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1540826987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-13T05:15:23.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:23 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] up:active 2024-04-13T05:15:23.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:23 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-13T05:15:23.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:23 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:15:23.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 -- 172.21.15.57:0/3096656273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514102010 msgr2=0x7f1514102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 --2- 172.21.15.57:0/3096656273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514102010 0x7f1514102430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1504009900 tx=0x7f150402bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:23.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 -- 172.21.15.57:0/3096656273 shutdown_connections 2024-04-13T05:15:23.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 --2- 172.21.15.57:0/3096656273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514103200 0x7f1514103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 --2- 172.21.15.57:0/3096656273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 0x7f151410a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 --2- 172.21.15.57:0/3096656273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514102010 0x7f1514102430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 -- 172.21.15.57:0/3096656273 >> 172.21.15.57:0/3096656273 conn(0x7f15140fd5a0 msgr2=0x7f15140ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:23.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 -- 172.21.15.57:0/3096656273 shutdown_connections 2024-04-13T05:15:23.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.688+0000 7f151be00640 1 -- 172.21.15.57:0/3096656273 wait complete. 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.689+0000 7f151be00640 1 Processor -- start 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.689+0000 7f151be00640 1 -- start start 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.689+0000 7f151be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514102010 0x7f1514075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.689+0000 7f151be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 0x7f15140743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f151be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 0x7f15140748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f151be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f151410db50 con 0x7f1514103200 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f151be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f151410d9d0 con 0x7f1514102010 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f151be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f151410dcd0 con 0x7f1514103bc0 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 0x7f15140743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 0x7f15140743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34898/0 (socket says 172.21.15.57:34898) 2024-04-13T05:15:23.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 -- 172.21.15.57:0/1188322832 learned_addr learned my addr 172.21.15.57:0/1188322832 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1519e00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 0x7f15140748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1519400640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514102010 0x7f1514075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 -- 172.21.15.57:0/1188322832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 msgr2=0x7f15140748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 0x7f15140748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 -- 172.21.15.57:0/1188322832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514102010 msgr2=0x7f1514075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514102010 0x7f1514075d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1518a00640 1 -- 172.21.15.57:0/1188322832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1514075180 con 0x7f1514103200 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1519e00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 0x7f15140748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.690+0000 7f1519400640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514102010 0x7f1514075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.691+0000 7f1518a00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 0x7f15140743b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f150000c920 tx=0x7f150000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.691+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500007c70 con 0x7f1514103200 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.691+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1500007e10 con 0x7f1514103200 2024-04-13T05:15:23.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.691+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500018820 con 0x7f1514103200 2024-04-13T05:15:23.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.691+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15141a2350 con 0x7f1514103200 2024-04-13T05:15:23.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.691+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141a28f0 con 0x7f1514103200 2024-04-13T05:15:23.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.693+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15141093f0 con 0x7f1514103200 2024-04-13T05:15:23.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.695+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f15000189c0 con 0x7f1514103200 2024-04-13T05:15:23.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.697+0000 7f1512000640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14f80a3c30 0x7f14f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:23.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.697+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f15000d0be0 con 0x7f1514103200 2024-04-13T05:15:23.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.697+0000 7f1519400640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14f80a3c30 0x7f14f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:23.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.698+0000 7f1519400640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14f80a3c30 0x7f14f80a60f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15040096c0 tx=0x7f1504000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:23.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.699+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1500014030 con 0x7f1514103200 2024-04-13T05:15:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:23 smithi083 ceph-mon[35133]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 325 B/s wr, 0 op/s; 61 B/s, 0 objects/s recovering 2024-04-13T05:15:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:23 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1540826987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-13T05:15:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:23 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] up:active 2024-04-13T05:15:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:23 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-13T05:15:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:23 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:15:23.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.939+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1514062490 con 0x7f14f80a3c30 2024-04-13T05:15:23.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.947+0000 7f1512000640 1 -- 172.21.15.57:0/1188322832 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1514062490 con 0x7f14f80a3c30 2024-04-13T05:15:23.949 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07 2024-04-13T05:15:23.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.949+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14f80a3c30 msgr2=0x7f14f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.949+0000 7f151be00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14f80a3c30 0x7f14f80a60f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15040096c0 tx=0x7f1504000bc0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.949+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 msgr2=0x7f15140743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.949+0000 7f151be00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 0x7f15140743b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f150000c920 tx=0x7f150000cde0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 shutdown_connections 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f14f80a3c30 0x7f14f80a60f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1514103bc0 0x7f15140748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1514103200 0x7f15140743b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 --2- 172.21.15.57:0/1188322832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1514102010 0x7f1514075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 >> 172.21.15.57:0/1188322832 conn(0x7f15140fd5a0 msgr2=0x7f15141065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 shutdown_connections 2024-04-13T05:15:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:23.950+0000 7f151be00640 1 -- 172.21.15.57:0/1188322832 wait complete. 2024-04-13T05:15:23.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-13T05:15:24.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 -- 172.21.15.57:0/3373770937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 msgr2=0x7fdffc10a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 --2- 172.21.15.57:0/3373770937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc10a3b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009ca0 tx=0x7fdff402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 -- 172.21.15.57:0/3373770937 shutdown_connections 2024-04-13T05:15:24.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 --2- 172.21.15.57:0/3373770937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc1030d0 0x7fdffc103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 --2- 172.21.15.57:0/3373770937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc10a3b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 --2- 172.21.15.57:0/3373770937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdffc101f00 0x7fdffc102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 -- 172.21.15.57:0/3373770937 >> 172.21.15.57:0/3373770937 conn(0x7fdffc0fd4c0 msgr2=0x7fdffc0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:24.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 -- 172.21.15.57:0/3373770937 shutdown_connections 2024-04-13T05:15:24.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.188+0000 7fe001e00640 1 -- 172.21.15.57:0/3373770937 wait complete. 2024-04-13T05:15:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.189+0000 7fe001e00640 1 Processor -- start 2024-04-13T05:15:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.189+0000 7fe001e00640 1 -- start start 2024-04-13T05:15:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc101f00 0x7fdffc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdffc1030d0 0x7fdffc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10da00 con 0x7fdffc103a90 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10d880 con 0x7fdffc101f00 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10db80 con 0x7fdffc1030d0 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34912/0 (socket says 172.21.15.57:34912) 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe001400640 1 -- 172.21.15.57:0/613427937 learned_addr learned my addr 172.21.15.57:0/613427937 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.190+0000 7fe000a00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc101f00 0x7fdffc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fdffbe00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdffc1030d0 0x7fdffc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fe001400640 1 -- 172.21.15.57:0/613427937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdffc1030d0 msgr2=0x7fdffc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fe001400640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdffc1030d0 0x7fdffc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fe001400640 1 -- 172.21.15.57:0/613427937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc101f00 msgr2=0x7fdffc1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fe001400640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc101f00 0x7fdffc1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fe001400640 1 -- 172.21.15.57:0/613427937 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc19fcc0 con 0x7fdffc103a90 2024-04-13T05:15:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.191+0000 7fe001400640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc19f650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdff4032380 tx=0x7fdff4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.192+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4041020 con 0x7fdffc103a90 2024-04-13T05:15:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.192+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdff4003e20 con 0x7fdffc103a90 2024-04-13T05:15:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.192+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc19ff50 con 0x7fdffc103a90 2024-04-13T05:15:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.192+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4039040 con 0x7fdffc103a90 2024-04-13T05:15:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.192+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1a0410 con 0x7fdffc103a90 2024-04-13T05:15:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.193+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc1093a0 con 0x7fdffc103a90 2024-04-13T05:15:24.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.197+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdff4044020 con 0x7fdffc103a90 2024-04-13T05:15:24.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.198+0000 7fdff9600640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfe40a3c30 0x7fdfe40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.199+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdff40f2a40 con 0x7fdffc103a90 2024-04-13T05:15:24.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.199+0000 7fe000a00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfe40a3c30 0x7fdfe40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.200+0000 7fe000a00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfe40a3c30 0x7fdfe40a60f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdff0005fd0 tx=0x7fdff000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:24.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.201+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdff404b070 con 0x7fdffc103a90 2024-04-13T05:15:24.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.345+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdff40bb030 con 0x7fdffc103a90 2024-04-13T05:15:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.451+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdffc062490 con 0x7fdfe40a3c30 2024-04-13T05:15:24.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.520+0000 7fdff9600640 1 -- 172.21.15.57:0/613427937 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdffc062490 con 0x7fdfe40a3c30 2024-04-13T05:15:24.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfe40a3c30 msgr2=0x7fdfe40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfe40a3c30 0x7fdfe40a60f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdff0005fd0 tx=0x7fdff000b390 comp rx=0 tx=0).stop 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 msgr2=0x7fdffc19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc19f650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdff4032380 tx=0x7fdff4032820 comp rx=0 tx=0).stop 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 shutdown_connections 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdfe40a3c30 0x7fdfe40a60f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdffc103a90 0x7fdffc19f650 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdffc1030d0 0x7fdffc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 --2- 172.21.15.57:0/613427937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdffc101f00 0x7fdffc1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 >> 172.21.15.57:0/613427937 conn(0x7fdffc0fd4c0 msgr2=0x7fdffc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 shutdown_connections 2024-04-13T05:15:24.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.523+0000 7fe001e00640 1 -- 172.21.15.57:0/613427937 wait complete. 2024-04-13T05:15:24.546 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-13T05:15:24.627 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:24 smithi057 ceph-mon[32331]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:24.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:24 smithi083 ceph-mon[35133]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 -- 172.21.15.57:0/434426823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4102950 msgr2=0x7f0dc4104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/434426823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4102950 0x7f0dc4104d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009900 tx=0x7f0db402bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 -- 172.21.15.57:0/434426823 shutdown_connections 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/434426823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4105280 0x7f0dc4107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/434426823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc4107bb0 0x7f0dc410a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/434426823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4102950 0x7f0dc4104d40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 -- 172.21.15.57:0/434426823 >> 172.21.15.57:0/434426823 conn(0x7f0dc40fc360 msgr2=0x7f0dc40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 -- 172.21.15.57:0/434426823 shutdown_connections 2024-04-13T05:15:24.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.764+0000 7f0dc3e00640 1 -- 172.21.15.57:0/434426823 wait complete. 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 Processor -- start 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 -- start start 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc4102950 0x7f0dc41022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 0x7f0dc4100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4107bb0 0x7f0dc4100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410da10 con 0x7f0dc4105280 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410d890 con 0x7f0dc4107bb0 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.765+0000 7f0dc3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410db90 con 0x7f0dc4102950 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 0x7f0dc4100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 0x7f0dc4100900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34932/0 (socket says 172.21.15.57:34932) 2024-04-13T05:15:24.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 -- 172.21.15.57:0/2210722189 learned_addr learned my addr 172.21.15.57:0/2210722189 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2a00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc4102950 0x7f0dc41022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc3400640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4107bb0 0x7f0dc4100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 -- 172.21.15.57:0/2210722189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc4102950 msgr2=0x7f0dc41022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc4102950 0x7f0dc41022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 -- 172.21.15.57:0/2210722189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4107bb0 msgr2=0x7f0dc4100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4107bb0 0x7f0dc4100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc2000640 1 -- 172.21.15.57:0/2210722189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc41016d0 con 0x7f0dc4105280 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.766+0000 7f0dc3400640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4107bb0 0x7f0dc4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:24.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.767+0000 7f0dc2000640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 0x7f0dc4100900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c970 tx=0x7f0db000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.767+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0014070 con 0x7f0dc4105280 2024-04-13T05:15:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.767+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc41a21b0 con 0x7f0dc4105280 2024-04-13T05:15:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.767+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db0004440 con 0x7f0dc4105280 2024-04-13T05:15:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.767+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0003e50 con 0x7f0dc4105280 2024-04-13T05:15:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.767+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a2700 con 0x7f0dc4105280 2024-04-13T05:15:24.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.768+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc40fde50 con 0x7f0dc4105280 2024-04-13T05:15:24.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.771+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0db0017070 con 0x7f0dc4105280 2024-04-13T05:15:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.772+0000 7f0dbb400640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0da80a3de0 0x7f0da80a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.772+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0db00d0590 con 0x7f0dc4105280 2024-04-13T05:15:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.772+0000 7f0dc2a00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0da80a3de0 0x7f0da80a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.773+0000 7f0dc2a00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0da80a3de0 0x7f0da80a62a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db402c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:24.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:24.777+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0db001e030 con 0x7f0dc4105280 2024-04-13T05:15:24.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:24 smithi189 ceph-mon[35531]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.034+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0dc4062400 con 0x7f0da80a3de0 2024-04-13T05:15:25.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.043+0000 7f0dbb400640 1 -- 172.21.15.57:0/2210722189 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0dc4062400 con 0x7f0da80a3de0 2024-04-13T05:15:25.044 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/qa/sv_1/ccc493bd-add7-459d-b711-9f479633876f 2024-04-13T05:15:25.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0da80a3de0 msgr2=0x7f0da80a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0da80a3de0 0x7f0da80a62a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db402c800 comp rx=0 tx=0).stop 2024-04-13T05:15:25.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 msgr2=0x7f0dc4100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 0x7f0dc4100900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c970 tx=0x7f0db000ce30 comp rx=0 tx=0).stop 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 shutdown_connections 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0da80a3de0 0x7f0da80a62a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dc4107bb0 0x7f0dc4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0dc4105280 0x7f0dc4100900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 --2- 172.21.15.57:0/2210722189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dc4102950 0x7f0dc41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 >> 172.21.15.57:0/2210722189 conn(0x7f0dc40fc360 msgr2=0x7f0dc4103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.045+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 shutdown_connections 2024-04-13T05:15:25.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.046+0000 7f0dc3e00640 1 -- 172.21.15.57:0/2210722189 wait complete. 2024-04-13T05:15:25.067 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 -- 172.21.15.57:0/652433132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 msgr2=0x7fc044109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 --2- 172.21.15.57:0/652433132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc044109c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc030009900 tx=0x7fc03002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 -- 172.21.15.57:0/652433132 shutdown_connections 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 --2- 172.21.15.57:0/652433132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410a1d0 0x7fc044110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 --2- 172.21.15.57:0/652433132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc044109c90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 --2- 172.21.15.57:0/652433132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc044108620 0x7fc044108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 -- 172.21.15.57:0/652433132 >> 172.21.15.57:0/652433132 conn(0x7fc0440775e0 msgr2=0x7fc044079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 -- 172.21.15.57:0/652433132 shutdown_connections 2024-04-13T05:15:25.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.284+0000 7fc04b600640 1 -- 172.21.15.57:0/652433132 wait complete. 2024-04-13T05:15:25.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.285+0000 7fc04b600640 1 Processor -- start 2024-04-13T05:15:25.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.285+0000 7fc04b600640 1 -- start start 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc04b600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044108620 0x7fc04419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc04b600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc04419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc04b600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc04410a1d0 0x7fc0441a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc04b600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441141b0 con 0x7fc044109810 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc04b600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc044114030 con 0x7fc044108620 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc04b600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc044114330 con 0x7fc04410a1d0 2024-04-13T05:15:25.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc043e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc04419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc043e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc04419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49014/0 (socket says 172.21.15.57:49014) 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc043e00640 1 -- 172.21.15.57:0/2871400672 learned_addr learned my addr 172.21.15.57:0/2871400672 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.286+0000 7fc048c00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044108620 0x7fc04419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc049600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc04410a1d0 0x7fc0441a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc043e00640 1 -- 172.21.15.57:0/2871400672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc04410a1d0 msgr2=0x7fc0441a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc043e00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc04410a1d0 0x7fc0441a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc043e00640 1 -- 172.21.15.57:0/2871400672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044108620 msgr2=0x7fc04419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc043e00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044108620 0x7fc04419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc043e00640 1 -- 172.21.15.57:0/2871400672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441a6440 con 0x7fc044109810 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc048c00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044108620 0x7fc04419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc043e00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc04419ed60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc0300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030043020 con 0x7fc044109810 2024-04-13T05:15:25.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0300054c0 con 0x7fc044109810 2024-04-13T05:15:25.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441a66d0 con 0x7fc044109810 2024-04-13T05:15:25.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030003d40 con 0x7fc044109810 2024-04-13T05:15:25.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.287+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a6c70 con 0x7fc044109810 2024-04-13T05:15:25.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.289+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04410f940 con 0x7fc044109810 2024-04-13T05:15:25.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.290+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc030046020 con 0x7fc044109810 2024-04-13T05:15:25.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.292+0000 7fc041600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc02c0a3cc0 0x7fc02c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.292+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc0300f2ce0 con 0x7fc044109810 2024-04-13T05:15:25.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.292+0000 7fc048c00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc02c0a3cc0 0x7fc02c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.293+0000 7fc048c00640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc02c0a3cc0 0x7fc02c0a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0380058c0 tx=0x7fc038005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:25.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.296+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc03004d020 con 0x7fc044109810 2024-04-13T05:15:25.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.537+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc04402d800 con 0x7fc044109810 2024-04-13T05:15:25.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.539+0000 7fc041600640 1 -- 172.21.15.57:0/2871400672 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7fc0300bf430 con 0x7fc044109810 2024-04-13T05:15:25.540 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:25.540 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:15:22.311895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:15:25.541 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.541+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc02c0a3cc0 msgr2=0x7fc02c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.541+0000 7fc04b600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc02c0a3cc0 0x7fc02c0a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0380058c0 tx=0x7fc038005850 comp rx=0 tx=0).stop 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.541+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 msgr2=0x7fc04419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.541+0000 7fc04b600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc04419ed60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc0300042f0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.541+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 shutdown_connections 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc02c0a3cc0 0x7fc02c0a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc04410a1d0 0x7fc0441a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc044109810 0x7fc04419ed60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 --2- 172.21.15.57:0/2871400672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044108620 0x7fc04419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 >> 172.21.15.57:0/2871400672 conn(0x7fc0440775e0 msgr2=0x7fc04410cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:25.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 shutdown_connections 2024-04-13T05:15:25.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.542+0000 7fc04b600640 1 -- 172.21.15.57:0/2871400672 wait complete. 2024-04-13T05:15:25.562 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:15:25.651 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:25 smithi057 ceph-mon[32331]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.651 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:25 smithi057 ceph-mon[32331]: pgmap v388: 129 pgs: 129 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s; 55 B/s, 0 objects/s recovering 2024-04-13T05:15:25.651 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:25 smithi057 ceph-mon[32331]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.651 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:25 smithi057 ceph-mon[32331]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-13T05:15:25.651 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:25 smithi057 ceph-mon[32331]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.651 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:25 smithi057 ceph-mon[32331]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.734 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:25 smithi189 ceph-mon[35531]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:25 smithi189 ceph-mon[35531]: pgmap v388: 129 pgs: 129 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s; 55 B/s, 0 objects/s recovering 2024-04-13T05:15:25.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:25 smithi189 ceph-mon[35531]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:25 smithi189 ceph-mon[35531]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-13T05:15:25.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:25 smithi189 ceph-mon[35531]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:25 smithi189 ceph-mon[35531]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:25 smithi083 ceph-mon[35133]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:25 smithi083 ceph-mon[35133]: pgmap v388: 129 pgs: 129 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s; 55 B/s, 0 objects/s recovering 2024-04-13T05:15:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:25 smithi083 ceph-mon[35133]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:25 smithi083 ceph-mon[35133]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-13T05:15:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:25 smithi083 ceph-mon[35133]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:25 smithi083 ceph-mon[35133]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:15:25.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.787+0000 7fdbe5000640 1 -- 172.21.15.57:0/3820270306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 msgr2=0x7fdbe01056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.787+0000 7fdbe5000640 1 --2- 172.21.15.57:0/3820270306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe01056b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ca0 tx=0x7fdbcc02c100 comp rx=0 tx=0).stop 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 -- 172.21.15.57:0/3820270306 shutdown_connections 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 --2- 172.21.15.57:0/3820270306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0105bf0 0x7fdbe010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 --2- 172.21.15.57:0/3820270306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe01056b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 --2- 172.21.15.57:0/3820270306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0104040 0x7fdbe0104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 -- 172.21.15.57:0/3820270306 >> 172.21.15.57:0/3820270306 conn(0x7fdbe00ff5d0 msgr2=0x7fdbe01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 -- 172.21.15.57:0/3820270306 shutdown_connections 2024-04-13T05:15:25.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.788+0000 7fdbe5000640 1 -- 172.21.15.57:0/3820270306 wait complete. 2024-04-13T05:15:25.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 Processor -- start 2024-04-13T05:15:25.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 -- start start 2024-04-13T05:15:25.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0104040 0x7fdbe019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0105bf0 0x7fdbe01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fbc0 con 0x7fdbe0105230 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fa40 con 0x7fdbe0104040 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbe5000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fd40 con 0x7fdbe0105bf0 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.789+0000 7fdbdea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49030/0 (socket says 172.21.15.57:49030) 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 -- 172.21.15.57:0/317851043 learned_addr learned my addr 172.21.15.57:0/317851043 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdf400640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0104040 0x7fdbe019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0105bf0 0x7fdbe01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 -- 172.21.15.57:0/317851043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0105bf0 msgr2=0x7fdbe01a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0105bf0 0x7fdbe01a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 -- 172.21.15.57:0/317851043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0104040 msgr2=0x7fdbe019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0104040 0x7fdbe019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 -- 172.21.15.57:0/317851043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe01a1e50 con 0x7fdbe0105230 2024-04-13T05:15:25.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.790+0000 7fdbdea00640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe019a770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098d0 tx=0x7fdbcc035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:25.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.791+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc039040 con 0x7fdbe0105230 2024-04-13T05:15:25.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.791+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbcc003e20 con 0x7fdbe0105230 2024-04-13T05:15:25.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.791+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0046f0 con 0x7fdbe0105230 2024-04-13T05:15:25.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.791+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe01a20e0 con 0x7fdbe0105230 2024-04-13T05:15:25.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.791+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe01a2600 con 0x7fdbe0105230 2024-04-13T05:15:25.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.792+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe010b360 con 0x7fdbe0105230 2024-04-13T05:15:25.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.794+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdbcc046030 con 0x7fdbe0105230 2024-04-13T05:15:25.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.796+0000 7fdbd7e00640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc40a3c40 0x7fdbc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:25.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.796+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdbcc0f2870 con 0x7fdbe0105230 2024-04-13T05:15:25.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.796+0000 7fdbdf400640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc40a3c40 0x7fdbc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:25.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.797+0000 7fdbdf400640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc40a3c40 0x7fdbc40a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00058c0 tx=0x7fdbd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:25.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:25.801+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbcc04d060 con 0x7fdbe0105230 2024-04-13T05:15:26.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.012+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbe002d800 con 0x7fdbe0105230 2024-04-13T05:15:26.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.016+0000 7fdbd7e00640 1 -- 172.21.15.57:0/317851043 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24583 (secure 0 0 0) 0x7fdbcc0bef60 con 0x7fdbe0105230 2024-04-13T05:15:26.018 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:26.018 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:15:13.150673+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc40a3c40 msgr2=0x7fdbc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc40a3c40 0x7fdbc40a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00058c0 tx=0x7fdbd0005850 comp rx=0 tx=0).stop 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 msgr2=0x7fdbe019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe019a770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098d0 tx=0x7fdbcc035e20 comp rx=0 tx=0).stop 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 shutdown_connections 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc40a3c40 0x7fdbc40a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0105bf0 0x7fdbe01a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0105230 0x7fdbe019a770 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 --2- 172.21.15.57:0/317851043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0104040 0x7fdbe019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 >> 172.21.15.57:0/317851043 conn(0x7fdbe00ff5d0 msgr2=0x7fdbe0108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 shutdown_connections 2024-04-13T05:15:26.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.019+0000 7fdbe5000640 1 -- 172.21.15.57:0/317851043 wait complete. 2024-04-13T05:15:26.043 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-13T05:15:26.043 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 -- 172.21.15.57:0/3696918719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c108620 msgr2=0x7fe08c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 --2- 172.21.15.57:0/3696918719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c108620 0x7fe08c108a40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009900 tx=0x7fe07c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 -- 172.21.15.57:0/3696918719 shutdown_connections 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 --2- 172.21.15.57:0/3696918719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c109810 0x7fe08c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 --2- 172.21.15.57:0/3696918719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe08c10a1d0 0x7fe08c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 --2- 172.21.15.57:0/3696918719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c108620 0x7fe08c108a40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 -- 172.21.15.57:0/3696918719 >> 172.21.15.57:0/3696918719 conn(0x7fe08c0775e0 msgr2=0x7fe08c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:26.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 -- 172.21.15.57:0/3696918719 shutdown_connections 2024-04-13T05:15:26.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.255+0000 7fe091a00640 1 -- 172.21.15.57:0/3696918719 wait complete. 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.256+0000 7fe091a00640 1 Processor -- start 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.256+0000 7fe091a00640 1 -- start start 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe091a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe08c108620 0x7fe08c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe091a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 0x7fe08c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe091a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c10a1d0 0x7fe08c1a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe091a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c1141a0 con 0x7fe08c109810 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe091a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c114020 con 0x7fe08c10a1d0 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe091a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c114320 con 0x7fe08c108620 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe082000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 0x7fe08c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe082000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 0x7fe08c19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49040/0 (socket says 172.21.15.57:49040) 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe082000640 1 -- 172.21.15.57:0/3670391768 learned_addr learned my addr 172.21.15.57:0/3670391768 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:26.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.257+0000 7fe08b400640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c10a1d0 0x7fe08c1a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe082000640 1 -- 172.21.15.57:0/3670391768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe08c108620 msgr2=0x7fe08c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe08aa00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe08c108620 0x7fe08c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe082000640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe08c108620 0x7fe08c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe082000640 1 -- 172.21.15.57:0/3670391768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c10a1d0 msgr2=0x7fe08c1a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe082000640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c10a1d0 0x7fe08c1a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe082000640 1 -- 172.21.15.57:0/3670391768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c1a64c0 con 0x7fe08c109810 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe08b400640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c10a1d0 0x7fe08c1a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe082000640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 0x7fe08c19ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe07400d950 tx=0x7fe07400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074017070 con 0x7fe08c109810 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe074009c30 con 0x7fe08c109810 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07400f620 con 0x7fe08c109810 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c1a67b0 con 0x7fe08c109810 2024-04-13T05:15:26.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.258+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c1a6cb0 con 0x7fe08c109810 2024-04-13T05:15:26.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.260+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08c10fa10 con 0x7fe08c109810 2024-04-13T05:15:26.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.261+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0740109a0 con 0x7fe08c109810 2024-04-13T05:15:26.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.264+0000 7fe083e00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe0700a3c40 0x7fe0700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.264+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe0740d0630 con 0x7fe08c109810 2024-04-13T05:15:26.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.264+0000 7fe08aa00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe0700a3c40 0x7fe0700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.265+0000 7fe08aa00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe0700a3c40 0x7fe0700a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0096c0 tx=0x7fe07c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:26.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.268+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe074009de0 con 0x7fe08c109810 2024-04-13T05:15:26.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.476+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe08c062400 con 0x7fe0700a3c40 2024-04-13T05:15:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.514+0000 7fe083e00640 1 -- 172.21.15.57:0/3670391768 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340499 (secure 0 0 0) 0x7fe08c062400 con 0x7fe0700a3c40 2024-04-13T05:15:26.521 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:26.523 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:15:26.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe0700a3c40 msgr2=0x7fe0700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe0700a3c40 0x7fe0700a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0096c0 tx=0x7fe07c005fd0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 msgr2=0x7fe08c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 0x7fe08c19ede0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe07400d950 tx=0x7fe07400de10 comp rx=0 tx=0).stop 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 shutdown_connections 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe0700a3c40 0x7fe0700a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08c10a1d0 0x7fe08c1a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe08c109810 0x7fe08c19ede0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 --2- 172.21.15.57:0/3670391768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe08c108620 0x7fe08c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 >> 172.21.15.57:0/3670391768 conn(0x7fe08c0775e0 msgr2=0x7fe08c10cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 shutdown_connections 2024-04-13T05:15:26.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.520+0000 7fe091a00640 1 -- 172.21.15.57:0/3670391768 wait complete. 2024-04-13T05:15:26.544 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-04-13T05:15:25.603681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617510,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":163,"num_read_kb":161,"num_write":321,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363236,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578588,"statfs":{"total":1151940427776,"available":1151568474112,"internally_reserved":0,"allocated":14422016,"data_stored":7897634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24283,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":37,"num_read_kb":52,"num_write":93,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176750+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:15:13.176750+0000","last_peered":"2024-04-13T05:15:13.176750+0000","last_clean":"2024-04-13T05:15:13.176750+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:15:13.176750+0000","last_undegraded":"2024-04-13T05:15:13.176750+0000","last_fullsized":"2024-04-13T05:15:13.176750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170460+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:15:13.170460+0000","last_peered":"2024-04-13T05:15:13.170460+0000","last_clean":"2024-04-13T05:15:13.170460+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:15:13.170460+0000","last_undegraded":"2024-04-13T05:15:13.170460+0000","last_fullsized":"2024-04-13T05:15:13.170460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169853+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:15:13.169853+0000","last_peered":"2024-04-13T05:15:13.169853+0000","last_clean":"2024-04-13T05:15:13.169853+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:15:13.169853+0000","last_undegraded":"2024-04-13T05:15:13.169853+0000","last_fullsized":"2024-04-13T05:15:13.169853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.161261+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:15:13.161261+0000","last_peered":"2024-04-13T05:15:13.161261+0000","last_clean":"2024-04-13T05:15:13.161261+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:15:13.161261+0000","last_undegraded":"2024-04-13T05:15:13.161261+0000","last_fullsized":"2024-04-13T05:15:13.161261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160257+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:15:13.160257+0000","last_peered":"2024-04-13T05:15:13.160257+0000","last_clean":"2024-04-13T05:15:13.160257+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:15:13.160257+0000","last_undegraded":"2024-04-13T05:15:13.160257+0000","last_fullsized":"2024-04-13T05:15:13.160257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.192477+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:15:13.192477+0000","last_peered":"2024-04-13T05:15:13.192477+0000","last_clean":"2024-04-13T05:15:13.192477+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:15:13.192477+0000","last_undegraded":"2024-04-13T05:15:13.192477+0000","last_fullsized":"2024-04-13T05:15:13.192477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165612+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:15:13.165612+0000","last_peered":"2024-04-13T05:15:13.165612+0000","last_clean":"2024-04-13T05:15:13.165612+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:15:13.165612+0000","last_undegraded":"2024-04-13T05:15:13.165612+0000","last_fullsized":"2024-04-13T05:15:13.165612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166644+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:15:13.166644+0000","last_peered":"2024-04-13T05:15:13.166644+0000","last_clean":"2024-04-13T05:15:13.166644+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:15:13.166644+0000","last_undegraded":"2024-04-13T05:15:13.166644+0000","last_fullsized":"2024-04-13T05:15:13.166644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.191032+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:15:13.191032+0000","last_peered":"2024-04-13T05:15:13.191032+0000","last_clean":"2024-04-13T05:15:13.191032+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:15:13.191032+0000","last_undegraded":"2024-04-13T05:15:13.191032+0000","last_fullsized":"2024-04-13T05:15:13.191032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.194276+0000","last_change":"2024-04-13T05:14:53.598758+0000","last_active":"2024-04-13T05:15:13.194276+0000","last_peered":"2024-04-13T05:15:13.194276+0000","last_clean":"2024-04-13T05:15:13.194276+0000","last_became_active":"2024-04-13T05:14:53.598265+0000","last_became_peered":"2024-04-13T05:14:53.598265+0000","last_unstale":"2024-04-13T05:15:13.194276+0000","last_undegraded":"2024-04-13T05:15:13.194276+0000","last_fullsized":"2024-04-13T05:15:13.194276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:53:47.570217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166466+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:15:13.166466+0000","last_peered":"2024-04-13T05:15:13.166466+0000","last_clean":"2024-04-13T05:15:13.166466+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:15:13.166466+0000","last_undegraded":"2024-04-13T05:15:13.166466+0000","last_fullsized":"2024-04-13T05:15:13.166466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.161143+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:15:13.161143+0000","last_peered":"2024-04-13T05:15:13.161143+0000","last_clean":"2024-04-13T05:15:13.161143+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:15:13.161143+0000","last_undegraded":"2024-04-13T05:15:13.161143+0000","last_fullsized":"2024-04-13T05:15:13.161143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.188954+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:15:13.188954+0000","last_peered":"2024-04-13T05:15:13.188954+0000","last_clean":"2024-04-13T05:15:13.188954+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:15:13.188954+0000","last_undegraded":"2024-04-13T05:15:13.188954+0000","last_fullsized":"2024-04-13T05:15:13.188954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169887+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:15:13.169887+0000","last_peered":"2024-04-13T05:15:13.169887+0000","last_clean":"2024-04-13T05:15:13.169887+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:15:13.169887+0000","last_undegraded":"2024-04-13T05:15:13.169887+0000","last_fullsized":"2024-04-13T05:15:13.169887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170523+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:15:13.170523+0000","last_peered":"2024-04-13T05:15:13.170523+0000","last_clean":"2024-04-13T05:15:13.170523+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:15:13.170523+0000","last_undegraded":"2024-04-13T05:15:13.170523+0000","last_fullsized":"2024-04-13T05:15:13.170523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189835+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:15:13.189835+0000","last_peered":"2024-04-13T05:15:13.189835+0000","last_clean":"2024-04-13T05:15:13.189835+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:15:13.189835+0000","last_undegraded":"2024-04-13T05:15:13.189835+0000","last_fullsized":"2024-04-13T05:15:13.189835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176522+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:15:13.176522+0000","last_peered":"2024-04-13T05:15:13.176522+0000","last_clean":"2024-04-13T05:15:13.176522+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:15:13.176522+0000","last_undegraded":"2024-04-13T05:15:13.176522+0000","last_fullsized":"2024-04-13T05:15:13.176522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160928+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:15:13.160928+0000","last_peered":"2024-04-13T05:15:13.160928+0000","last_clean":"2024-04-13T05:15:13.160928+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:15:13.160928+0000","last_undegraded":"2024-04-13T05:15:13.160928+0000","last_fullsized":"2024-04-13T05:15:13.160928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176172+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:15:13.176172+0000","last_peered":"2024-04-13T05:15:13.176172+0000","last_clean":"2024-04-13T05:15:13.176172+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:15:13.176172+0000","last_undegraded":"2024-04-13T05:15:13.176172+0000","last_fullsized":"2024-04-13T05:15:13.176172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.179323+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:15:13.179323+0000","last_peered":"2024-04-13T05:15:13.179323+0000","last_clean":"2024-04-13T05:15:13.179323+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:15:13.179323+0000","last_undegraded":"2024-04-13T05:15:13.179323+0000","last_fullsized":"2024-04-13T05:15:13.179323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.316802+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:15:21.316802+0000","last_peered":"2024-04-13T05:15:21.316802+0000","last_clean":"2024-04-13T05:15:21.316802+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:15:21.316802+0000","last_undegraded":"2024-04-13T05:15:21.316802+0000","last_fullsized":"2024-04-13T05:15:21.316802+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165721+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:15:13.165721+0000","last_peered":"2024-04-13T05:15:13.165721+0000","last_clean":"2024-04-13T05:15:13.165721+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:15:13.165721+0000","last_undegraded":"2024-04-13T05:15:13.165721+0000","last_fullsized":"2024-04-13T05:15:13.165721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159313+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:15:13.159313+0000","last_peered":"2024-04-13T05:15:13.159313+0000","last_clean":"2024-04-13T05:15:13.159313+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:15:13.159313+0000","last_undegraded":"2024-04-13T05:15:13.159313+0000","last_fullsized":"2024-04-13T05:15:13.159313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166063+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:15:13.166063+0000","last_peered":"2024-04-13T05:15:13.166063+0000","last_clean":"2024-04-13T05:15:13.166063+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:15:13.166063+0000","last_undegraded":"2024-04-13T05:15:13.166063+0000","last_fullsized":"2024-04-13T05:15:13.166063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170842+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:15:13.170842+0000","last_peered":"2024-04-13T05:15:13.170842+0000","last_clean":"2024-04-13T05:15:13.170842+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:15:13.170842+0000","last_undegraded":"2024-04-13T05:15:13.170842+0000","last_fullsized":"2024-04-13T05:15:13.170842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.197015+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:15:13.197015+0000","last_peered":"2024-04-13T05:15:13.197015+0000","last_clean":"2024-04-13T05:15:13.197015+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:15:13.197015+0000","last_undegraded":"2024-04-13T05:15:13.197015+0000","last_fullsized":"2024-04-13T05:15:13.197015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.316372+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:15:21.316372+0000","last_peered":"2024-04-13T05:15:21.316372+0000","last_clean":"2024-04-13T05:15:21.316372+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:15:21.316372+0000","last_undegraded":"2024-04-13T05:15:21.316372+0000","last_fullsized":"2024-04-13T05:15:21.316372+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189867+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:15:13.189867+0000","last_peered":"2024-04-13T05:15:13.189867+0000","last_clean":"2024-04-13T05:15:13.189867+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:15:13.189867+0000","last_undegraded":"2024-04-13T05:15:13.189867+0000","last_fullsized":"2024-04-13T05:15:13.189867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159393+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:15:13.159393+0000","last_peered":"2024-04-13T05:15:13.159393+0000","last_clean":"2024-04-13T05:15:13.159393+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:15:13.159393+0000","last_undegraded":"2024-04-13T05:15:13.159393+0000","last_fullsized":"2024-04-13T05:15:13.159393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160899+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:15:13.160899+0000","last_peered":"2024-04-13T05:15:13.160899+0000","last_clean":"2024-04-13T05:15:13.160899+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:15:13.160899+0000","last_undegraded":"2024-04-13T05:15:13.160899+0000","last_fullsized":"2024-04-13T05:15:13.160899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170807+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:15:13.170807+0000","last_peered":"2024-04-13T05:15:13.170807+0000","last_clean":"2024-04-13T05:15:13.170807+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:15:13.170807+0000","last_undegraded":"2024-04-13T05:15:13.170807+0000","last_fullsized":"2024-04-13T05:15:13.170807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.175725+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:15:13.175725+0000","last_peered":"2024-04-13T05:15:13.175725+0000","last_clean":"2024-04-13T05:15:13.175725+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:15:13.175725+0000","last_undegraded":"2024-04-13T05:15:13.175725+0000","last_fullsized":"2024-04-13T05:15:13.175725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:22.329068+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:15:22.329068+0000","last_peered":"2024-04-13T05:15:22.329068+0000","last_clean":"2024-04-13T05:15:22.329068+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:15:22.329068+0000","last_undegraded":"2024-04-13T05:15:22.329068+0000","last_fullsized":"2024-04-13T05:15:22.329068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160716+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:15:13.160716+0000","last_peered":"2024-04-13T05:15:13.160716+0000","last_clean":"2024-04-13T05:15:13.160716+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:15:13.160716+0000","last_undegraded":"2024-04-13T05:15:13.160716+0000","last_fullsized":"2024-04-13T05:15:13.160716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189675+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:15:13.189675+0000","last_peered":"2024-04-13T05:15:13.189675+0000","last_clean":"2024-04-13T05:15:13.189675+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:15:13.189675+0000","last_undegraded":"2024-04-13T05:15:13.189675+0000","last_fullsized":"2024-04-13T05:15:13.189675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.315699+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:15:21.315699+0000","last_peered":"2024-04-13T05:15:21.315699+0000","last_clean":"2024-04-13T05:15:21.315699+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:15:21.315699+0000","last_undegraded":"2024-04-13T05:15:21.315699+0000","last_fullsized":"2024-04-13T05:15:21.315699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171189+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:15:13.171189+0000","last_peered":"2024-04-13T05:15:13.171189+0000","last_clean":"2024-04-13T05:15:13.171189+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:15:13.171189+0000","last_undegraded":"2024-04-13T05:15:13.171189+0000","last_fullsized":"2024-04-13T05:15:13.171189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176413+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:15:13.176413+0000","last_peered":"2024-04-13T05:15:13.176413+0000","last_clean":"2024-04-13T05:15:13.176413+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:15:13.176413+0000","last_undegraded":"2024-04-13T05:15:13.176413+0000","last_fullsized":"2024-04-13T05:15:13.176413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.317883+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:15:21.317883+0000","last_peered":"2024-04-13T05:15:21.317883+0000","last_clean":"2024-04-13T05:15:21.317883+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:15:21.317883+0000","last_undegraded":"2024-04-13T05:15:21.317883+0000","last_fullsized":"2024-04-13T05:15:21.317883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167032+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:15:13.167032+0000","last_peered":"2024-04-13T05:15:13.167032+0000","last_clean":"2024-04-13T05:15:13.167032+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:15:13.167032+0000","last_undegraded":"2024-04-13T05:15:13.167032+0000","last_fullsized":"2024-04-13T05:15:13.167032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170295+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:15:13.170295+0000","last_peered":"2024-04-13T05:15:13.170295+0000","last_clean":"2024-04-13T05:15:13.170295+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:15:13.170295+0000","last_undegraded":"2024-04-13T05:15:13.170295+0000","last_fullsized":"2024-04-13T05:15:13.170295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169472+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:15:13.169472+0000","last_peered":"2024-04-13T05:15:13.169472+0000","last_clean":"2024-04-13T05:15:13.169472+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:15:13.169472+0000","last_undegraded":"2024-04-13T05:15:13.169472+0000","last_fullsized":"2024-04-13T05:15:13.169472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.158792+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:15:13.158792+0000","last_peered":"2024-04-13T05:15:13.158792+0000","last_clean":"2024-04-13T05:15:13.158792+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:15:13.158792+0000","last_undegraded":"2024-04-13T05:15:13.158792+0000","last_fullsized":"2024-04-13T05:15:13.158792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166169+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:15:13.166169+0000","last_peered":"2024-04-13T05:15:13.166169+0000","last_clean":"2024-04-13T05:15:13.166169+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:15:13.166169+0000","last_undegraded":"2024-04-13T05:15:13.166169+0000","last_fullsized":"2024-04-13T05:15:13.166169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:23.332274+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:15:23.332274+0000","last_peered":"2024-04-13T05:15:23.332274+0000","last_clean":"2024-04-13T05:15:23.332274+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:15:23.332274+0000","last_undegraded":"2024-04-13T05:15:23.332274+0000","last_fullsized":"2024-04-13T05:15:23.332274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.190326+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:15:13.190326+0000","last_peered":"2024-04-13T05:15:13.190326+0000","last_clean":"2024-04-13T05:15:13.190326+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:15:13.190326+0000","last_undegraded":"2024-04-13T05:15:13.190326+0000","last_fullsized":"2024-04-13T05:15:13.190326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.158339+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:15:13.158339+0000","last_peered":"2024-04-13T05:15:13.158339+0000","last_clean":"2024-04-13T05:15:13.158339+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:15:13.158339+0000","last_undegraded":"2024-04-13T05:15:13.158339+0000","last_fullsized":"2024-04-13T05:15:13.158339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.297007+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:15:20.297007+0000","last_peered":"2024-04-13T05:15:20.297007+0000","last_clean":"2024-04-13T05:15:20.297007+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:15:20.297007+0000","last_undegraded":"2024-04-13T05:15:20.297007+0000","last_fullsized":"2024-04-13T05:15:20.297007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171204+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:15:13.171204+0000","last_peered":"2024-04-13T05:15:13.171204+0000","last_clean":"2024-04-13T05:15:13.171204+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:15:13.171204+0000","last_undegraded":"2024-04-13T05:15:13.171204+0000","last_fullsized":"2024-04-13T05:15:13.171204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.161182+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:15:13.161182+0000","last_peered":"2024-04-13T05:15:13.161182+0000","last_clean":"2024-04-13T05:15:13.161182+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:15:13.161182+0000","last_undegraded":"2024-04-13T05:15:13.161182+0000","last_fullsized":"2024-04-13T05:15:13.161182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.299233+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:15:20.299233+0000","last_peered":"2024-04-13T05:15:20.299233+0000","last_clean":"2024-04-13T05:15:20.299233+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:15:20.299233+0000","last_undegraded":"2024-04-13T05:15:20.299233+0000","last_fullsized":"2024-04-13T05:15:20.299233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.190373+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:15:13.190373+0000","last_peered":"2024-04-13T05:15:13.190373+0000","last_clean":"2024-04-13T05:15:13.190373+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:15:13.190373+0000","last_undegraded":"2024-04-13T05:15:13.190373+0000","last_fullsized":"2024-04-13T05:15:13.190373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.158810+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:15:13.158810+0000","last_peered":"2024-04-13T05:15:13.158810+0000","last_clean":"2024-04-13T05:15:13.158810+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:15:13.158810+0000","last_undegraded":"2024-04-13T05:15:13.158810+0000","last_fullsized":"2024-04-13T05:15:13.158810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'19","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:24.335910+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:15:24.335910+0000","last_peered":"2024-04-13T05:15:24.335910+0000","last_clean":"2024-04-13T05:15:24.335910+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:15:24.335910+0000","last_undegraded":"2024-04-13T05:15:24.335910+0000","last_fullsized":"2024-04-13T05:15:24.335910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":28,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.162496+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:15:13.162496+0000","last_peered":"2024-04-13T05:15:13.162496+0000","last_clean":"2024-04-13T05:15:13.162496+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:15:13.162496+0000","last_undegraded":"2024-04-13T05:15:13.162496+0000","last_fullsized":"2024-04-13T05:15:13.162496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171445+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:15:13.171445+0000","last_peered":"2024-04-13T05:15:13.171445+0000","last_clean":"2024-04-13T05:15:13.171445+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:15:13.171445+0000","last_undegraded":"2024-04-13T05:15:13.171445+0000","last_fullsized":"2024-04-13T05:15:13.171445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.317658+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:15:21.317658+0000","last_peered":"2024-04-13T05:15:21.317658+0000","last_clean":"2024-04-13T05:15:21.317658+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:15:21.317658+0000","last_undegraded":"2024-04-13T05:15:21.317658+0000","last_fullsized":"2024-04-13T05:15:21.317658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.187970+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:15:13.187970+0000","last_peered":"2024-04-13T05:15:13.187970+0000","last_clean":"2024-04-13T05:15:13.187970+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:15:13.187970+0000","last_undegraded":"2024-04-13T05:15:13.187970+0000","last_fullsized":"2024-04-13T05:15:13.187970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.175850+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:15:13.175850+0000","last_peered":"2024-04-13T05:15:13.175850+0000","last_clean":"2024-04-13T05:15:13.175850+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:15:13.175850+0000","last_undegraded":"2024-04-13T05:15:13.175850+0000","last_fullsized":"2024-04-13T05:15:13.175850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.301331+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:15:20.301331+0000","last_peered":"2024-04-13T05:15:20.301331+0000","last_clean":"2024-04-13T05:15:20.301331+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:15:20.301331+0000","last_undegraded":"2024-04-13T05:15:20.301331+0000","last_fullsized":"2024-04-13T05:15:20.301331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166585+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:15:13.166585+0000","last_peered":"2024-04-13T05:15:13.166585+0000","last_clean":"2024-04-13T05:15:13.166585+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:15:13.166585+0000","last_undegraded":"2024-04-13T05:15:13.166585+0000","last_fullsized":"2024-04-13T05:15:13.166585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159854+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:15:13.159854+0000","last_peered":"2024-04-13T05:15:13.159854+0000","last_clean":"2024-04-13T05:15:13.159854+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:15:13.159854+0000","last_undegraded":"2024-04-13T05:15:13.159854+0000","last_fullsized":"2024-04-13T05:15:13.159854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:23.336765+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:15:23.336765+0000","last_peered":"2024-04-13T05:15:23.336765+0000","last_clean":"2024-04-13T05:15:23.336765+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:15:23.336765+0000","last_undegraded":"2024-04-13T05:15:23.336765+0000","last_fullsized":"2024-04-13T05:15:23.336765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165684+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:15:13.165684+0000","last_peered":"2024-04-13T05:15:13.165684+0000","last_clean":"2024-04-13T05:15:13.165684+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:15:13.165684+0000","last_undegraded":"2024-04-13T05:15:13.165684+0000","last_fullsized":"2024-04-13T05:15:13.165684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189348+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:15:13.189348+0000","last_peered":"2024-04-13T05:15:13.189348+0000","last_clean":"2024-04-13T05:15:13.189348+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:15:13.189348+0000","last_undegraded":"2024-04-13T05:15:13.189348+0000","last_fullsized":"2024-04-13T05:15:13.189348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.316317+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:15:21.316317+0000","last_peered":"2024-04-13T05:15:21.316317+0000","last_clean":"2024-04-13T05:15:21.316317+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:15:21.316317+0000","last_undegraded":"2024-04-13T05:15:21.316317+0000","last_fullsized":"2024-04-13T05:15:21.316317+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171226+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:15:13.171226+0000","last_peered":"2024-04-13T05:15:13.171226+0000","last_clean":"2024-04-13T05:15:13.171226+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:15:13.171226+0000","last_undegraded":"2024-04-13T05:15:13.171226+0000","last_fullsized":"2024-04-13T05:15:13.171226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171040+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:15:13.171040+0000","last_peered":"2024-04-13T05:15:13.171040+0000","last_clean":"2024-04-13T05:15:13.171040+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:15:13.171040+0000","last_undegraded":"2024-04-13T05:15:13.171040+0000","last_fullsized":"2024-04-13T05:15:13.171040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.164121+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:15:13.164121+0000","last_peered":"2024-04-13T05:15:13.164121+0000","last_clean":"2024-04-13T05:15:13.164121+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:15:13.164121+0000","last_undegraded":"2024-04-13T05:15:13.164121+0000","last_fullsized":"2024-04-13T05:15:13.164121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.190714+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:15:13.190714+0000","last_peered":"2024-04-13T05:15:13.190714+0000","last_clean":"2024-04-13T05:15:13.190714+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:15:13.190714+0000","last_undegraded":"2024-04-13T05:15:13.190714+0000","last_fullsized":"2024-04-13T05:15:13.190714+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160616+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:15:13.160616+0000","last_peered":"2024-04-13T05:15:13.160616+0000","last_clean":"2024-04-13T05:15:13.160616+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:15:13.160616+0000","last_undegraded":"2024-04-13T05:15:13.160616+0000","last_fullsized":"2024-04-13T05:15:13.160616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160006+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:15:13.160006+0000","last_peered":"2024-04-13T05:15:13.160006+0000","last_clean":"2024-04-13T05:15:13.160006+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:15:13.160006+0000","last_undegraded":"2024-04-13T05:15:13.160006+0000","last_fullsized":"2024-04-13T05:15:13.160006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.192694+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:15:13.192694+0000","last_peered":"2024-04-13T05:15:13.192694+0000","last_clean":"2024-04-13T05:15:13.192694+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:15:13.192694+0000","last_undegraded":"2024-04-13T05:15:13.192694+0000","last_fullsized":"2024-04-13T05:15:13.192694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189011+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:15:13.189011+0000","last_peered":"2024-04-13T05:15:13.189011+0000","last_clean":"2024-04-13T05:15:13.189011+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:15:13.189011+0000","last_undegraded":"2024-04-13T05:15:13.189011+0000","last_fullsized":"2024-04-13T05:15:13.189011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167435+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:15:13.167435+0000","last_peered":"2024-04-13T05:15:13.167435+0000","last_clean":"2024-04-13T05:15:13.167435+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:15:13.167435+0000","last_undegraded":"2024-04-13T05:15:13.167435+0000","last_fullsized":"2024-04-13T05:15:13.167435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.193237+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:15:13.193237+0000","last_peered":"2024-04-13T05:15:13.193237+0000","last_clean":"2024-04-13T05:15:13.193237+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:15:13.193237+0000","last_undegraded":"2024-04-13T05:15:13.193237+0000","last_fullsized":"2024-04-13T05:15:13.193237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189463+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:15:13.189463+0000","last_peered":"2024-04-13T05:15:13.189463+0000","last_clean":"2024-04-13T05:15:13.189463+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:15:13.189463+0000","last_undegraded":"2024-04-13T05:15:13.189463+0000","last_fullsized":"2024-04-13T05:15:13.189463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169965+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:15:13.169965+0000","last_peered":"2024-04-13T05:15:13.169965+0000","last_clean":"2024-04-13T05:15:13.169965+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:15:13.169965+0000","last_undegraded":"2024-04-13T05:15:13.169965+0000","last_fullsized":"2024-04-13T05:15:13.169965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:22.330628+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:15:22.330628+0000","last_peered":"2024-04-13T05:15:22.330628+0000","last_clean":"2024-04-13T05:15:22.330628+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:15:22.330628+0000","last_undegraded":"2024-04-13T05:15:22.330628+0000","last_fullsized":"2024-04-13T05:15:22.330628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166232+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:15:13.166232+0000","last_peered":"2024-04-13T05:15:13.166232+0000","last_clean":"2024-04-13T05:15:13.166232+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:15:13.166232+0000","last_undegraded":"2024-04-13T05:15:13.166232+0000","last_fullsized":"2024-04-13T05:15:13.166232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167580+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:15:13.167580+0000","last_peered":"2024-04-13T05:15:13.167580+0000","last_clean":"2024-04-13T05:15:13.167580+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:15:13.167580+0000","last_undegraded":"2024-04-13T05:15:13.167580+0000","last_fullsized":"2024-04-13T05:15:13.167580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:22.325780+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:15:22.325780+0000","last_peered":"2024-04-13T05:15:22.325780+0000","last_clean":"2024-04-13T05:15:22.325780+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:15:22.325780+0000","last_undegraded":"2024-04-13T05:15:22.325780+0000","last_fullsized":"2024-04-13T05:15:22.325780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165351+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:15:13.165351+0000","last_peered":"2024-04-13T05:15:13.165351+0000","last_clean":"2024-04-13T05:15:13.165351+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:15:13.165351+0000","last_undegraded":"2024-04-13T05:15:13.165351+0000","last_fullsized":"2024-04-13T05:15:13.165351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.196100+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:15:13.196100+0000","last_peered":"2024-04-13T05:15:13.196100+0000","last_clean":"2024-04-13T05:15:13.196100+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:15:13.196100+0000","last_undegraded":"2024-04-13T05:15:13.196100+0000","last_fullsized":"2024-04-13T05:15:13.196100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.186051+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:15:13.186051+0000","last_peered":"2024-04-13T05:15:13.186051+0000","last_clean":"2024-04-13T05:15:13.186051+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:15:13.186051+0000","last_undegraded":"2024-04-13T05:15:13.186051+0000","last_fullsized":"2024-04-13T05:15:13.186051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.313048+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:15:21.313048+0000","last_peered":"2024-04-13T05:15:21.313048+0000","last_clean":"2024-04-13T05:15:21.313048+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:15:21.313048+0000","last_undegraded":"2024-04-13T05:15:21.313048+0000","last_fullsized":"2024-04-13T05:15:21.313048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176016+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:15:13.176016+0000","last_peered":"2024-04-13T05:15:13.176016+0000","last_clean":"2024-04-13T05:15:13.176016+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:15:13.176016+0000","last_undegraded":"2024-04-13T05:15:13.176016+0000","last_fullsized":"2024-04-13T05:15:13.176016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.297045+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:15:20.297045+0000","last_peered":"2024-04-13T05:15:20.297045+0000","last_clean":"2024-04-13T05:15:20.297045+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:15:20.297045+0000","last_undegraded":"2024-04-13T05:15:20.297045+0000","last_fullsized":"2024-04-13T05:15:20.297045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189072+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:15:13.189072+0000","last_peered":"2024-04-13T05:15:13.189072+0000","last_clean":"2024-04-13T05:15:13.189072+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:15:13.189072+0000","last_undegraded":"2024-04-13T05:15:13.189072+0000","last_fullsized":"2024-04-13T05:15:13.189072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166839+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:15:13.166839+0000","last_peered":"2024-04-13T05:15:13.166839+0000","last_clean":"2024-04-13T05:15:13.166839+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:15:13.166839+0000","last_undegraded":"2024-04-13T05:15:13.166839+0000","last_fullsized":"2024-04-13T05:15:13.166839+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:23.646422+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:15:23.646422+0000","last_peered":"2024-04-13T05:15:23.646422+0000","last_clean":"2024-04-13T05:15:23.646422+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:15:23.646422+0000","last_undegraded":"2024-04-13T05:15:23.646422+0000","last_fullsized":"2024-04-13T05:15:23.646422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166503+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:15:13.166503+0000","last_peered":"2024-04-13T05:15:13.166503+0000","last_clean":"2024-04-13T05:15:13.166503+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:15:13.166503+0000","last_undegraded":"2024-04-13T05:15:13.166503+0000","last_fullsized":"2024-04-13T05:15:13.166503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.168261+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:15:13.168261+0000","last_peered":"2024-04-13T05:15:13.168261+0000","last_clean":"2024-04-13T05:15:13.168261+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:15:13.168261+0000","last_undegraded":"2024-04-13T05:15:13.168261+0000","last_fullsized":"2024-04-13T05:15:13.168261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.295825+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:15:20.295825+0000","last_peered":"2024-04-13T05:15:20.295825+0000","last_clean":"2024-04-13T05:15:20.295825+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:15:20.295825+0000","last_undegraded":"2024-04-13T05:15:20.295825+0000","last_fullsized":"2024-04-13T05:15:20.295825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170645+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:15:13.170645+0000","last_peered":"2024-04-13T05:15:13.170645+0000","last_clean":"2024-04-13T05:15:13.170645+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:15:13.170645+0000","last_undegraded":"2024-04-13T05:15:13.170645+0000","last_fullsized":"2024-04-13T05:15:13.170645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160036+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:15:13.160036+0000","last_peered":"2024-04-13T05:15:13.160036+0000","last_clean":"2024-04-13T05:15:13.160036+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:15:13.160036+0000","last_undegraded":"2024-04-13T05:15:13.160036+0000","last_fullsized":"2024-04-13T05:15:13.160036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.302015+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:15:20.302015+0000","last_peered":"2024-04-13T05:15:20.302015+0000","last_clean":"2024-04-13T05:15:20.302015+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:15:20.302015+0000","last_undegraded":"2024-04-13T05:15:20.302015+0000","last_fullsized":"2024-04-13T05:15:20.302015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159556+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:15:13.159556+0000","last_peered":"2024-04-13T05:15:13.159556+0000","last_clean":"2024-04-13T05:15:13.159556+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:15:13.159556+0000","last_undegraded":"2024-04-13T05:15:13.159556+0000","last_fullsized":"2024-04-13T05:15:13.159556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160029+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:15:13.160029+0000","last_peered":"2024-04-13T05:15:13.160029+0000","last_clean":"2024-04-13T05:15:13.160029+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:15:13.160029+0000","last_undegraded":"2024-04-13T05:15:13.160029+0000","last_fullsized":"2024-04-13T05:15:13.160029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.298929+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:15:20.298929+0000","last_peered":"2024-04-13T05:15:20.298929+0000","last_clean":"2024-04-13T05:15:20.298929+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:15:20.298929+0000","last_undegraded":"2024-04-13T05:15:20.298929+0000","last_fullsized":"2024-04-13T05:15:20.298929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176375+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:15:13.176375+0000","last_peered":"2024-04-13T05:15:13.176375+0000","last_clean":"2024-04-13T05:15:13.176375+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:15:13.176375+0000","last_undegraded":"2024-04-13T05:15:13.176375+0000","last_fullsized":"2024-04-13T05:15:13.176375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160761+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:15:13.160761+0000","last_peered":"2024-04-13T05:15:13.160761+0000","last_clean":"2024-04-13T05:15:13.160761+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:15:13.160761+0000","last_undegraded":"2024-04-13T05:15:13.160761+0000","last_fullsized":"2024-04-13T05:15:13.160761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:15.426478+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:15:15.426478+0000","last_peered":"2024-04-13T05:15:15.426478+0000","last_clean":"2024-04-13T05:15:15.426478+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:15:15.426478+0000","last_undegraded":"2024-04-13T05:15:15.426478+0000","last_fullsized":"2024-04-13T05:15:15.426478+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167755+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:15:13.167755+0000","last_peered":"2024-04-13T05:15:13.167755+0000","last_clean":"2024-04-13T05:15:13.167755+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:15:13.167755+0000","last_undegraded":"2024-04-13T05:15:13.167755+0000","last_fullsized":"2024-04-13T05:15:13.167755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159818+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:15:13.159818+0000","last_peered":"2024-04-13T05:15:13.159818+0000","last_clean":"2024-04-13T05:15:13.159818+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:15:13.159818+0000","last_undegraded":"2024-04-13T05:15:13.159818+0000","last_fullsized":"2024-04-13T05:15:13.159818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.302063+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:15:20.302063+0000","last_peered":"2024-04-13T05:15:20.302063+0000","last_clean":"2024-04-13T05:15:20.302063+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:15:20.302063+0000","last_undegraded":"2024-04-13T05:15:20.302063+0000","last_fullsized":"2024-04-13T05:15:20.302063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167619+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:15:13.167619+0000","last_peered":"2024-04-13T05:15:13.167619+0000","last_clean":"2024-04-13T05:15:13.167619+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:15:13.167619+0000","last_undegraded":"2024-04-13T05:15:13.167619+0000","last_fullsized":"2024-04-13T05:15:13.167619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.187605+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:15:13.187605+0000","last_peered":"2024-04-13T05:15:13.187605+0000","last_clean":"2024-04-13T05:15:13.187605+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:15:13.187605+0000","last_undegraded":"2024-04-13T05:15:13.187605+0000","last_fullsized":"2024-04-13T05:15:13.187605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.317877+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:15:21.317877+0000","last_peered":"2024-04-13T05:15:21.317877+0000","last_clean":"2024-04-13T05:15:21.317877+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:15:21.317877+0000","last_undegraded":"2024-04-13T05:15:21.317877+0000","last_fullsized":"2024-04-13T05:15:21.317877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166807+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:15:13.166807+0000","last_peered":"2024-04-13T05:15:13.166807+0000","last_clean":"2024-04-13T05:15:13.166807+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:15:13.166807+0000","last_undegraded":"2024-04-13T05:15:13.166807+0000","last_fullsized":"2024-04-13T05:15:13.166807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170195+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:15:13.170195+0000","last_peered":"2024-04-13T05:15:13.170195+0000","last_clean":"2024-04-13T05:15:13.170195+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:15:13.170195+0000","last_undegraded":"2024-04-13T05:15:13.170195+0000","last_fullsized":"2024-04-13T05:15:13.170195+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171150+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:15:13.171150+0000","last_peered":"2024-04-13T05:15:13.171150+0000","last_clean":"2024-04-13T05:15:13.171150+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:15:13.171150+0000","last_undegraded":"2024-04-13T05:15:13.171150+0000","last_fullsized":"2024-04-13T05:15:13.171150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166200+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:15:13.166200+0000","last_peered":"2024-04-13T05:15:13.166200+0000","last_clean":"2024-04-13T05:15:13.166200+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:15:13.166200+0000","last_undegraded":"2024-04-13T05:15:13.166200+0000","last_fullsized":"2024-04-13T05:15:13.166200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166692+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:15:13.166692+0000","last_peered":"2024-04-13T05:15:13.166692+0000","last_clean":"2024-04-13T05:15:13.166692+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:15:13.166692+0000","last_undegraded":"2024-04-13T05:15:13.166692+0000","last_fullsized":"2024-04-13T05:15:13.166692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189354+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:15:13.189354+0000","last_peered":"2024-04-13T05:15:13.189354+0000","last_clean":"2024-04-13T05:15:13.189354+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:15:13.189354+0000","last_undegraded":"2024-04-13T05:15:13.189354+0000","last_fullsized":"2024-04-13T05:15:13.189354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167272+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:15:13.167272+0000","last_peered":"2024-04-13T05:15:13.167272+0000","last_clean":"2024-04-13T05:15:13.167272+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:15:13.167272+0000","last_undegraded":"2024-04-13T05:15:13.167272+0000","last_fullsized":"2024-04-13T05:15:13.167272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176515+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:15:13.176515+0000","last_peered":"2024-04-13T05:15:13.176515+0000","last_clean":"2024-04-13T05:15:13.176515+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:15:13.176515+0000","last_undegraded":"2024-04-13T05:15:13.176515+0000","last_fullsized":"2024-04-13T05:15:13.176515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169509+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:15:13.169509+0000","last_peered":"2024-04-13T05:15:13.169509+0000","last_clean":"2024-04-13T05:15:13.169509+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:15:13.169509+0000","last_undegraded":"2024-04-13T05:15:13.169509+0000","last_fullsized":"2024-04-13T05:15:13.169509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169841+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:15:13.169841+0000","last_peered":"2024-04-13T05:15:13.169841+0000","last_clean":"2024-04-13T05:15:13.169841+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:15:13.169841+0000","last_undegraded":"2024-04-13T05:15:13.169841+0000","last_fullsized":"2024-04-13T05:15:13.169841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170586+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:15:13.170586+0000","last_peered":"2024-04-13T05:15:13.170586+0000","last_clean":"2024-04-13T05:15:13.170586+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:15:13.170586+0000","last_undegraded":"2024-04-13T05:15:13.170586+0000","last_fullsized":"2024-04-13T05:15:13.170586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189148+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:15:13.189148+0000","last_peered":"2024-04-13T05:15:13.189148+0000","last_clean":"2024-04-13T05:15:13.189148+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:15:13.189148+0000","last_undegraded":"2024-04-13T05:15:13.189148+0000","last_fullsized":"2024-04-13T05:15:13.189148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176955+0000","last_change":"2024-04-13T05:14:53.610337+0000","last_active":"2024-04-13T05:15:13.176955+0000","last_peered":"2024-04-13T05:15:13.176955+0000","last_clean":"2024-04-13T05:15:13.176955+0000","last_became_active":"2024-04-13T05:14:53.608706+0000","last_became_peered":"2024-04-13T05:14:53.608706+0000","last_unstale":"2024-04-13T05:15:13.176955+0000","last_undegraded":"2024-04-13T05:15:13.176955+0000","last_fullsized":"2024-04-13T05:15:13.176955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:31.307210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176140+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:15:13.176140+0000","last_peered":"2024-04-13T05:15:13.176140+0000","last_clean":"2024-04-13T05:15:13.176140+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:15:13.176140+0000","last_undegraded":"2024-04-13T05:15:13.176140+0000","last_fullsized":"2024-04-13T05:15:13.176140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.183601+0000","last_change":"2024-04-13T05:15:13.183601+0000","last_active":"2024-04-13T05:15:13.183601+0000","last_peered":"2024-04-13T05:15:13.183601+0000","last_clean":"2024-04-13T05:15:13.183601+0000","last_became_active":"2024-04-13T05:15:13.182961+0000","last_became_peered":"2024-04-13T05:15:13.182961+0000","last_unstale":"2024-04-13T05:15:13.183601+0000","last_undegraded":"2024-04-13T05:15:13.183601+0000","last_fullsized":"2024-04-13T05:15:13.183601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:54.504787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169714+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:15:13.169714+0000","last_peered":"2024-04-13T05:15:13.169714+0000","last_clean":"2024-04-13T05:15:13.169714+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:15:13.169714+0000","last_undegraded":"2024-04-13T05:15:13.169714+0000","last_fullsized":"2024-04-13T05:15:13.169714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166803+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:15:13.166803+0000","last_peered":"2024-04-13T05:15:13.166803+0000","last_clean":"2024-04-13T05:15:13.166803+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:15:13.166803+0000","last_undegraded":"2024-04-13T05:15:13.166803+0000","last_fullsized":"2024-04-13T05:15:13.166803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189751+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:15:13.189751+0000","last_peered":"2024-04-13T05:15:13.189751+0000","last_clean":"2024-04-13T05:15:13.189751+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:15:13.189751+0000","last_undegraded":"2024-04-13T05:15:13.189751+0000","last_fullsized":"2024-04-13T05:15:13.189751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.188874+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:15:13.188874+0000","last_peered":"2024-04-13T05:15:13.188874+0000","last_clean":"2024-04-13T05:15:13.188874+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:15:13.188874+0000","last_undegraded":"2024-04-13T05:15:13.188874+0000","last_fullsized":"2024-04-13T05:15:13.188874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176612+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:15:13.176612+0000","last_peered":"2024-04-13T05:15:13.176612+0000","last_clean":"2024-04-13T05:15:13.176612+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:15:13.176612+0000","last_undegraded":"2024-04-13T05:15:13.176612+0000","last_fullsized":"2024-04-13T05:15:13.176612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27142,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":37,"num_read_kb":52,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":100191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350999,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":507478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1118278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":511810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":502956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1095351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":503494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":527179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":503084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":504417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1650688,"data_stored":1094644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":24379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:15:26.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T05:15:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:26 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2871400672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:26 smithi057 ceph-mon[32331]: pgmap v389: 129 pgs: 129 active+clean; 603 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.3 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-13T05:15:26.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:26 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/317851043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:26.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:26 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2871400672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:26.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:26 smithi083 ceph-mon[35133]: pgmap v389: 129 pgs: 129 active+clean; 603 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.3 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-13T05:15:26.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:26 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/317851043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 -- 172.21.15.57:0/3300783825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518103970 msgr2=0x7f451810a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 --2- 172.21.15.57:0/3300783825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518103970 0x7f451810a290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4510009900 tx=0x7f451002bc30 comp rx=0 tx=0).stop 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 -- 172.21.15.57:0/3300783825 shutdown_connections 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 --2- 172.21.15.57:0/3300783825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518103970 0x7f451810a290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 --2- 172.21.15.57:0/3300783825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518102fb0 0x7f4518103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 --2- 172.21.15.57:0/3300783825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4518101e60 0x7f4518102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 -- 172.21.15.57:0/3300783825 >> 172.21.15.57:0/3300783825 conn(0x7f45180fd3d0 msgr2=0x7f45180ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:26.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 -- 172.21.15.57:0/3300783825 shutdown_connections 2024-04-13T05:15:26.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.813+0000 7f451de00640 1 -- 172.21.15.57:0/3300783825 wait complete. 2024-04-13T05:15:26.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.814+0000 7f451de00640 1 Processor -- start 2024-04-13T05:15:26.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.814+0000 7f451de00640 1 -- start start 2024-04-13T05:15:26.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 0x7f451819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4518102fb0 0x7f451819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518103970 0x7f45181a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451de00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f451810d9e0 con 0x7f4518101e60 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f451810d860 con 0x7f4518102fb0 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f451810db60 con 0x7f4518103970 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 0x7f451819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 0x7f451819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49064/0 (socket says 172.21.15.57:49064) 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451ca00640 1 -- 172.21.15.57:0/3259913048 learned_addr learned my addr 172.21.15.57:0/3259913048 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f450fe00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4518102fb0 0x7f451819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451d400640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518103970 0x7f45181a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451ca00640 1 -- 172.21.15.57:0/3259913048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518103970 msgr2=0x7f45181a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.815+0000 7f451ca00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518103970 0x7f45181a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451ca00640 1 -- 172.21.15.57:0/3259913048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4518102fb0 msgr2=0x7f451819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451ca00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4518102fb0 0x7f451819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451ca00640 1 -- 172.21.15.57:0/3259913048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45181a1e70 con 0x7f4518101e60 2024-04-13T05:15:26.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451d400640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518103970 0x7f45181a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451ca00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 0x7f451819a2a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f450800eab0 tx=0x7f450800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450800ce40 con 0x7f4518101e60 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4508011920 con 0x7f4518101e60 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508010a00 con 0x7f4518101e60 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45181a2160 con 0x7f4518101e60 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.816+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45181a2660 con 0x7f4518101e60 2024-04-13T05:15:26.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.817+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45181a2920 con 0x7f4518101e60 2024-04-13T05:15:26.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.820+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4508011ad0 con 0x7f4518101e60 2024-04-13T05:15:26.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.821+0000 7f4516000640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44fc0a3cc0 0x7f44fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.821+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f45080d0fe0 con 0x7f4518101e60 2024-04-13T05:15:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.822+0000 7f450fe00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44fc0a3cc0 0x7f44fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.823+0000 7f450fe00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44fc0a3cc0 0x7f44fc0a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4500005fd0 tx=0x7f4500005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:26.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:26.826+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4508016030 con 0x7f4518101e60 2024-04-13T05:15:26.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:26 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2871400672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:15:26.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:26 smithi189 ceph-mon[35531]: pgmap v389: 129 pgs: 129 active+clean; 603 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.3 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-13T05:15:26.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:26 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/317851043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:27.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.032+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45181a2af0 con 0x7f44fc0a3cc0 2024-04-13T05:15:27.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.062+0000 7f4516000640 1 -- 172.21.15.57:0/3259913048 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340499 (secure 0 0 0) 0x7f45181a2af0 con 0x7f44fc0a3cc0 2024-04-13T05:15:27.068 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:27.070 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:15:27.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44fc0a3cc0 msgr2=0x7f44fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:27.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44fc0a3cc0 0x7f44fc0a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4500005fd0 tx=0x7f4500005850 comp rx=0 tx=0).stop 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 msgr2=0x7f451819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 0x7f451819a2a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f450800eab0 tx=0x7f450800ef70 comp rx=0 tx=0).stop 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 shutdown_connections 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44fc0a3cc0 0x7f44fc0a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4518102fb0 0x7f451819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4518103970 0x7f45181a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 --2- 172.21.15.57:0/3259913048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4518101e60 0x7f451819a2a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.069+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 >> 172.21.15.57:0/3259913048 conn(0x7f45180fd3d0 msgr2=0x7f4518108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.070+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 shutdown_connections 2024-04-13T05:15:27.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:27.070+0000 7f451de00640 1 -- 172.21.15.57:0/3259913048 wait complete. 2024-04-13T05:15:27.095 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-04-13T05:15:25.603681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617510,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":163,"num_read_kb":161,"num_write":321,"num_write_kb":3006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363236,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578588,"statfs":{"total":1151940427776,"available":1151568474112,"internally_reserved":0,"allocated":14422016,"data_stored":7897634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24283,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":37,"num_read_kb":52,"num_write":93,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176750+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:15:13.176750+0000","last_peered":"2024-04-13T05:15:13.176750+0000","last_clean":"2024-04-13T05:15:13.176750+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:15:13.176750+0000","last_undegraded":"2024-04-13T05:15:13.176750+0000","last_fullsized":"2024-04-13T05:15:13.176750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170460+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:15:13.170460+0000","last_peered":"2024-04-13T05:15:13.170460+0000","last_clean":"2024-04-13T05:15:13.170460+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:15:13.170460+0000","last_undegraded":"2024-04-13T05:15:13.170460+0000","last_fullsized":"2024-04-13T05:15:13.170460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169853+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:15:13.169853+0000","last_peered":"2024-04-13T05:15:13.169853+0000","last_clean":"2024-04-13T05:15:13.169853+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:15:13.169853+0000","last_undegraded":"2024-04-13T05:15:13.169853+0000","last_fullsized":"2024-04-13T05:15:13.169853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.161261+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:15:13.161261+0000","last_peered":"2024-04-13T05:15:13.161261+0000","last_clean":"2024-04-13T05:15:13.161261+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:15:13.161261+0000","last_undegraded":"2024-04-13T05:15:13.161261+0000","last_fullsized":"2024-04-13T05:15:13.161261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160257+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:15:13.160257+0000","last_peered":"2024-04-13T05:15:13.160257+0000","last_clean":"2024-04-13T05:15:13.160257+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:15:13.160257+0000","last_undegraded":"2024-04-13T05:15:13.160257+0000","last_fullsized":"2024-04-13T05:15:13.160257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.192477+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:15:13.192477+0000","last_peered":"2024-04-13T05:15:13.192477+0000","last_clean":"2024-04-13T05:15:13.192477+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:15:13.192477+0000","last_undegraded":"2024-04-13T05:15:13.192477+0000","last_fullsized":"2024-04-13T05:15:13.192477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165612+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:15:13.165612+0000","last_peered":"2024-04-13T05:15:13.165612+0000","last_clean":"2024-04-13T05:15:13.165612+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:15:13.165612+0000","last_undegraded":"2024-04-13T05:15:13.165612+0000","last_fullsized":"2024-04-13T05:15:13.165612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166644+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:15:13.166644+0000","last_peered":"2024-04-13T05:15:13.166644+0000","last_clean":"2024-04-13T05:15:13.166644+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:15:13.166644+0000","last_undegraded":"2024-04-13T05:15:13.166644+0000","last_fullsized":"2024-04-13T05:15:13.166644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.191032+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:15:13.191032+0000","last_peered":"2024-04-13T05:15:13.191032+0000","last_clean":"2024-04-13T05:15:13.191032+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:15:13.191032+0000","last_undegraded":"2024-04-13T05:15:13.191032+0000","last_fullsized":"2024-04-13T05:15:13.191032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.194276+0000","last_change":"2024-04-13T05:14:53.598758+0000","last_active":"2024-04-13T05:15:13.194276+0000","last_peered":"2024-04-13T05:15:13.194276+0000","last_clean":"2024-04-13T05:15:13.194276+0000","last_became_active":"2024-04-13T05:14:53.598265+0000","last_became_peered":"2024-04-13T05:14:53.598265+0000","last_unstale":"2024-04-13T05:15:13.194276+0000","last_undegraded":"2024-04-13T05:15:13.194276+0000","last_fullsized":"2024-04-13T05:15:13.194276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:53:47.570217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166466+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:15:13.166466+0000","last_peered":"2024-04-13T05:15:13.166466+0000","last_clean":"2024-04-13T05:15:13.166466+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:15:13.166466+0000","last_undegraded":"2024-04-13T05:15:13.166466+0000","last_fullsized":"2024-04-13T05:15:13.166466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.161143+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:15:13.161143+0000","last_peered":"2024-04-13T05:15:13.161143+0000","last_clean":"2024-04-13T05:15:13.161143+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:15:13.161143+0000","last_undegraded":"2024-04-13T05:15:13.161143+0000","last_fullsized":"2024-04-13T05:15:13.161143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.188954+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:15:13.188954+0000","last_peered":"2024-04-13T05:15:13.188954+0000","last_clean":"2024-04-13T05:15:13.188954+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:15:13.188954+0000","last_undegraded":"2024-04-13T05:15:13.188954+0000","last_fullsized":"2024-04-13T05:15:13.188954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169887+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:15:13.169887+0000","last_peered":"2024-04-13T05:15:13.169887+0000","last_clean":"2024-04-13T05:15:13.169887+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:15:13.169887+0000","last_undegraded":"2024-04-13T05:15:13.169887+0000","last_fullsized":"2024-04-13T05:15:13.169887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170523+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:15:13.170523+0000","last_peered":"2024-04-13T05:15:13.170523+0000","last_clean":"2024-04-13T05:15:13.170523+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:15:13.170523+0000","last_undegraded":"2024-04-13T05:15:13.170523+0000","last_fullsized":"2024-04-13T05:15:13.170523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189835+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:15:13.189835+0000","last_peered":"2024-04-13T05:15:13.189835+0000","last_clean":"2024-04-13T05:15:13.189835+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:15:13.189835+0000","last_undegraded":"2024-04-13T05:15:13.189835+0000","last_fullsized":"2024-04-13T05:15:13.189835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176522+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:15:13.176522+0000","last_peered":"2024-04-13T05:15:13.176522+0000","last_clean":"2024-04-13T05:15:13.176522+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:15:13.176522+0000","last_undegraded":"2024-04-13T05:15:13.176522+0000","last_fullsized":"2024-04-13T05:15:13.176522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160928+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:15:13.160928+0000","last_peered":"2024-04-13T05:15:13.160928+0000","last_clean":"2024-04-13T05:15:13.160928+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:15:13.160928+0000","last_undegraded":"2024-04-13T05:15:13.160928+0000","last_fullsized":"2024-04-13T05:15:13.160928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176172+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:15:13.176172+0000","last_peered":"2024-04-13T05:15:13.176172+0000","last_clean":"2024-04-13T05:15:13.176172+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:15:13.176172+0000","last_undegraded":"2024-04-13T05:15:13.176172+0000","last_fullsized":"2024-04-13T05:15:13.176172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.179323+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:15:13.179323+0000","last_peered":"2024-04-13T05:15:13.179323+0000","last_clean":"2024-04-13T05:15:13.179323+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:15:13.179323+0000","last_undegraded":"2024-04-13T05:15:13.179323+0000","last_fullsized":"2024-04-13T05:15:13.179323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.316802+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:15:21.316802+0000","last_peered":"2024-04-13T05:15:21.316802+0000","last_clean":"2024-04-13T05:15:21.316802+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:15:21.316802+0000","last_undegraded":"2024-04-13T05:15:21.316802+0000","last_fullsized":"2024-04-13T05:15:21.316802+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165721+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:15:13.165721+0000","last_peered":"2024-04-13T05:15:13.165721+0000","last_clean":"2024-04-13T05:15:13.165721+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:15:13.165721+0000","last_undegraded":"2024-04-13T05:15:13.165721+0000","last_fullsized":"2024-04-13T05:15:13.165721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159313+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:15:13.159313+0000","last_peered":"2024-04-13T05:15:13.159313+0000","last_clean":"2024-04-13T05:15:13.159313+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:15:13.159313+0000","last_undegraded":"2024-04-13T05:15:13.159313+0000","last_fullsized":"2024-04-13T05:15:13.159313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166063+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:15:13.166063+0000","last_peered":"2024-04-13T05:15:13.166063+0000","last_clean":"2024-04-13T05:15:13.166063+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:15:13.166063+0000","last_undegraded":"2024-04-13T05:15:13.166063+0000","last_fullsized":"2024-04-13T05:15:13.166063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170842+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:15:13.170842+0000","last_peered":"2024-04-13T05:15:13.170842+0000","last_clean":"2024-04-13T05:15:13.170842+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:15:13.170842+0000","last_undegraded":"2024-04-13T05:15:13.170842+0000","last_fullsized":"2024-04-13T05:15:13.170842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.197015+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:15:13.197015+0000","last_peered":"2024-04-13T05:15:13.197015+0000","last_clean":"2024-04-13T05:15:13.197015+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:15:13.197015+0000","last_undegraded":"2024-04-13T05:15:13.197015+0000","last_fullsized":"2024-04-13T05:15:13.197015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.316372+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:15:21.316372+0000","last_peered":"2024-04-13T05:15:21.316372+0000","last_clean":"2024-04-13T05:15:21.316372+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:15:21.316372+0000","last_undegraded":"2024-04-13T05:15:21.316372+0000","last_fullsized":"2024-04-13T05:15:21.316372+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189867+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:15:13.189867+0000","last_peered":"2024-04-13T05:15:13.189867+0000","last_clean":"2024-04-13T05:15:13.189867+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:15:13.189867+0000","last_undegraded":"2024-04-13T05:15:13.189867+0000","last_fullsized":"2024-04-13T05:15:13.189867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159393+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:15:13.159393+0000","last_peered":"2024-04-13T05:15:13.159393+0000","last_clean":"2024-04-13T05:15:13.159393+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:15:13.159393+0000","last_undegraded":"2024-04-13T05:15:13.159393+0000","last_fullsized":"2024-04-13T05:15:13.159393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160899+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:15:13.160899+0000","last_peered":"2024-04-13T05:15:13.160899+0000","last_clean":"2024-04-13T05:15:13.160899+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:15:13.160899+0000","last_undegraded":"2024-04-13T05:15:13.160899+0000","last_fullsized":"2024-04-13T05:15:13.160899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170807+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:15:13.170807+0000","last_peered":"2024-04-13T05:15:13.170807+0000","last_clean":"2024-04-13T05:15:13.170807+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:15:13.170807+0000","last_undegraded":"2024-04-13T05:15:13.170807+0000","last_fullsized":"2024-04-13T05:15:13.170807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.175725+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:15:13.175725+0000","last_peered":"2024-04-13T05:15:13.175725+0000","last_clean":"2024-04-13T05:15:13.175725+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:15:13.175725+0000","last_undegraded":"2024-04-13T05:15:13.175725+0000","last_fullsized":"2024-04-13T05:15:13.175725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:22.329068+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:15:22.329068+0000","last_peered":"2024-04-13T05:15:22.329068+0000","last_clean":"2024-04-13T05:15:22.329068+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:15:22.329068+0000","last_undegraded":"2024-04-13T05:15:22.329068+0000","last_fullsized":"2024-04-13T05:15:22.329068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160716+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:15:13.160716+0000","last_peered":"2024-04-13T05:15:13.160716+0000","last_clean":"2024-04-13T05:15:13.160716+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:15:13.160716+0000","last_undegraded":"2024-04-13T05:15:13.160716+0000","last_fullsized":"2024-04-13T05:15:13.160716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189675+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:15:13.189675+0000","last_peered":"2024-04-13T05:15:13.189675+0000","last_clean":"2024-04-13T05:15:13.189675+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:15:13.189675+0000","last_undegraded":"2024-04-13T05:15:13.189675+0000","last_fullsized":"2024-04-13T05:15:13.189675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.315699+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:15:21.315699+0000","last_peered":"2024-04-13T05:15:21.315699+0000","last_clean":"2024-04-13T05:15:21.315699+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:15:21.315699+0000","last_undegraded":"2024-04-13T05:15:21.315699+0000","last_fullsized":"2024-04-13T05:15:21.315699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171189+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:15:13.171189+0000","last_peered":"2024-04-13T05:15:13.171189+0000","last_clean":"2024-04-13T05:15:13.171189+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:15:13.171189+0000","last_undegraded":"2024-04-13T05:15:13.171189+0000","last_fullsized":"2024-04-13T05:15:13.171189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176413+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:15:13.176413+0000","last_peered":"2024-04-13T05:15:13.176413+0000","last_clean":"2024-04-13T05:15:13.176413+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:15:13.176413+0000","last_undegraded":"2024-04-13T05:15:13.176413+0000","last_fullsized":"2024-04-13T05:15:13.176413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.317883+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:15:21.317883+0000","last_peered":"2024-04-13T05:15:21.317883+0000","last_clean":"2024-04-13T05:15:21.317883+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:15:21.317883+0000","last_undegraded":"2024-04-13T05:15:21.317883+0000","last_fullsized":"2024-04-13T05:15:21.317883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167032+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:15:13.167032+0000","last_peered":"2024-04-13T05:15:13.167032+0000","last_clean":"2024-04-13T05:15:13.167032+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:15:13.167032+0000","last_undegraded":"2024-04-13T05:15:13.167032+0000","last_fullsized":"2024-04-13T05:15:13.167032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170295+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:15:13.170295+0000","last_peered":"2024-04-13T05:15:13.170295+0000","last_clean":"2024-04-13T05:15:13.170295+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:15:13.170295+0000","last_undegraded":"2024-04-13T05:15:13.170295+0000","last_fullsized":"2024-04-13T05:15:13.170295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169472+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:15:13.169472+0000","last_peered":"2024-04-13T05:15:13.169472+0000","last_clean":"2024-04-13T05:15:13.169472+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:15:13.169472+0000","last_undegraded":"2024-04-13T05:15:13.169472+0000","last_fullsized":"2024-04-13T05:15:13.169472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.158792+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:15:13.158792+0000","last_peered":"2024-04-13T05:15:13.158792+0000","last_clean":"2024-04-13T05:15:13.158792+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:15:13.158792+0000","last_undegraded":"2024-04-13T05:15:13.158792+0000","last_fullsized":"2024-04-13T05:15:13.158792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166169+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:15:13.166169+0000","last_peered":"2024-04-13T05:15:13.166169+0000","last_clean":"2024-04-13T05:15:13.166169+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:15:13.166169+0000","last_undegraded":"2024-04-13T05:15:13.166169+0000","last_fullsized":"2024-04-13T05:15:13.166169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:23.332274+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:15:23.332274+0000","last_peered":"2024-04-13T05:15:23.332274+0000","last_clean":"2024-04-13T05:15:23.332274+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:15:23.332274+0000","last_undegraded":"2024-04-13T05:15:23.332274+0000","last_fullsized":"2024-04-13T05:15:23.332274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.190326+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:15:13.190326+0000","last_peered":"2024-04-13T05:15:13.190326+0000","last_clean":"2024-04-13T05:15:13.190326+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:15:13.190326+0000","last_undegraded":"2024-04-13T05:15:13.190326+0000","last_fullsized":"2024-04-13T05:15:13.190326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.158339+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:15:13.158339+0000","last_peered":"2024-04-13T05:15:13.158339+0000","last_clean":"2024-04-13T05:15:13.158339+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:15:13.158339+0000","last_undegraded":"2024-04-13T05:15:13.158339+0000","last_fullsized":"2024-04-13T05:15:13.158339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.297007+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:15:20.297007+0000","last_peered":"2024-04-13T05:15:20.297007+0000","last_clean":"2024-04-13T05:15:20.297007+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:15:20.297007+0000","last_undegraded":"2024-04-13T05:15:20.297007+0000","last_fullsized":"2024-04-13T05:15:20.297007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171204+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:15:13.171204+0000","last_peered":"2024-04-13T05:15:13.171204+0000","last_clean":"2024-04-13T05:15:13.171204+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:15:13.171204+0000","last_undegraded":"2024-04-13T05:15:13.171204+0000","last_fullsized":"2024-04-13T05:15:13.171204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.161182+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:15:13.161182+0000","last_peered":"2024-04-13T05:15:13.161182+0000","last_clean":"2024-04-13T05:15:13.161182+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:15:13.161182+0000","last_undegraded":"2024-04-13T05:15:13.161182+0000","last_fullsized":"2024-04-13T05:15:13.161182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.299233+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:15:20.299233+0000","last_peered":"2024-04-13T05:15:20.299233+0000","last_clean":"2024-04-13T05:15:20.299233+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:15:20.299233+0000","last_undegraded":"2024-04-13T05:15:20.299233+0000","last_fullsized":"2024-04-13T05:15:20.299233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.190373+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:15:13.190373+0000","last_peered":"2024-04-13T05:15:13.190373+0000","last_clean":"2024-04-13T05:15:13.190373+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:15:13.190373+0000","last_undegraded":"2024-04-13T05:15:13.190373+0000","last_fullsized":"2024-04-13T05:15:13.190373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.158810+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:15:13.158810+0000","last_peered":"2024-04-13T05:15:13.158810+0000","last_clean":"2024-04-13T05:15:13.158810+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:15:13.158810+0000","last_undegraded":"2024-04-13T05:15:13.158810+0000","last_fullsized":"2024-04-13T05:15:13.158810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'19","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:24.335910+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:15:24.335910+0000","last_peered":"2024-04-13T05:15:24.335910+0000","last_clean":"2024-04-13T05:15:24.335910+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:15:24.335910+0000","last_undegraded":"2024-04-13T05:15:24.335910+0000","last_fullsized":"2024-04-13T05:15:24.335910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":28,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.162496+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:15:13.162496+0000","last_peered":"2024-04-13T05:15:13.162496+0000","last_clean":"2024-04-13T05:15:13.162496+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:15:13.162496+0000","last_undegraded":"2024-04-13T05:15:13.162496+0000","last_fullsized":"2024-04-13T05:15:13.162496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171445+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:15:13.171445+0000","last_peered":"2024-04-13T05:15:13.171445+0000","last_clean":"2024-04-13T05:15:13.171445+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:15:13.171445+0000","last_undegraded":"2024-04-13T05:15:13.171445+0000","last_fullsized":"2024-04-13T05:15:13.171445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.317658+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:15:21.317658+0000","last_peered":"2024-04-13T05:15:21.317658+0000","last_clean":"2024-04-13T05:15:21.317658+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:15:21.317658+0000","last_undegraded":"2024-04-13T05:15:21.317658+0000","last_fullsized":"2024-04-13T05:15:21.317658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.187970+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:15:13.187970+0000","last_peered":"2024-04-13T05:15:13.187970+0000","last_clean":"2024-04-13T05:15:13.187970+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:15:13.187970+0000","last_undegraded":"2024-04-13T05:15:13.187970+0000","last_fullsized":"2024-04-13T05:15:13.187970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.175850+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:15:13.175850+0000","last_peered":"2024-04-13T05:15:13.175850+0000","last_clean":"2024-04-13T05:15:13.175850+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:15:13.175850+0000","last_undegraded":"2024-04-13T05:15:13.175850+0000","last_fullsized":"2024-04-13T05:15:13.175850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.301331+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:15:20.301331+0000","last_peered":"2024-04-13T05:15:20.301331+0000","last_clean":"2024-04-13T05:15:20.301331+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:15:20.301331+0000","last_undegraded":"2024-04-13T05:15:20.301331+0000","last_fullsized":"2024-04-13T05:15:20.301331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166585+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:15:13.166585+0000","last_peered":"2024-04-13T05:15:13.166585+0000","last_clean":"2024-04-13T05:15:13.166585+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:15:13.166585+0000","last_undegraded":"2024-04-13T05:15:13.166585+0000","last_fullsized":"2024-04-13T05:15:13.166585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159854+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:15:13.159854+0000","last_peered":"2024-04-13T05:15:13.159854+0000","last_clean":"2024-04-13T05:15:13.159854+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:15:13.159854+0000","last_undegraded":"2024-04-13T05:15:13.159854+0000","last_fullsized":"2024-04-13T05:15:13.159854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:23.336765+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:15:23.336765+0000","last_peered":"2024-04-13T05:15:23.336765+0000","last_clean":"2024-04-13T05:15:23.336765+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:15:23.336765+0000","last_undegraded":"2024-04-13T05:15:23.336765+0000","last_fullsized":"2024-04-13T05:15:23.336765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165684+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:15:13.165684+0000","last_peered":"2024-04-13T05:15:13.165684+0000","last_clean":"2024-04-13T05:15:13.165684+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:15:13.165684+0000","last_undegraded":"2024-04-13T05:15:13.165684+0000","last_fullsized":"2024-04-13T05:15:13.165684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189348+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:15:13.189348+0000","last_peered":"2024-04-13T05:15:13.189348+0000","last_clean":"2024-04-13T05:15:13.189348+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:15:13.189348+0000","last_undegraded":"2024-04-13T05:15:13.189348+0000","last_fullsized":"2024-04-13T05:15:13.189348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.316317+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:15:21.316317+0000","last_peered":"2024-04-13T05:15:21.316317+0000","last_clean":"2024-04-13T05:15:21.316317+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:15:21.316317+0000","last_undegraded":"2024-04-13T05:15:21.316317+0000","last_fullsized":"2024-04-13T05:15:21.316317+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171226+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:15:13.171226+0000","last_peered":"2024-04-13T05:15:13.171226+0000","last_clean":"2024-04-13T05:15:13.171226+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:15:13.171226+0000","last_undegraded":"2024-04-13T05:15:13.171226+0000","last_fullsized":"2024-04-13T05:15:13.171226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171040+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:15:13.171040+0000","last_peered":"2024-04-13T05:15:13.171040+0000","last_clean":"2024-04-13T05:15:13.171040+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:15:13.171040+0000","last_undegraded":"2024-04-13T05:15:13.171040+0000","last_fullsized":"2024-04-13T05:15:13.171040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.164121+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:15:13.164121+0000","last_peered":"2024-04-13T05:15:13.164121+0000","last_clean":"2024-04-13T05:15:13.164121+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:15:13.164121+0000","last_undegraded":"2024-04-13T05:15:13.164121+0000","last_fullsized":"2024-04-13T05:15:13.164121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.190714+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:15:13.190714+0000","last_peered":"2024-04-13T05:15:13.190714+0000","last_clean":"2024-04-13T05:15:13.190714+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:15:13.190714+0000","last_undegraded":"2024-04-13T05:15:13.190714+0000","last_fullsized":"2024-04-13T05:15:13.190714+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160616+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:15:13.160616+0000","last_peered":"2024-04-13T05:15:13.160616+0000","last_clean":"2024-04-13T05:15:13.160616+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:15:13.160616+0000","last_undegraded":"2024-04-13T05:15:13.160616+0000","last_fullsized":"2024-04-13T05:15:13.160616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160006+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:15:13.160006+0000","last_peered":"2024-04-13T05:15:13.160006+0000","last_clean":"2024-04-13T05:15:13.160006+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:15:13.160006+0000","last_undegraded":"2024-04-13T05:15:13.160006+0000","last_fullsized":"2024-04-13T05:15:13.160006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.192694+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:15:13.192694+0000","last_peered":"2024-04-13T05:15:13.192694+0000","last_clean":"2024-04-13T05:15:13.192694+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:15:13.192694+0000","last_undegraded":"2024-04-13T05:15:13.192694+0000","last_fullsized":"2024-04-13T05:15:13.192694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189011+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:15:13.189011+0000","last_peered":"2024-04-13T05:15:13.189011+0000","last_clean":"2024-04-13T05:15:13.189011+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:15:13.189011+0000","last_undegraded":"2024-04-13T05:15:13.189011+0000","last_fullsized":"2024-04-13T05:15:13.189011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167435+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:15:13.167435+0000","last_peered":"2024-04-13T05:15:13.167435+0000","last_clean":"2024-04-13T05:15:13.167435+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:15:13.167435+0000","last_undegraded":"2024-04-13T05:15:13.167435+0000","last_fullsized":"2024-04-13T05:15:13.167435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.193237+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:15:13.193237+0000","last_peered":"2024-04-13T05:15:13.193237+0000","last_clean":"2024-04-13T05:15:13.193237+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:15:13.193237+0000","last_undegraded":"2024-04-13T05:15:13.193237+0000","last_fullsized":"2024-04-13T05:15:13.193237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189463+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:15:13.189463+0000","last_peered":"2024-04-13T05:15:13.189463+0000","last_clean":"2024-04-13T05:15:13.189463+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:15:13.189463+0000","last_undegraded":"2024-04-13T05:15:13.189463+0000","last_fullsized":"2024-04-13T05:15:13.189463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169965+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:15:13.169965+0000","last_peered":"2024-04-13T05:15:13.169965+0000","last_clean":"2024-04-13T05:15:13.169965+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:15:13.169965+0000","last_undegraded":"2024-04-13T05:15:13.169965+0000","last_fullsized":"2024-04-13T05:15:13.169965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:22.330628+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:15:22.330628+0000","last_peered":"2024-04-13T05:15:22.330628+0000","last_clean":"2024-04-13T05:15:22.330628+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:15:22.330628+0000","last_undegraded":"2024-04-13T05:15:22.330628+0000","last_fullsized":"2024-04-13T05:15:22.330628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166232+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:15:13.166232+0000","last_peered":"2024-04-13T05:15:13.166232+0000","last_clean":"2024-04-13T05:15:13.166232+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:15:13.166232+0000","last_undegraded":"2024-04-13T05:15:13.166232+0000","last_fullsized":"2024-04-13T05:15:13.166232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167580+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:15:13.167580+0000","last_peered":"2024-04-13T05:15:13.167580+0000","last_clean":"2024-04-13T05:15:13.167580+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:15:13.167580+0000","last_undegraded":"2024-04-13T05:15:13.167580+0000","last_fullsized":"2024-04-13T05:15:13.167580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:22.325780+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:15:22.325780+0000","last_peered":"2024-04-13T05:15:22.325780+0000","last_clean":"2024-04-13T05:15:22.325780+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:15:22.325780+0000","last_undegraded":"2024-04-13T05:15:22.325780+0000","last_fullsized":"2024-04-13T05:15:22.325780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.165351+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:15:13.165351+0000","last_peered":"2024-04-13T05:15:13.165351+0000","last_clean":"2024-04-13T05:15:13.165351+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:15:13.165351+0000","last_undegraded":"2024-04-13T05:15:13.165351+0000","last_fullsized":"2024-04-13T05:15:13.165351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.196100+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:15:13.196100+0000","last_peered":"2024-04-13T05:15:13.196100+0000","last_clean":"2024-04-13T05:15:13.196100+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:15:13.196100+0000","last_undegraded":"2024-04-13T05:15:13.196100+0000","last_fullsized":"2024-04-13T05:15:13.196100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.186051+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:15:13.186051+0000","last_peered":"2024-04-13T05:15:13.186051+0000","last_clean":"2024-04-13T05:15:13.186051+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:15:13.186051+0000","last_undegraded":"2024-04-13T05:15:13.186051+0000","last_fullsized":"2024-04-13T05:15:13.186051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.313048+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:15:21.313048+0000","last_peered":"2024-04-13T05:15:21.313048+0000","last_clean":"2024-04-13T05:15:21.313048+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:15:21.313048+0000","last_undegraded":"2024-04-13T05:15:21.313048+0000","last_fullsized":"2024-04-13T05:15:21.313048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176016+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:15:13.176016+0000","last_peered":"2024-04-13T05:15:13.176016+0000","last_clean":"2024-04-13T05:15:13.176016+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:15:13.176016+0000","last_undegraded":"2024-04-13T05:15:13.176016+0000","last_fullsized":"2024-04-13T05:15:13.176016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.297045+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:15:20.297045+0000","last_peered":"2024-04-13T05:15:20.297045+0000","last_clean":"2024-04-13T05:15:20.297045+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:15:20.297045+0000","last_undegraded":"2024-04-13T05:15:20.297045+0000","last_fullsized":"2024-04-13T05:15:20.297045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189072+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:15:13.189072+0000","last_peered":"2024-04-13T05:15:13.189072+0000","last_clean":"2024-04-13T05:15:13.189072+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:15:13.189072+0000","last_undegraded":"2024-04-13T05:15:13.189072+0000","last_fullsized":"2024-04-13T05:15:13.189072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166839+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:15:13.166839+0000","last_peered":"2024-04-13T05:15:13.166839+0000","last_clean":"2024-04-13T05:15:13.166839+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:15:13.166839+0000","last_undegraded":"2024-04-13T05:15:13.166839+0000","last_fullsized":"2024-04-13T05:15:13.166839+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:23.646422+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:15:23.646422+0000","last_peered":"2024-04-13T05:15:23.646422+0000","last_clean":"2024-04-13T05:15:23.646422+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:15:23.646422+0000","last_undegraded":"2024-04-13T05:15:23.646422+0000","last_fullsized":"2024-04-13T05:15:23.646422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166503+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:15:13.166503+0000","last_peered":"2024-04-13T05:15:13.166503+0000","last_clean":"2024-04-13T05:15:13.166503+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:15:13.166503+0000","last_undegraded":"2024-04-13T05:15:13.166503+0000","last_fullsized":"2024-04-13T05:15:13.166503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.168261+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:15:13.168261+0000","last_peered":"2024-04-13T05:15:13.168261+0000","last_clean":"2024-04-13T05:15:13.168261+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:15:13.168261+0000","last_undegraded":"2024-04-13T05:15:13.168261+0000","last_fullsized":"2024-04-13T05:15:13.168261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.295825+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:15:20.295825+0000","last_peered":"2024-04-13T05:15:20.295825+0000","last_clean":"2024-04-13T05:15:20.295825+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:15:20.295825+0000","last_undegraded":"2024-04-13T05:15:20.295825+0000","last_fullsized":"2024-04-13T05:15:20.295825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170645+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:15:13.170645+0000","last_peered":"2024-04-13T05:15:13.170645+0000","last_clean":"2024-04-13T05:15:13.170645+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:15:13.170645+0000","last_undegraded":"2024-04-13T05:15:13.170645+0000","last_fullsized":"2024-04-13T05:15:13.170645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160036+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:15:13.160036+0000","last_peered":"2024-04-13T05:15:13.160036+0000","last_clean":"2024-04-13T05:15:13.160036+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:15:13.160036+0000","last_undegraded":"2024-04-13T05:15:13.160036+0000","last_fullsized":"2024-04-13T05:15:13.160036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.302015+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:15:20.302015+0000","last_peered":"2024-04-13T05:15:20.302015+0000","last_clean":"2024-04-13T05:15:20.302015+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:15:20.302015+0000","last_undegraded":"2024-04-13T05:15:20.302015+0000","last_fullsized":"2024-04-13T05:15:20.302015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159556+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:15:13.159556+0000","last_peered":"2024-04-13T05:15:13.159556+0000","last_clean":"2024-04-13T05:15:13.159556+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:15:13.159556+0000","last_undegraded":"2024-04-13T05:15:13.159556+0000","last_fullsized":"2024-04-13T05:15:13.159556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160029+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:15:13.160029+0000","last_peered":"2024-04-13T05:15:13.160029+0000","last_clean":"2024-04-13T05:15:13.160029+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:15:13.160029+0000","last_undegraded":"2024-04-13T05:15:13.160029+0000","last_fullsized":"2024-04-13T05:15:13.160029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.298929+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:15:20.298929+0000","last_peered":"2024-04-13T05:15:20.298929+0000","last_clean":"2024-04-13T05:15:20.298929+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:15:20.298929+0000","last_undegraded":"2024-04-13T05:15:20.298929+0000","last_fullsized":"2024-04-13T05:15:20.298929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176375+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:15:13.176375+0000","last_peered":"2024-04-13T05:15:13.176375+0000","last_clean":"2024-04-13T05:15:13.176375+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:15:13.176375+0000","last_undegraded":"2024-04-13T05:15:13.176375+0000","last_fullsized":"2024-04-13T05:15:13.176375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.160761+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:15:13.160761+0000","last_peered":"2024-04-13T05:15:13.160761+0000","last_clean":"2024-04-13T05:15:13.160761+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:15:13.160761+0000","last_undegraded":"2024-04-13T05:15:13.160761+0000","last_fullsized":"2024-04-13T05:15:13.160761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:15.426478+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:15:15.426478+0000","last_peered":"2024-04-13T05:15:15.426478+0000","last_clean":"2024-04-13T05:15:15.426478+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:15:15.426478+0000","last_undegraded":"2024-04-13T05:15:15.426478+0000","last_fullsized":"2024-04-13T05:15:15.426478+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167755+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:15:13.167755+0000","last_peered":"2024-04-13T05:15:13.167755+0000","last_clean":"2024-04-13T05:15:13.167755+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:15:13.167755+0000","last_undegraded":"2024-04-13T05:15:13.167755+0000","last_fullsized":"2024-04-13T05:15:13.167755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.159818+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:15:13.159818+0000","last_peered":"2024-04-13T05:15:13.159818+0000","last_clean":"2024-04-13T05:15:13.159818+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:15:13.159818+0000","last_undegraded":"2024-04-13T05:15:13.159818+0000","last_fullsized":"2024-04-13T05:15:13.159818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:20.302063+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:15:20.302063+0000","last_peered":"2024-04-13T05:15:20.302063+0000","last_clean":"2024-04-13T05:15:20.302063+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:15:20.302063+0000","last_undegraded":"2024-04-13T05:15:20.302063+0000","last_fullsized":"2024-04-13T05:15:20.302063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167619+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:15:13.167619+0000","last_peered":"2024-04-13T05:15:13.167619+0000","last_clean":"2024-04-13T05:15:13.167619+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:15:13.167619+0000","last_undegraded":"2024-04-13T05:15:13.167619+0000","last_fullsized":"2024-04-13T05:15:13.167619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.187605+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:15:13.187605+0000","last_peered":"2024-04-13T05:15:13.187605+0000","last_clean":"2024-04-13T05:15:13.187605+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:15:13.187605+0000","last_undegraded":"2024-04-13T05:15:13.187605+0000","last_fullsized":"2024-04-13T05:15:13.187605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:21.317877+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:15:21.317877+0000","last_peered":"2024-04-13T05:15:21.317877+0000","last_clean":"2024-04-13T05:15:21.317877+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:15:21.317877+0000","last_undegraded":"2024-04-13T05:15:21.317877+0000","last_fullsized":"2024-04-13T05:15:21.317877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166807+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:15:13.166807+0000","last_peered":"2024-04-13T05:15:13.166807+0000","last_clean":"2024-04-13T05:15:13.166807+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:15:13.166807+0000","last_undegraded":"2024-04-13T05:15:13.166807+0000","last_fullsized":"2024-04-13T05:15:13.166807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170195+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:15:13.170195+0000","last_peered":"2024-04-13T05:15:13.170195+0000","last_clean":"2024-04-13T05:15:13.170195+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:15:13.170195+0000","last_undegraded":"2024-04-13T05:15:13.170195+0000","last_fullsized":"2024-04-13T05:15:13.170195+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.171150+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:15:13.171150+0000","last_peered":"2024-04-13T05:15:13.171150+0000","last_clean":"2024-04-13T05:15:13.171150+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:15:13.171150+0000","last_undegraded":"2024-04-13T05:15:13.171150+0000","last_fullsized":"2024-04-13T05:15:13.171150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166200+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:15:13.166200+0000","last_peered":"2024-04-13T05:15:13.166200+0000","last_clean":"2024-04-13T05:15:13.166200+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:15:13.166200+0000","last_undegraded":"2024-04-13T05:15:13.166200+0000","last_fullsized":"2024-04-13T05:15:13.166200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166692+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:15:13.166692+0000","last_peered":"2024-04-13T05:15:13.166692+0000","last_clean":"2024-04-13T05:15:13.166692+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:15:13.166692+0000","last_undegraded":"2024-04-13T05:15:13.166692+0000","last_fullsized":"2024-04-13T05:15:13.166692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189354+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:15:13.189354+0000","last_peered":"2024-04-13T05:15:13.189354+0000","last_clean":"2024-04-13T05:15:13.189354+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:15:13.189354+0000","last_undegraded":"2024-04-13T05:15:13.189354+0000","last_fullsized":"2024-04-13T05:15:13.189354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.167272+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:15:13.167272+0000","last_peered":"2024-04-13T05:15:13.167272+0000","last_clean":"2024-04-13T05:15:13.167272+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:15:13.167272+0000","last_undegraded":"2024-04-13T05:15:13.167272+0000","last_fullsized":"2024-04-13T05:15:13.167272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176515+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:15:13.176515+0000","last_peered":"2024-04-13T05:15:13.176515+0000","last_clean":"2024-04-13T05:15:13.176515+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:15:13.176515+0000","last_undegraded":"2024-04-13T05:15:13.176515+0000","last_fullsized":"2024-04-13T05:15:13.176515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169509+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:15:13.169509+0000","last_peered":"2024-04-13T05:15:13.169509+0000","last_clean":"2024-04-13T05:15:13.169509+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:15:13.169509+0000","last_undegraded":"2024-04-13T05:15:13.169509+0000","last_fullsized":"2024-04-13T05:15:13.169509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169841+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:15:13.169841+0000","last_peered":"2024-04-13T05:15:13.169841+0000","last_clean":"2024-04-13T05:15:13.169841+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:15:13.169841+0000","last_undegraded":"2024-04-13T05:15:13.169841+0000","last_fullsized":"2024-04-13T05:15:13.169841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.170586+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:15:13.170586+0000","last_peered":"2024-04-13T05:15:13.170586+0000","last_clean":"2024-04-13T05:15:13.170586+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:15:13.170586+0000","last_undegraded":"2024-04-13T05:15:13.170586+0000","last_fullsized":"2024-04-13T05:15:13.170586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189148+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:15:13.189148+0000","last_peered":"2024-04-13T05:15:13.189148+0000","last_clean":"2024-04-13T05:15:13.189148+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:15:13.189148+0000","last_undegraded":"2024-04-13T05:15:13.189148+0000","last_fullsized":"2024-04-13T05:15:13.189148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176955+0000","last_change":"2024-04-13T05:14:53.610337+0000","last_active":"2024-04-13T05:15:13.176955+0000","last_peered":"2024-04-13T05:15:13.176955+0000","last_clean":"2024-04-13T05:15:13.176955+0000","last_became_active":"2024-04-13T05:14:53.608706+0000","last_became_peered":"2024-04-13T05:14:53.608706+0000","last_unstale":"2024-04-13T05:15:13.176955+0000","last_undegraded":"2024-04-13T05:15:13.176955+0000","last_fullsized":"2024-04-13T05:15:13.176955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:31.307210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176140+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:15:13.176140+0000","last_peered":"2024-04-13T05:15:13.176140+0000","last_clean":"2024-04-13T05:15:13.176140+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:15:13.176140+0000","last_undegraded":"2024-04-13T05:15:13.176140+0000","last_fullsized":"2024-04-13T05:15:13.176140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.183601+0000","last_change":"2024-04-13T05:15:13.183601+0000","last_active":"2024-04-13T05:15:13.183601+0000","last_peered":"2024-04-13T05:15:13.183601+0000","last_clean":"2024-04-13T05:15:13.183601+0000","last_became_active":"2024-04-13T05:15:13.182961+0000","last_became_peered":"2024-04-13T05:15:13.182961+0000","last_unstale":"2024-04-13T05:15:13.183601+0000","last_undegraded":"2024-04-13T05:15:13.183601+0000","last_fullsized":"2024-04-13T05:15:13.183601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:54.504787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.169714+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:15:13.169714+0000","last_peered":"2024-04-13T05:15:13.169714+0000","last_clean":"2024-04-13T05:15:13.169714+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:15:13.169714+0000","last_undegraded":"2024-04-13T05:15:13.169714+0000","last_fullsized":"2024-04-13T05:15:13.169714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.166803+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:15:13.166803+0000","last_peered":"2024-04-13T05:15:13.166803+0000","last_clean":"2024-04-13T05:15:13.166803+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:15:13.166803+0000","last_undegraded":"2024-04-13T05:15:13.166803+0000","last_fullsized":"2024-04-13T05:15:13.166803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.189751+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:15:13.189751+0000","last_peered":"2024-04-13T05:15:13.189751+0000","last_clean":"2024-04-13T05:15:13.189751+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:15:13.189751+0000","last_undegraded":"2024-04-13T05:15:13.189751+0000","last_fullsized":"2024-04-13T05:15:13.189751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.188874+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:15:13.188874+0000","last_peered":"2024-04-13T05:15:13.188874+0000","last_clean":"2024-04-13T05:15:13.188874+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:15:13.188874+0000","last_undegraded":"2024-04-13T05:15:13.188874+0000","last_fullsized":"2024-04-13T05:15:13.188874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-13T05:15:13.176612+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:15:13.176612+0000","last_peered":"2024-04-13T05:15:13.176612+0000","last_clean":"2024-04-13T05:15:13.176612+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:15:13.176612+0000","last_undegraded":"2024-04-13T05:15:13.176612+0000","last_fullsized":"2024-04-13T05:15:13.176612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27142,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":37,"num_read_kb":52,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":100191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350999,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":507478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1118278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":511810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":502956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1095351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":503494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":527179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":503084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":504417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1650688,"data_stored":1094644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":24379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:15:27.099 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-13T05:15:27.100 INFO:tasks.cephadm:Setting up client nodes... 2024-04-13T05:15:27.100 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-13T05:15:27.622 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:27.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:27 smithi083 ceph-mon[35133]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:15:27.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:27 smithi083 ceph-mon[35133]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:15:27.769 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:27 smithi057 ceph-mon[32331]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:15:27.769 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:27 smithi057 ceph-mon[32331]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:15:27.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:27 smithi189 ceph-mon[35531]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:15:27.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:27 smithi189 ceph-mon[35531]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:15:28.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:28 smithi083 ceph-mon[35133]: pgmap v390: 129 pgs: 129 active+clean; 603 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-13T05:15:28.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:28 smithi057 ceph-mon[32331]: pgmap v390: 129 pgs: 129 active+clean; 603 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-13T05:15:28.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:28 smithi189 ceph-mon[35531]: pgmap v390: 129 pgs: 129 active+clean; 603 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 -- 172.21.15.57:0/336639114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c04075510 msgr2=0x7f8c04075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 --2- 172.21.15.57:0/336639114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c04075510 0x7f8c04075990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009990 tx=0x7f8bf4009ca0 comp rx=0 tx=0).stop 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 -- 172.21.15.57:0/336639114 shutdown_connections 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 --2- 172.21.15.57:0/336639114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c0410ef50 0x7f8c04111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 --2- 172.21.15.57:0/336639114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c04076ae0 0x7f8c04074f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 --2- 172.21.15.57:0/336639114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c04075510 0x7f8c04075990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 -- 172.21.15.57:0/336639114 >> 172.21.15.57:0/336639114 conn(0x7f8c040fdef0 msgr2=0x7f8c04100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.853+0000 7f8c0b200700 1 -- 172.21.15.57:0/336639114 shutdown_connections 2024-04-13T05:15:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.854+0000 7f8c0b200700 1 -- 172.21.15.57:0/336639114 wait complete. 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 Processor -- start 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 -- start start 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c04075510 0x7f8c0419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c04076ae0 0x7f8c0419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 0x7f8c041a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04114a50 con 0x7f8c0410ef50 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041148d0 con 0x7f8c04076ae0 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.855+0000 7f8c0b200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04114bd0 con 0x7f8c04075510 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 0x7f8c041a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 0x7f8c041a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49084/0 (socket says 172.21.15.57:49084) 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 -- 172.21.15.57:0/3621108150 learned_addr learned my addr 172.21.15.57:0/3621108150 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:29.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c09400700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c04076ae0 0x7f8c0419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c09e00700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c04075510 0x7f8c0419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 -- 172.21.15.57:0/3621108150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c04075510 msgr2=0x7f8c0419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c04075510 0x7f8c0419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 -- 172.21.15.57:0/3621108150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c04076ae0 msgr2=0x7f8c0419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c04076ae0 0x7f8c0419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c0a800700 1 -- 172.21.15.57:0/3621108150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041a6ba0 con 0x7f8c0410ef50 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.856+0000 7f8c09e00700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c04075510 0x7f8c0419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.857+0000 7f8c0a800700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 0x7f8c041a6500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007da0 tx=0x7f8c0000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:29.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.857+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00016020 con 0x7f8c0410ef50 2024-04-13T05:15:29.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.857+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c0000ad00 con 0x7f8c0410ef50 2024-04-13T05:15:29.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.857+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c000115e0 con 0x7f8c0410ef50 2024-04-13T05:15:29.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.857+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c041a6e90 con 0x7f8c0410ef50 2024-04-13T05:15:29.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.857+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041a7390 con 0x7f8c0410ef50 2024-04-13T05:15:29.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.858+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c04064110 con 0x7f8c0410ef50 2024-04-13T05:15:29.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.861+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8c00019020 con 0x7f8c0410ef50 2024-04-13T05:15:29.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.862+0000 7f8bfea00700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8bec0a3e00 0x7f8bec0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:29.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.862+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8c000d0720 con 0x7f8c0410ef50 2024-04-13T05:15:29.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.862+0000 7f8c09e00700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8bec0a3e00 0x7f8bec0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:29.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.863+0000 7f8c09e00700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8bec0a3e00 0x7f8bec0a62c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005fd0 tx=0x7f8bf8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:29.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:29.867+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c00020020 con 0x7f8c0410ef50 2024-04-13T05:15:30.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.197+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8c0402e070 con 0x7f8c0410ef50 2024-04-13T05:15:30.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.202+0000 7f8bfea00700 1 -- 172.21.15.57:0/3621108150 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f8c0009ccf0 con 0x7f8c0410ef50 2024-04-13T05:15:30.205 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2024-04-13T05:15:30.205 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDyFBpmkKHpCxAAxFhDBRi7X/pTdvQej784Iw== 2024-04-13T05:15:30.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8bec0a3e00 msgr2=0x7f8bec0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:30.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8bec0a3e00 0x7f8bec0a62c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005fd0 tx=0x7f8bf8005e30 comp rx=0 tx=0).stop 2024-04-13T05:15:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 msgr2=0x7f8c041a6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 0x7f8c041a6500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007da0 tx=0x7f8c0000a340 comp rx=0 tx=0).stop 2024-04-13T05:15:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 shutdown_connections 2024-04-13T05:15:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c04075510 0x7f8c0419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8bec0a3e00 0x7f8bec0a62c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:30.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0410ef50 0x7f8c041a6500 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 --2- 172.21.15.57:0/3621108150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c04076ae0 0x7f8c0419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 >> 172.21.15.57:0/3621108150 conn(0x7f8c040fdef0 msgr2=0x7f8c041022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 shutdown_connections 2024-04-13T05:15:30.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:30.206+0000 7f8c0b200700 1 -- 172.21.15.57:0/3621108150 wait complete. 2024-04-13T05:15:31.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:30 smithi083 ceph-mon[35133]: pgmap v391: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-13T05:15:31.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:30 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3621108150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-13T05:15:31.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:30 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3621108150' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-13T05:15:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:30 smithi057 ceph-mon[32331]: pgmap v391: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-13T05:15:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:30 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3621108150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-13T05:15:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:30 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3621108150' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-13T05:15:31.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:30 smithi189 ceph-mon[35531]: pgmap v391: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-13T05:15:31.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:30 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3621108150' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-13T05:15:31.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:30 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3621108150' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-13T05:15:31.146 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:15:31.146 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-13T05:15:31.146 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-13T05:15:31.309 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-13T05:15:31.309 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-13T05:15:31.309 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph mgr dump --format=json 2024-04-13T05:15:31.821 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:33.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:32 smithi083 ceph-mon[35133]: pgmap v392: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-13T05:15:33.018 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:32 smithi057 ceph-mon[32331]: pgmap v392: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-13T05:15:33.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:32 smithi189 ceph-mon[35531]: pgmap v392: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.137+0000 7f2751400700 1 -- 172.21.15.57:0/203964141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109d30 msgr2=0x7f274c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.137+0000 7f2751400700 1 --2- 172.21.15.57:0/203964141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109d30 0x7f274c10dd30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f27340099c0 tx=0x7f2734009cd0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 -- 172.21.15.57:0/203964141 shutdown_connections 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 --2- 172.21.15.57:0/203964141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109380 0x7f274c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 --2- 172.21.15.57:0/203964141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 0x7f274c115fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 --2- 172.21.15.57:0/203964141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109d30 0x7f274c10dd30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 -- 172.21.15.57:0/203964141 >> 172.21.15.57:0/203964141 conn(0x7f274c076280 msgr2=0x7f274c0786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 -- 172.21.15.57:0/203964141 shutdown_connections 2024-04-13T05:15:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.138+0000 7f2751400700 1 -- 172.21.15.57:0/203964141 wait complete. 2024-04-13T05:15:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.139+0000 7f2751400700 1 Processor -- start 2024-04-13T05:15:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.139+0000 7f2751400700 1 -- start start 2024-04-13T05:15:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.139+0000 7f2751400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 0x7f274c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f2751400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109d30 0x7f274c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f2751400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 0x7f274c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f2751400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f274c119280 con 0x7f274c109380 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f2751400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f274c119100 con 0x7f274c10e410 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f2751400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f274c119400 con 0x7f274c109d30 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f274a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 0x7f274c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f274a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 0x7f274c1a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49104/0 (socket says 172.21.15.57:49104) 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f274a800700 1 -- 172.21.15.57:0/4124210072 learned_addr learned my addr 172.21.15.57:0/4124210072 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f2743e00700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109d30 0x7f274c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f274b200700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 0x7f274c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f274a800700 1 -- 172.21.15.57:0/4124210072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109d30 msgr2=0x7f274c1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.140+0000 7f274a800700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109d30 0x7f274c1a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f274a800700 1 -- 172.21.15.57:0/4124210072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 msgr2=0x7f274c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f274a800700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 0x7f274c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f274a800700 1 -- 172.21.15.57:0/4124210072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c19fd40 con 0x7f274c109380 2024-04-13T05:15:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f2743e00700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109d30 0x7f274c1a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:34.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f274b200700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 0x7f274c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f274a800700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 0x7f274c1a53d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f273c00d950 tx=0x7f273c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c016070 con 0x7f274c109380 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f273c010460 con 0x7f274c109380 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c1a0030 con 0x7f274c109380 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c00f620 con 0x7f274c109380 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.141+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1ac2a0 con 0x7f274c109380 2024-04-13T05:15:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.143+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c0697a0 con 0x7f274c109380 2024-04-13T05:15:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.145+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f273c019070 con 0x7f274c109380 2024-04-13T05:15:34.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.147+0000 7f2743400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27300a3d50 0x7f27300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:34.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.147+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f273c0d08b0 con 0x7f274c109380 2024-04-13T05:15:34.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.147+0000 7f2743e00700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27300a3d50 0x7f27300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:34.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.148+0000 7f2743e00700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27300a3d50 0x7f27300a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:34.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.150+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f273c09cd70 con 0x7f274c109380 2024-04-13T05:15:34.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.451+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f274c02dfa0 con 0x7f274c109380 2024-04-13T05:15:34.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.466+0000 7f2743400700 1 -- 172.21.15.57:0/4124210072 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7f273c09c670 con 0x7f274c109380 2024-04-13T05:15:34.473 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:34.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.474+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27300a3d50 msgr2=0x7f27300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:34.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.474+0000 7f2751400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27300a3d50 0x7f27300a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734005ab0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.474+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 msgr2=0x7f274c1a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.474+0000 7f2751400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 0x7f274c1a53d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f273c00d950 tx=0x7f273c00dd10 comp rx=0 tx=0).stop 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 shutdown_connections 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27300a3d50 0x7f27300a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f274c10e410 0x7f274c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f274c109380 0x7f274c1a53d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 --2- 172.21.15.57:0/4124210072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f274c109d30 0x7f274c1a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 >> 172.21.15.57:0/4124210072 conn(0x7f274c076280 msgr2=0x7f274c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 shutdown_connections 2024-04-13T05:15:34.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:34.475+0000 7f2751400700 1 -- 172.21.15.57:0/4124210072 wait complete. 2024-04-13T05:15:35.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:34 smithi083 ceph-mon[35133]: pgmap v393: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-13T05:15:35.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:34 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4124210072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-13T05:15:35.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:34 smithi057 ceph-mon[32331]: pgmap v393: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-13T05:15:35.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:34 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4124210072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-13T05:15:35.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:34 smithi189 ceph-mon[35531]: pgmap v393: 129 pgs: 129 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-04-13T05:15:35.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:34 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4124210072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-13T05:15:35.315 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":3239836246},{"type":"v1","addr":"172.21.15.57:6801","nonce":3239836246}]},"active_addr":"172.21.15.57:6801/3239836246","active_change":"2024-04-13T05:05:11.407901+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":43445990}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2869899963}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2799412934}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":4167373040}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":3445686642}]}]} 2024-04-13T05:15:35.320 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-13T05:15:35.320 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-13T05:15:35.320 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd dump --format=json 2024-04-13T05:15:35.836 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:37.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:36 smithi083 ceph-mon[35133]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-13T05:15:37.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:36 smithi057 ceph-mon[32331]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-13T05:15:37.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:36 smithi189 ceph-mon[35531]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-13T05:15:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.177+0000 7f99afc00700 1 -- 172.21.15.57:0/2399705562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a81034c0 msgr2=0x7f99a810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.177+0000 7f99afc00700 1 --2- 172.21.15.57:0/2399705562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a81034c0 0x7f99a810f920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b0a0 tx=0x7f99a000b3b0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.177+0000 7f99afc00700 1 -- 172.21.15.57:0/2399705562 shutdown_connections 2024-04-13T05:15:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.177+0000 7f99afc00700 1 --2- 172.21.15.57:0/2399705562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a81034c0 0x7f99a810f920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.178+0000 7f99afc00700 1 --2- 172.21.15.57:0/2399705562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a8108ac0 0x7f99a8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.178+0000 7f99afc00700 1 --2- 172.21.15.57:0/2399705562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a8102b00 0x7f99a8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.178+0000 7f99afc00700 1 -- 172.21.15.57:0/2399705562 >> 172.21.15.57:0/2399705562 conn(0x7f99a8079c10 msgr2=0x7f99a807a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.178+0000 7f99afc00700 1 -- 172.21.15.57:0/2399705562 shutdown_connections 2024-04-13T05:15:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.178+0000 7f99afc00700 1 -- 172.21.15.57:0/2399705562 wait complete. 2024-04-13T05:15:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.179+0000 7f99afc00700 1 Processor -- start 2024-04-13T05:15:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.179+0000 7f99afc00700 1 -- start start 2024-04-13T05:15:38.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99afc00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 0x7f99a8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:38.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99afc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a81034c0 0x7f99a81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99afc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a8108ac0 0x7f99a819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99afc00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112ca0 con 0x7f99a8102b00 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99afc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112b20 con 0x7f99a81034c0 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99afc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112e20 con 0x7f99a8108ac0 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.180+0000 7f99ad200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 0x7f99a8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 0x7f99a8199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51878/0 (socket says 172.21.15.57:51878) 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 -- 172.21.15.57:0/2177856491 learned_addr learned my addr 172.21.15.57:0/2177856491 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99a7e00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a81034c0 0x7f99a81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99adc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a8108ac0 0x7f99a819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 -- 172.21.15.57:0/2177856491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a8108ac0 msgr2=0x7f99a819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a8108ac0 0x7f99a819da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 -- 172.21.15.57:0/2177856491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a81034c0 msgr2=0x7f99a81996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a81034c0 0x7f99a81996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.181+0000 7f99ad200700 1 -- 172.21.15.57:0/2177856491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a819e180 con 0x7f99a8102b00 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.182+0000 7f99ad200700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 0x7f99a8199170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f999c00d950 tx=0x7f999c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.182+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c016070 con 0x7f99a8102b00 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.182+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a819e470 con 0x7f99a8102b00 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.182+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f999c009df0 con 0x7f99a8102b00 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.182+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a5cf0 con 0x7f99a8102b00 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.182+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c020060 con 0x7f99a8102b00 2024-04-13T05:15:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.184+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a80697a0 con 0x7f99a8102b00 2024-04-13T05:15:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.186+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f999c019070 con 0x7f99a8102b00 2024-04-13T05:15:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.187+0000 7f99a5600700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f99900a3cd0 0x7f99900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.187+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f999c0d0810 con 0x7f99a8102b00 2024-04-13T05:15:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.187+0000 7f99a7e00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f99900a3cd0 0x7f99900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.188+0000 7f99a7e00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f99900a3cd0 0x7f99900a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99a807a2f0 tx=0x7f999400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:38.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.192+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f999c09ccd0 con 0x7f99a8102b00 2024-04-13T05:15:38.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.427+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99a802dfa0 con 0x7f99a8102b00 2024-04-13T05:15:38.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.429+0000 7f99a5600700 1 -- 172.21.15.57:0/2177856491 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24583 (secure 0 0 0) 0x7f999c09c5d0 con 0x7f99a8102b00 2024-04-13T05:15:38.431 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:38.432 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:15:13.150673+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:38.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.432+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f99900a3cd0 msgr2=0x7f99900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:38.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.432+0000 7f99afc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f99900a3cd0 0x7f99900a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99a807a2f0 tx=0x7f999400a300 comp rx=0 tx=0).stop 2024-04-13T05:15:38.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.432+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 msgr2=0x7f99a8199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.432+0000 7f99afc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 0x7f99a8199170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f999c00d950 tx=0x7f999c00dc60 comp rx=0 tx=0).stop 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 shutdown_connections 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99a8108ac0 0x7f99a819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f99900a3cd0 0x7f99900a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99a8102b00 0x7f99a8199170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 --2- 172.21.15.57:0/2177856491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99a81034c0 0x7f99a81996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 >> 172.21.15.57:0/2177856491 conn(0x7f99a8079c10 msgr2=0x7f99a8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 shutdown_connections 2024-04-13T05:15:38.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:38.433+0000 7f99afc00700 1 -- 172.21.15.57:0/2177856491 wait complete. 2024-04-13T05:15:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:38 smithi083 ceph-mon[35133]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-13T05:15:39.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:38 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2177856491' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:39.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:38 smithi057 ceph-mon[32331]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-13T05:15:39.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:38 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2177856491' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:39.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:38 smithi189 ceph-mon[35531]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-04-13T05:15:39.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:38 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2177856491' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:39.279 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-13T05:15:39.279 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd dump --format=json 2024-04-13T05:15:39.755 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:40.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:40 smithi057 ceph-mon[32331]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-04-13T05:15:41.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:40 smithi083 ceph-mon[35133]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-04-13T05:15:41.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:40 smithi189 ceph-mon[35531]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 -- 172.21.15.57:0/228239585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c105900 msgr2=0x7f4c1c109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 --2- 172.21.15.57:0/228239585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c105900 0x7f4c1c109950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4c080099c0 tx=0x7f4c08009cd0 comp rx=0 tx=0).stop 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 -- 172.21.15.57:0/228239585 shutdown_connections 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 --2- 172.21.15.57:0/228239585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c1c104f50 0x7f4c1c105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 --2- 172.21.15.57:0/228239585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c10a030 0x7f4c1c111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 --2- 172.21.15.57:0/228239585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c105900 0x7f4c1c109950 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 -- 172.21.15.57:0/228239585 >> 172.21.15.57:0/228239585 conn(0x7f4c1c1007e0 msgr2=0x7f4c1c102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:41.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 -- 172.21.15.57:0/228239585 shutdown_connections 2024-04-13T05:15:41.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.987+0000 7f4c22400700 1 -- 172.21.15.57:0/228239585 wait complete. 2024-04-13T05:15:41.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.988+0000 7f4c22400700 1 Processor -- start 2024-04-13T05:15:41.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.988+0000 7f4c22400700 1 -- start start 2024-04-13T05:15:41.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.988+0000 7f4c22400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 0x7f4c1c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:41.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.988+0000 7f4c22400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c105900 0x7f4c1c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:41.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c22400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c1c10a030 0x7f4c1c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c22400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c114ec0 con 0x7f4c1c104f50 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c22400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c114d40 con 0x7f4c1c105900 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c22400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c115040 con 0x7f4c1c10a030 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 0x7f4c1c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 0x7f4c1c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51894/0 (socket says 172.21.15.57:51894) 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 -- 172.21.15.57:0/775537473 learned_addr learned my addr 172.21.15.57:0/775537473 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1be00700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c1c10a030 0x7f4c1c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1aa00700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c105900 0x7f4c1c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 -- 172.21.15.57:0/775537473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c1c10a030 msgr2=0x7f4c1c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c1c10a030 0x7f4c1c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 -- 172.21.15.57:0/775537473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c105900 msgr2=0x7f4c1c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c105900 0x7f4c1c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1b400700 1 -- 172.21.15.57:0/775537473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c1a03c0 con 0x7f4c1c104f50 2024-04-13T05:15:41.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.989+0000 7f4c1aa00700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c105900 0x7f4c1c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:15:41.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.990+0000 7f4c1b400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 0x7f4c1c19b3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00d980 tx=0x7f4c0c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:41.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.990+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c017070 con 0x7f4c1c104f50 2024-04-13T05:15:41.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.990+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1c1a06b0 con 0x7f4c1c104f50 2024-04-13T05:15:41.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.990+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c0c004830 con 0x7f4c1c104f50 2024-04-13T05:15:41.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.990+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c005440 con 0x7f4c1c104f50 2024-04-13T05:15:41.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.990+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1c1a7f30 con 0x7f4c1c104f50 2024-04-13T05:15:41.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.992+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1c0697a0 con 0x7f4c1c104f50 2024-04-13T05:15:41.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.994+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4c0c0055e0 con 0x7f4c1c104f50 2024-04-13T05:15:41.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.995+0000 7f4c13e00700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c000a3d50 0x7f4c000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:41.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.995+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4c0c0d0830 con 0x7f4c1c104f50 2024-04-13T05:15:41.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.995+0000 7f4c1aa00700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c000a3d50 0x7f4c000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:41.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:41.996+0000 7f4c1aa00700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c000a3d50 0x7f4c000a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006010 tx=0x7f4c08009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:42.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.000+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c0c020020 con 0x7f4c1c104f50 2024-04-13T05:15:42.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.232+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c1c02dfa0 con 0x7f4c1c104f50 2024-04-13T05:15:42.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.236+0000 7f4c13e00700 1 -- 172.21.15.57:0/775537473 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24583 (secure 0 0 0) 0x7f4c0c09ccf0 con 0x7f4c1c104f50 2024-04-13T05:15:42.239 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:15:42.239 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:15:13.150673+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:15:42.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.240+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c000a3d50 msgr2=0x7f4c000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:42.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.240+0000 7f4c22400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c000a3d50 0x7f4c000a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006010 tx=0x7f4c08009f10 comp rx=0 tx=0).stop 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.240+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 msgr2=0x7f4c1c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.240+0000 7f4c22400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 0x7f4c1c19b3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00d980 tx=0x7f4c0c00dd40 comp rx=0 tx=0).stop 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 shutdown_connections 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c1c10a030 0x7f4c1c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c000a3d50 0x7f4c000a6210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c1c104f50 0x7f4c1c19b3b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 --2- 172.21.15.57:0/775537473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c1c105900 0x7f4c1c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 >> 172.21.15.57:0/775537473 conn(0x7f4c1c1007e0 msgr2=0x7f4c1c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 shutdown_connections 2024-04-13T05:15:42.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:42.241+0000 7f4c22400700 1 -- 172.21.15.57:0/775537473 wait complete. 2024-04-13T05:15:43.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:42 smithi083 ceph-mon[35133]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-13T05:15:43.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:42 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/775537473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:43.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:42 smithi057 ceph-mon[32331]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-13T05:15:43.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:42 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/775537473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:43.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:42 smithi189 ceph-mon[35531]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-13T05:15:43.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:42 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/775537473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:15:43.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-13T05:15:43.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-13T05:15:43.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-13T05:15:43.175 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-13T05:15:43.175 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-13T05:15:43.176 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-13T05:15:43.176 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-13T05:15:43.176 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-13T05:15:43.177 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-13T05:15:43.177 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-13T05:15:43.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-13T05:15:43.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-13T05:15:44.225 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.339 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.342 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.390 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.393 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.397 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.560 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:45.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:44 smithi083 ceph-mon[35133]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-13T05:15:45.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:44 smithi057 ceph-mon[32331]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-13T05:15:45.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:44 smithi189 ceph-mon[35531]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-13T05:15:46.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:46 smithi057 ceph-mon[32331]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-13T05:15:47.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:46 smithi083 ceph-mon[35133]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-13T05:15:47.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:46 smithi189 ceph-mon[35531]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-13T05:15:47.408 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:47.513 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:48.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:48 smithi057 ceph-mon[32331]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-13T05:15:49.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:48 smithi083 ceph-mon[35133]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-13T05:15:49.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:48 smithi189 ceph-mon[35531]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-13T05:15:49.880 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:15:50.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:50 smithi057 ceph-mon[32331]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-13T05:15:51.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:50 smithi083 ceph-mon[35133]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-13T05:15:51.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:50 smithi189 ceph-mon[35531]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-13T05:15:52.975 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:52 smithi057 ceph-mon[32331]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-13T05:15:53.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:52 smithi189 ceph-mon[35531]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-13T05:15:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:52 smithi083 ceph-mon[35133]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-13T05:15:55.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:54 smithi057 ceph-mon[32331]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-13T05:15:55.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:54 smithi189 ceph-mon[35531]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-13T05:15:55.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:54 smithi083 ceph-mon[35133]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-13T05:15:56.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:56 smithi057 ceph-mon[32331]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-13T05:15:56.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:15:56.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:56.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 -- 172.21.15.57:0/193669049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 msgr2=0x7ff92c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 --2- 172.21.15.57:0/193669049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c10dd40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9140099c0 tx=0x7ff914009cd0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 -- 172.21.15.57:0/193669049 shutdown_connections 2024-04-13T05:15:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 --2- 172.21.15.57:0/193669049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff92c109340 0x7ff92c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 --2- 172.21.15.57:0/193669049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c10e420 0x7ff92c115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 --2- 172.21.15.57:0/193669049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c10dd40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.732+0000 7ff931600700 1 -- 172.21.15.57:0/193669049 >> 172.21.15.57:0/193669049 conn(0x7ff92c076280 msgr2=0x7ff92c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.733+0000 7ff931600700 1 -- 172.21.15.57:0/193669049 shutdown_connections 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.733+0000 7ff931600700 1 -- 172.21.15.57:0/193669049 wait complete. 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 Processor -- start 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 -- start start 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c109340 0x7ff92c1a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff92c10e420 0x7ff92c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c119360 con 0x7ff92c109cf0 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1191e0 con 0x7ff92c109340 2024-04-13T05:15:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.734+0000 7ff931600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c1194e0 con 0x7ff92c10e420 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c1a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45006/0 (socket says 172.21.15.57:45006) 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 -- 172.21.15.57:0/3511605517 learned_addr learned my addr 172.21.15.57:0/3511605517 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff92aa00700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c109340 0x7ff92c1a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff92b400700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff92c10e420 0x7ff92c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 -- 172.21.15.57:0/3511605517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff92c10e420 msgr2=0x7ff92c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff92c10e420 0x7ff92c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 -- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c109340 msgr2=0x7ff92c1a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff92c109340 0x7ff92c1a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.735+0000 7ff923e00700 1 -- 172.21.15.57:0/3511605517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c19fda0 con 0x7ff92c109cf0 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.736+0000 7ff923e00700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c1a5970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff91400c010 tx=0x7ff9140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.736+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914027070 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.736+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff914003680 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.736+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c1a0030 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.736+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914022b20 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.736+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1ac300 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.737+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff92c063240 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.739+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff91402a070 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.740+0000 7ff923400700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9100a3d50 0x7ff9100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff9140d23d0 con 0x7ff92c109cf0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff92aa00700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9100a3d50 0x7ff9100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff923400700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] conn(0x7ff9100aeeb0 0x7ff9100b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 --> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff91402be50 con 0x7ff9100aeeb0 2024-04-13T05:15:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff9140d2780 con 0x7ff92c109cf0 2024-04-13T05:15:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff92b400700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] conn(0x7ff9100aeeb0 0x7ff9100b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.741+0000 7ff92aa00700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9100a3d50 0x7ff9100a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009a10 tx=0x7ff91c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.742+0000 7ff92b400700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] conn(0x7ff9100aeeb0 0x7ff9100b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.743+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== osd.10 v2:172.21.15.83:6824/3322982382 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff91402be50 con 0x7ff9100aeeb0 2024-04-13T05:15:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.765+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 --> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff92c063640 con 0x7ff9100aeeb0 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.766+0000 7ff923400700 1 -- 172.21.15.57:0/3511605517 <== osd.10 v2:172.21.15.83:6824/3322982382 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff92c063640 con 0x7ff9100aeeb0 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] conn(0x7ff9100aeeb0 msgr2=0x7ff9100b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] conn(0x7ff9100aeeb0 0x7ff9100b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9100a3d50 msgr2=0x7ff9100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9100a3d50 0x7ff9100a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009a10 tx=0x7ff91c009190 comp rx=0 tx=0).stop 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 msgr2=0x7ff92c1a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 --2- 172.21.15.57:0/3511605517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92c109cf0 0x7ff92c1a5970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff91400c010 tx=0x7ff9140057c0 comp rx=0 tx=0).stop 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff92b400700 1 -- 172.21.15.57:0/3511605517 reap_dead start 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 shutdown_connections 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.767+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 >> 172.21.15.57:0/3511605517 conn(0x7ff92c076280 msgr2=0x7ff92c076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.768+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 shutdown_connections 2024-04-13T05:15:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:56.768+0000 7ff931600700 1 -- 172.21.15.57:0/3511605517 wait complete. 2024-04-13T05:15:57.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:56 smithi189 ceph-mon[35531]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-13T05:15:57.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:15:57.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:57.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:57.250 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:56 smithi083 ceph-mon[35133]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-13T05:15:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:15:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:58 smithi057 ceph-mon[32331]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-13T05:15:59.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:15:58 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:58 smithi189 ceph-mon[35531]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-13T05:15:59.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:15:58 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:58 smithi083 ceph-mon[35133]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-13T05:15:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:15:58 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.720+0000 7f1b98400700 1 -- 172.21.15.57:0/1372938066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b901034c0 msgr2=0x7f1b9010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.720+0000 7f1b98400700 1 --2- 172.21.15.57:0/1372938066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b901034c0 0x7f1b9010f920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b0a0 tx=0x7f1b8800b3b0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.720+0000 7f1b98400700 1 -- 172.21.15.57:0/1372938066 shutdown_connections 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.720+0000 7f1b98400700 1 --2- 172.21.15.57:0/1372938066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b901034c0 0x7f1b9010f920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.720+0000 7f1b98400700 1 --2- 172.21.15.57:0/1372938066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b90108ac0 0x7f1b90108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.721+0000 7f1b98400700 1 --2- 172.21.15.57:0/1372938066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b90102b00 0x7f1b90102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.721+0000 7f1b98400700 1 -- 172.21.15.57:0/1372938066 >> 172.21.15.57:0/1372938066 conn(0x7f1b90079c10 msgr2=0x7f1b9007a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.724+0000 7f1b98400700 1 -- 172.21.15.57:0/1372938066 shutdown_connections 2024-04-13T05:15:59.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.724+0000 7f1b98400700 1 -- 172.21.15.57:0/1372938066 wait complete. 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.725+0000 7f1b98400700 1 Processor -- start 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.725+0000 7f1b98400700 1 -- start start 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b98400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b90102b00 0x7f1b90074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b98400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b901034c0 0x7f1b900752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b98400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b90108ac0 0x7f1b90075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b98400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90112c70 con 0x7f1b90102b00 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b98400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90112af0 con 0x7f1b90108ac0 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b98400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90112df0 con 0x7f1b901034c0 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b95a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b90102b00 0x7f1b90074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b95a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b90102b00 0x7f1b90074db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45050/0 (socket says 172.21.15.57:45050) 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b95a00700 1 -- 172.21.15.57:0/2582446051 learned_addr learned my addr 172.21.15.57:0/2582446051 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.726+0000 7f1b96400700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b90108ac0 0x7f1b90075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:59.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95000700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b901034c0 0x7f1b900752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95a00700 1 -- 172.21.15.57:0/2582446051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b901034c0 msgr2=0x7f1b900752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95a00700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b901034c0 0x7f1b900752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95a00700 1 -- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b90108ac0 msgr2=0x7f1b90075830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95a00700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b90108ac0 0x7f1b90075830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95a00700 1 -- 172.21.15.57:0/2582446051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b901a7640 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.727+0000 7f1b95a00700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b90102b00 0x7f1b90074db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e9a0 tx=0x7f1b8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.728+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400cbd0 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.728+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b8400cd70 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.728+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84018680 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.728+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b901a7930 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.728+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b901a7e20 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.729+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b90063180 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.732+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b8401f020 con 0x7f1b90102b00 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.733+0000 7f1b8e000700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1b780a3d50 0x7f1b780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:59.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.733+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1b840d1360 con 0x7f1b90102b00 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.733+0000 7f1b8e000700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] conn(0x7f1b780aeeb0 0x7f1b780b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.733+0000 7f1b95000700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1b780a3d50 0x7f1b780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.733+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 --> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b780b1a20 con 0x7f1b780aeeb0 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.733+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f1b840d1710 con 0x7f1b90102b00 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.734+0000 7f1b96400700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] conn(0x7f1b780aeeb0 0x7f1b780b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.734+0000 7f1b95000700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1b780a3d50 0x7f1b780a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b9007a210 tx=0x7f1b7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.735+0000 7f1b96400700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] conn(0x7f1b780aeeb0 0x7f1b780b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:15:59.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.736+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== osd.4 v2:172.21.15.83:6808/4161937818 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b780b1a20 con 0x7f1b780aeeb0 2024-04-13T05:15:59.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.760+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 --> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b90063600 con 0x7f1b780aeeb0 2024-04-13T05:15:59.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.761+0000 7f1b8e000700 1 -- 172.21.15.57:0/2582446051 <== osd.4 v2:172.21.15.83:6808/4161937818 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b90063600 con 0x7f1b780aeeb0 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] conn(0x7f1b780aeeb0 msgr2=0x7f1b780b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] conn(0x7f1b780aeeb0 0x7f1b780b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1b780a3d50 msgr2=0x7f1b780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1b780a3d50 0x7f1b780a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b9007a210 tx=0x7f1b7c009260 comp rx=0 tx=0).stop 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b90102b00 msgr2=0x7f1b90074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 --2- 172.21.15.57:0/2582446051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b90102b00 0x7f1b90074db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e9a0 tx=0x7f1b8400ecb0 comp rx=0 tx=0).stop 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b96400700 1 -- 172.21.15.57:0/2582446051 reap_dead start 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 shutdown_connections 2024-04-13T05:15:59.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 >> 172.21.15.57:0/2582446051 conn(0x7f1b90079c10 msgr2=0x7f1b901017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:15:59.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 shutdown_connections 2024-04-13T05:15:59.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:59.762+0000 7f1b98400700 1 -- 172.21.15.57:0/2582446051 wait complete. 2024-04-13T05:16:00.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 -- 172.21.15.57:0/1107234862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 msgr2=0x7f084410edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 --2- 172.21.15.57:0/1107234862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 0x7f084410edf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0830009990 tx=0x7f0830009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 -- 172.21.15.57:0/1107234862 shutdown_connections 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 --2- 172.21.15.57:0/1107234862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 0x7f0844101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 --2- 172.21.15.57:0/1107234862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084410f330 0x7f0844111720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 --2- 172.21.15.57:0/1107234862 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 0x7f084410edf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 -- 172.21.15.57:0/1107234862 >> 172.21.15.57:0/1107234862 conn(0x7f08440fd440 msgr2=0x7f08440ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 -- 172.21.15.57:0/1107234862 shutdown_connections 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.089+0000 7f084ac00700 1 -- 172.21.15.57:0/1107234862 wait complete. 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.090+0000 7f084ac00700 1 Processor -- start 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.090+0000 7f084ac00700 1 -- start start 2024-04-13T05:16:00.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084ac00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 0x7f084419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084ac00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 0x7f084419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084ac00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084410f330 0x7f084419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084ac00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114e30 con 0x7f0844102130 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084ac00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114cb0 con 0x7f084410f330 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084ac00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114fb0 con 0x7f0844101780 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f0849800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 0x7f084419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f0849800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 0x7f084419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:58260/0 (socket says 172.21.15.57:58260) 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f0849800700 1 -- 172.21.15.57:0/243634648 learned_addr learned my addr 172.21.15.57:0/243634648 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:00.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f0848e00700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 0x7f084419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.091+0000 7f084a200700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084410f330 0x7f084419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0849800700 1 -- 172.21.15.57:0/243634648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084410f330 msgr2=0x7f084419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0849800700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084410f330 0x7f084419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0849800700 1 -- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 msgr2=0x7f084419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0849800700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0844102130 0x7f084419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0849800700 1 -- 172.21.15.57:0/243634648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08441a02b0 con 0x7f0844101780 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f084a200700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f084410f330 0x7f084419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0849800700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 0x7f084419b2a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f083800ef10 tx=0x7f083800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800ca70 con 0x7f0844101780 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0838019070 con 0x7f0844101780 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08441a0540 con 0x7f0844101780 2024-04-13T05:16:00.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838014480 con 0x7f0844101780 2024-04-13T05:16:00.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.092+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a7e20 con 0x7f0844101780 2024-04-13T05:16:00.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.093+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08440631c0 con 0x7f0844101780 2024-04-13T05:16:00.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.095+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0838012050 con 0x7f0844101780 2024-04-13T05:16:00.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.097+0000 7f0842000700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f082c0a3cd0 0x7f082c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.097+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08380d1480 con 0x7f0844101780 2024-04-13T05:16:00.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.097+0000 7f0848e00700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f082c0a3cd0 0x7f082c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.097+0000 7f0842000700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] conn(0x7f082c0aedb0 0x7f082c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.097+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 --> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0838015ec0 con 0x7f082c0aedb0 2024-04-13T05:16:00.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.097+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f08380d1830 con 0x7f0844101780 2024-04-13T05:16:00.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.098+0000 7f084a200700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] conn(0x7f082c0aedb0 0x7f082c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.098+0000 7f0848e00700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f082c0a3cd0 0x7f082c0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f083000b2e0 tx=0x7f0830005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:00.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.098+0000 7f084a200700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] conn(0x7f082c0aedb0 0x7f082c0b1250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.099+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== osd.3 v2:172.21.15.57:6810/744892058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0838015ec0 con 0x7f082c0aedb0 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.118+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 --> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0844063640 con 0x7f082c0aedb0 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.118+0000 7f0842000700 1 -- 172.21.15.57:0/243634648 <== osd.3 v2:172.21.15.57:6810/744892058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0844063640 con 0x7f082c0aedb0 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] conn(0x7f082c0aedb0 msgr2=0x7f082c0b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] conn(0x7f082c0aedb0 0x7f082c0b1250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f082c0a3cd0 msgr2=0x7f082c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f082c0a3cd0 0x7f082c0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f083000b2e0 tx=0x7f0830005960 comp rx=0 tx=0).stop 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 msgr2=0x7f084419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 --2- 172.21.15.57:0/243634648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0844101780 0x7f084419b2a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f083800ef10 tx=0x7f083800c3a0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084a200700 1 -- 172.21.15.57:0/243634648 reap_dead start 2024-04-13T05:16:00.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 shutdown_connections 2024-04-13T05:16:00.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 >> 172.21.15.57:0/243634648 conn(0x7f08440fd440 msgr2=0x7f08440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:00.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 shutdown_connections 2024-04-13T05:16:00.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:15:58.119+0000 7f084ac00700 1 -- 172.21.15.57:0/243634648 wait complete. 2024-04-13T05:16:00.707 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:00 smithi057 ceph-mon[32331]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-13T05:16:00.707 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:00 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:00.707 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:00 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 -- 172.21.15.57:0/3797055974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958075600 msgr2=0x7ff95810efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 --2- 172.21.15.57:0/3797055974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958075600 0x7ff95810efa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff944009990 tx=0x7ff944009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 -- 172.21.15.57:0/3797055974 shutdown_connections 2024-04-13T05:16:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 --2- 172.21.15.57:0/3797055974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff958074ce0 0x7ff9580750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 --2- 172.21.15.57:0/3797055974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95810f4e0 0x7ff9581118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 --2- 172.21.15.57:0/3797055974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958075600 0x7ff95810efa0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 -- 172.21.15.57:0/3797055974 >> 172.21.15.57:0/3797055974 conn(0x7ff958079b80 msgr2=0x7ff958079f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 -- 172.21.15.57:0/3797055974 shutdown_connections 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.028+0000 7ff95fe00700 1 -- 172.21.15.57:0/3797055974 wait complete. 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.029+0000 7ff95fe00700 1 Processor -- start 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.029+0000 7ff95fe00700 1 -- start start 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.029+0000 7ff95fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958074ce0 0x7ff95819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.029+0000 7ff95fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff958075600 0x7ff95819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.029+0000 7ff95fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff95810f4e0 0x7ff95819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95fe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff958114f00 con 0x7ff958074ce0 2024-04-13T05:16:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff958114d80 con 0x7ff958075600 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95fe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff958115080 con 0x7ff95810f4e0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958074ce0 0x7ff95819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958074ce0 0x7ff95819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45070/0 (socket says 172.21.15.57:45070) 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 -- 172.21.15.57:0/773157994 learned_addr learned my addr 172.21.15.57:0/773157994 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95de00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff95810f4e0 0x7ff95819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95ca00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff958075600 0x7ff95819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 -- 172.21.15.57:0/773157994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff95810f4e0 msgr2=0x7ff95819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff95810f4e0 0x7ff95819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 -- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff958075600 msgr2=0x7ff95819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff958075600 0x7ff95819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.030+0000 7ff95d400700 1 -- 172.21.15.57:0/773157994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9581a03d0 con 0x7ff958074ce0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.031+0000 7ff95d400700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958074ce0 0x7ff95819b410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff94c007f10 tx=0x7ff94c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.031+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00dc90 con 0x7ff958074ce0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.031+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff94c00de30 con 0x7ff958074ce0 2024-04-13T05:16:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.031+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0133f0 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.031+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9581a06c0 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.031+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9581a7f40 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.032+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9580631c0 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.034+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff94c01a020 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.036+0000 7ff956000700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9400a3d50 0x7ff9400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.036+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff94c0d0800 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.036+0000 7ff95ca00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9400a3d50 0x7ff9400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.036+0000 7ff956000700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] conn(0x7ff9400aeeb0 0x7ff9400b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.036+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 --> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9400b1a20 con 0x7ff9400aeeb0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.036+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff94c0d0bb0 con 0x7ff958074ce0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.037+0000 7ff95ca00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9400a3d50 0x7ff9400a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b5c0 tx=0x7ff94401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.037+0000 7ff95de00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] conn(0x7ff9400aeeb0 0x7ff9400b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.038+0000 7ff95de00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] conn(0x7ff9400aeeb0 0x7ff9400b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.040+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== osd.1 v2:172.21.15.83:6800/3349778520 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9400b1a20 con 0x7ff9400aeeb0 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.059+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 --> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff958063640 con 0x7ff9400aeeb0 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.060+0000 7ff956000700 1 -- 172.21.15.57:0/773157994 <== osd.1 v2:172.21.15.83:6800/3349778520 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff958063640 con 0x7ff9400aeeb0 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.060+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] conn(0x7ff9400aeeb0 msgr2=0x7ff9400b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.060+0000 7ff95fe00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] conn(0x7ff9400aeeb0 0x7ff9400b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9400a3d50 msgr2=0x7ff9400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff9400a3d50 0x7ff9400a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b5c0 tx=0x7ff94401a040 comp rx=0 tx=0).stop 2024-04-13T05:16:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958074ce0 msgr2=0x7ff95819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 --2- 172.21.15.57:0/773157994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff958074ce0 0x7ff95819b410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff94c007f10 tx=0x7ff94c00d3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95de00700 1 -- 172.21.15.57:0/773157994 reap_dead start 2024-04-13T05:16:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 shutdown_connections 2024-04-13T05:16:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 >> 172.21.15.57:0/773157994 conn(0x7ff958079b80 msgr2=0x7ff9580ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 shutdown_connections 2024-04-13T05:16:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.061+0000 7ff95fe00700 1 -- 172.21.15.57:0/773157994 wait complete. 2024-04-13T05:16:01.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:00 smithi083 ceph-mon[35133]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-13T05:16:01.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:00 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:01.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:00 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:01.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:00 smithi189 ceph-mon[35531]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-13T05:16:01.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:00 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:01.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:00 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 -- 172.21.15.57:0/1325132414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc102370 msgr2=0x7f32fc10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 --2- 172.21.15.57:0/1325132414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc102370 0x7f32fc10e860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f32f800b0a0 tx=0x7f32f800b3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 -- 172.21.15.57:0/1325132414 shutdown_connections 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 --2- 172.21.15.57:0/1325132414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32fc107a00 0x7f32fc107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 --2- 172.21.15.57:0/1325132414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc102370 0x7f32fc10e860 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 --2- 172.21.15.57:0/1325132414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32fc1019b0 0x7f32fc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 -- 172.21.15.57:0/1325132414 >> 172.21.15.57:0/1325132414 conn(0x7f32fc078960 msgr2=0x7f32fc078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:01.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 -- 172.21.15.57:0/1325132414 shutdown_connections 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.811+0000 7f3304800700 1 -- 172.21.15.57:0/1325132414 wait complete. 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.812+0000 7f3304800700 1 Processor -- start 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 -- start start 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1019b0 0x7f32fc19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32fc102370 0x7f32fc19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32fc107a00 0x7f32fc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc111b60 con 0x7f32fc1019b0 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1119e0 con 0x7f32fc102370 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3304800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc111ce0 con 0x7f32fc107a00 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.813+0000 7f3301e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1019b0 0x7f32fc19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1019b0 0x7f32fc19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45106/0 (socket says 172.21.15.57:45106) 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 -- 172.21.15.57:0/686966309 learned_addr learned my addr 172.21.15.57:0/686966309 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301400700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32fc102370 0x7f32fc19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3302800700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32fc107a00 0x7f32fc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:01.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 -- 172.21.15.57:0/686966309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32fc107a00 msgr2=0x7f32fc19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32fc107a00 0x7f32fc19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 -- 172.21.15.57:0/686966309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32fc102370 msgr2=0x7f32fc19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32fc102370 0x7f32fc19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 -- 172.21.15.57:0/686966309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc1a0320 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.814+0000 7f3301e00700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1019b0 0x7f32fc19b310 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d980 tx=0x7f32f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.815+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0017070 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.815+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f0005020 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.815+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc1a0610 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.815+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0009b50 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.815+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc1a7e90 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.816+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32fc06b170 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.818+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f32f0009cf0 con 0x7f32fc1019b0 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.820+0000 7f32f6a00700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32e40a3d50 0x7f32e40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:01.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32f00d0cc0 con 0x7f32fc1019b0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f3301400700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32e40a3d50 0x7f32e40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f32f6a00700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] conn(0x7f32e40aeeb0 0x7f32e40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 --> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32e40b1a20 con 0x7f32e40aeeb0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f32f00d1070 con 0x7f32fc1019b0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f3302800700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] conn(0x7f32e40aeeb0 0x7f32e40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.821+0000 7f3301400700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32e40a3d50 0x7f32e40a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32fc101960 tx=0x7f32ec0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.822+0000 7f3302800700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] conn(0x7f32e40aeeb0 0x7f32e40b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.824+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== osd.9 v2:172.21.15.57:6826/772937506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32e40b1a20 con 0x7f32e40aeeb0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.842+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 --> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32fc1030d0 con 0x7f32e40aeeb0 2024-04-13T05:16:01.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.843+0000 7f32f6a00700 1 -- 172.21.15.57:0/686966309 <== osd.9 v2:172.21.15.57:6826/772937506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32fc1030d0 con 0x7f32e40aeeb0 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.843+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] conn(0x7f32e40aeeb0 msgr2=0x7f32e40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] conn(0x7f32e40aeeb0 0x7f32e40b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32e40a3d50 msgr2=0x7f32e40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32e40a3d50 0x7f32e40a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32fc101960 tx=0x7f32ec0073d0 comp rx=0 tx=0).stop 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1019b0 msgr2=0x7f32fc19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 --2- 172.21.15.57:0/686966309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1019b0 0x7f32fc19b310 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d980 tx=0x7f32f000dc90 comp rx=0 tx=0).stop 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3302800700 1 -- 172.21.15.57:0/686966309 reap_dead start 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 shutdown_connections 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 >> 172.21.15.57:0/686966309 conn(0x7f32fc078960 msgr2=0x7f32fc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 shutdown_connections 2024-04-13T05:16:01.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.844+0000 7f3304800700 1 -- 172.21.15.57:0/686966309 wait complete. 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.222+0000 7f43d0c00700 1 -- 172.21.15.57:0/2528511518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc109340 msgr2=0x7f43cc109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.222+0000 7f43d0c00700 1 --2- 172.21.15.57:0/2528511518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc109340 0x7f43cc109720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0099c0 tx=0x7f43bc009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 -- 172.21.15.57:0/2528511518 shutdown_connections 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 --2- 172.21.15.57:0/2528511518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43cc10e420 0x7f43cc115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 --2- 172.21.15.57:0/2528511518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc109340 0x7f43cc109720 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 --2- 172.21.15.57:0/2528511518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43cc109cf0 0x7f43cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 -- 172.21.15.57:0/2528511518 >> 172.21.15.57:0/2528511518 conn(0x7f43cc076280 msgr2=0x7f43cc1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:02.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 -- 172.21.15.57:0/2528511518 shutdown_connections 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.223+0000 7f43d0c00700 1 -- 172.21.15.57:0/2528511518 wait complete. 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.224+0000 7f43d0c00700 1 Processor -- start 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.224+0000 7f43d0c00700 1 -- start start 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43d0c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43cc109340 0x7f43cc1a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43d0c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43cc109cf0 0x7f43cc1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43d0c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc10e420 0x7f43cc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43d0c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc119300 con 0x7f43cc10e420 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43d0c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc119180 con 0x7f43cc109cf0 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43d0c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc119480 con 0x7f43cc109340 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43ca800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc10e420 0x7f43cc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43ca800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc10e420 0x7f43cc19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45122/0 (socket says 172.21.15.57:45122) 2024-04-13T05:16:02.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43ca800700 1 -- 172.21.15.57:0/1928789773 learned_addr learned my addr 172.21.15.57:0/1928789773 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43c9e00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43cc109340 0x7f43cc1a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.225+0000 7f43c3e00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43cc109cf0 0x7f43cc1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43ca800700 1 -- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43cc109340 msgr2=0x7f43cc1a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43ca800700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43cc109340 0x7f43cc1a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43ca800700 1 -- 172.21.15.57:0/1928789773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43cc109cf0 msgr2=0x7f43cc1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43ca800700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43cc109cf0 0x7f43cc1a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43ca800700 1 -- 172.21.15.57:0/1928789773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc19fda0 con 0x7f43cc10e420 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43ca800700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc10e420 0x7f43cc19f640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007ee0 tx=0x7f43c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400ddd0 con 0x7f43cc10e420 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43c4004020 con 0x7f43cc10e420 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc1a0090 con 0x7f43cc10e420 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4015660 con 0x7f43cc10e420 2024-04-13T05:16:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.226+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc1ac300 con 0x7f43cc10e420 2024-04-13T05:16:02.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.227+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43cc063200 con 0x7f43cc10e420 2024-04-13T05:16:02.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.231+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f43c4003a00 con 0x7f43cc10e420 2024-04-13T05:16:02.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.232+0000 7f43c2a00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43b00a3cd0 0x7f43b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:02.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.233+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f43c40d0f30 con 0x7f43cc10e420 2024-04-13T05:16:02.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.233+0000 7f43c9e00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43b00a3cd0 0x7f43b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:02.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.233+0000 7f43c2a00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] conn(0x7f43b00aee30 0x7f43b00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:02.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.233+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 --> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43c400be50 con 0x7f43b00aee30 2024-04-13T05:16:02.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.233+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f43c40d12e0 con 0x7f43cc10e420 2024-04-13T05:16:02.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.234+0000 7f43c9e00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43b00a3cd0 0x7f43b00a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:02.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.234+0000 7f43c3e00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] conn(0x7f43b00aee30 0x7f43b00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:02.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.235+0000 7f43c3e00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] conn(0x7f43b00aee30 0x7f43b00b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.237+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== osd.8 v2:172.21.15.189:6816/2146992789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43c400be50 con 0x7f43b00aee30 2024-04-13T05:16:02.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.256+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 --> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43cc063680 con 0x7f43b00aee30 2024-04-13T05:16:02.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.257+0000 7f43c2a00700 1 -- 172.21.15.57:0/1928789773 <== osd.8 v2:172.21.15.189:6816/2146992789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f43cc063680 con 0x7f43b00aee30 2024-04-13T05:16:02.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.257+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] conn(0x7f43b00aee30 msgr2=0x7f43b00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:02.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.257+0000 7f43d0c00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] conn(0x7f43b00aee30 0x7f43b00b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43b00a3cd0 msgr2=0x7f43b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43b00a3cd0 0x7f43b00a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc005fb0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc10e420 msgr2=0x7f43cc19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 --2- 172.21.15.57:0/1928789773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43cc10e420 0x7f43cc19f640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007ee0 tx=0x7f43c400d3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43ca800700 1 -- 172.21.15.57:0/1928789773 reap_dead start 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 shutdown_connections 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 >> 172.21.15.57:0/1928789773 conn(0x7f43cc076280 msgr2=0x7f43cc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 shutdown_connections 2024-04-13T05:16:02.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.258+0000 7f43d0c00700 1 -- 172.21.15.57:0/1928789773 wait complete. 2024-04-13T05:16:02.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:02 smithi057 ceph-mon[32331]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:03.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:02 smithi083 ceph-mon[35133]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:03.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:02 smithi189 ceph-mon[35531]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:03.129 INFO:teuthology.orchestra.run.smithi057.stdout:55834574948 2024-04-13T05:16:03.129 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.673+0000 7f921a200700 1 -- 172.21.15.57:0/1785915587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406a7c0 msgr2=0x7f921406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.673+0000 7f921a200700 1 --2- 172.21.15.57:0/1785915587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406a7c0 0x7f921406ac20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9200009990 tx=0x7f9200009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.673+0000 7f921a200700 1 -- 172.21.15.57:0/1785915587 shutdown_connections 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.674+0000 7f921a200700 1 --2- 172.21.15.57:0/1785915587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921406b160 0x7f921410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.674+0000 7f921a200700 1 --2- 172.21.15.57:0/1785915587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9214108aa0 0x7f9214108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.674+0000 7f921a200700 1 --2- 172.21.15.57:0/1785915587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406a7c0 0x7f921406ac20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.674+0000 7f921a200700 1 -- 172.21.15.57:0/1785915587 >> 172.21.15.57:0/1785915587 conn(0x7f9214078960 msgr2=0x7f9214078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.674+0000 7f921a200700 1 -- 172.21.15.57:0/1785915587 shutdown_connections 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.674+0000 7f921a200700 1 -- 172.21.15.57:0/1785915587 wait complete. 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 Processor -- start 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 -- start start 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921406a7c0 0x7f921419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:03.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406b160 0x7f921419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9214108aa0 0x7f921419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9214112c90 con 0x7f921406b160 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9214112b10 con 0x7f9214108aa0 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.675+0000 7f921a200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9214112e10 con 0x7f921406a7c0 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406b160 0x7f921419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406b160 0x7f921419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45090/0 (socket says 172.21.15.57:45090) 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 -- 172.21.15.57:0/36628023 learned_addr learned my addr 172.21.15.57:0/36628023 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9213e00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9214108aa0 0x7f921419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9213400700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921406a7c0 0x7f921419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 -- 172.21.15.57:0/36628023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921406a7c0 msgr2=0x7f921419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921406a7c0 0x7f921419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 -- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9214108aa0 msgr2=0x7f921419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:03.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9214108aa0 0x7f921419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9212a00700 1 -- 172.21.15.57:0/36628023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92141a03f0 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9213e00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9214108aa0 0x7f921419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.676+0000 7f9213400700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921406a7c0 0x7f921419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.677+0000 7f9212a00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406b160 0x7f921419b920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9200000c00 tx=0x7f9200005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.677+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200027070 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.677+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9200022470 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.677+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92141a0680 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.677+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200017630 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.677+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92141a7f60 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.678+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9214102b50 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.681+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f920002a070 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.683+0000 7f920be00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91f80a3cd0 0x7f91f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.683+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f92000d1a10 con 0x7f921406b160 2024-04-13T05:16:03.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.683+0000 7f9213400700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91f80a3cd0 0x7f91f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.684+0000 7f920be00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] conn(0x7f91f80aee30 0x7f91f80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.684+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 --> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9200019e50 con 0x7f91f80aee30 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.684+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f92000d1dc0 con 0x7f921406b160 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.684+0000 7f9213400700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91f80a3cd0 0x7f91f80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fd0 tx=0x7f920400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.684+0000 7f9213e00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] conn(0x7f91f80aee30 0x7f91f80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.685+0000 7f9213e00700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] conn(0x7f91f80aee30 0x7f91f80b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.686+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== osd.7 v2:172.21.15.83:6816/977203651 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9200019e50 con 0x7f91f80aee30 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.705+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 --> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92141041e0 con 0x7f91f80aee30 2024-04-13T05:16:03.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f920be00700 1 -- 172.21.15.57:0/36628023 <== osd.7 v2:172.21.15.83:6816/977203651 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f92141041e0 con 0x7f91f80aee30 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] conn(0x7f91f80aee30 msgr2=0x7f91f80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f921a200700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] conn(0x7f91f80aee30 0x7f91f80b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91f80a3cd0 msgr2=0x7f91f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f921a200700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91f80a3cd0 0x7f91f80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fd0 tx=0x7f920400b040 comp rx=0 tx=0).stop 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406b160 msgr2=0x7f921419b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f921a200700 1 --2- 172.21.15.57:0/36628023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f921406b160 0x7f921419b920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9200000c00 tx=0x7f9200005ef0 comp rx=0 tx=0).stop 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.706+0000 7f9213e00700 1 -- 172.21.15.57:0/36628023 reap_dead start 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.707+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 shutdown_connections 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.707+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 >> 172.21.15.57:0/36628023 conn(0x7f9214078960 msgr2=0x7f92140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:03.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.707+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 shutdown_connections 2024-04-13T05:16:03.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:00.707+0000 7f921a200700 1 -- 172.21.15.57:0/36628023 wait complete. 2024-04-13T05:16:04.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:04.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:04.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 -- 172.21.15.57:0/3994651076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 msgr2=0x7f9ebc108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:04.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 --2- 172.21.15.57:0/3994651076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc108ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009990 tx=0x7f9eac009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 -- 172.21.15.57:0/3994651076 shutdown_connections 2024-04-13T05:16:04.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 --2- 172.21.15.57:0/3994651076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ebc1034c0 0x7f9ebc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 --2- 172.21.15.57:0/3994651076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc108ea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 --2- 172.21.15.57:0/3994651076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ebc102b00 0x7f9ebc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 -- 172.21.15.57:0/3994651076 >> 172.21.15.57:0/3994651076 conn(0x7f9ebc079c10 msgr2=0x7f9ebc07a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 -- 172.21.15.57:0/3994651076 shutdown_connections 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.941+0000 7f9ec2600700 1 -- 172.21.15.57:0/3994651076 wait complete. 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.943+0000 7f9ec2600700 1 Processor -- start 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.943+0000 7f9ec2600700 1 -- start start 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ec2600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ebc102b00 0x7f9ebc074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ec2600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ebc1034c0 0x7f9ebc0752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ec2600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ec2600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc112cd0 con 0x7f9ebc108ac0 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ec2600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc112b50 con 0x7f9ebc1034c0 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ec2600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc112e50 con 0x7f9ebc102b00 2024-04-13T05:16:04.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ebbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ebbe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc075830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45134/0 (socket says 172.21.15.57:45134) 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ebbe00700 1 -- 172.21.15.57:0/55958319 learned_addr learned my addr 172.21.15.57:0/55958319 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.944+0000 7f9ebb400700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ebc102b00 0x7f9ebc074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebaa00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ebc1034c0 0x7f9ebc0752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebbe00700 1 -- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ebc102b00 msgr2=0x7f9ebc074db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebbe00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ebc102b00 0x7f9ebc074db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebbe00700 1 -- 172.21.15.57:0/55958319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ebc1034c0 msgr2=0x7f9ebc0752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebbe00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ebc1034c0 0x7f9ebc0752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebbe00700 1 -- 172.21.15.57:0/55958319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc1a7720 con 0x7f9ebc108ac0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebaa00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ebc1034c0 0x7f9ebc0752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebb400700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ebc102b00 0x7f9ebc074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.945+0000 7f9ebbe00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc075830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4007d70 tx=0x7f9eb400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.946+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4016020 con 0x7f9ebc108ac0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.946+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9eb400ad40 con 0x7f9ebc108ac0 2024-04-13T05:16:04.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.946+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1a7a10 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.946+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4011620 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.946+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a7f50 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.947+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ebc063180 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.950+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9eb4019020 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.951+0000 7f9eb3e00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ea00a3d50 0x7f9ea00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.951+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9eb40d0d70 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.952+0000 7f9ebb400700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ea00a3d50 0x7f9ea00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.952+0000 7f9eb3e00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] conn(0x7f9ea00aeeb0 0x7f9ea00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.952+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 --> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9eb4019ec0 con 0x7f9ea00aeeb0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.952+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f9eb40d1120 con 0x7f9ebc108ac0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.952+0000 7f9ebb400700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ea00a3d50 0x7f9ea00a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00b5c0 tx=0x7f9eac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.953+0000 7f9ebaa00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] conn(0x7f9ea00aeeb0 0x7f9ea00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:04.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.954+0000 7f9ebaa00700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] conn(0x7f9ea00aeeb0 0x7f9ea00b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:04.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.957+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== osd.11 v2:172.21.15.189:6824/368428802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9eb4019ec0 con 0x7f9ea00aeeb0 2024-04-13T05:16:04.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.976+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 --> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ebc063600 con 0x7f9ea00aeeb0 2024-04-13T05:16:04.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.977+0000 7f9eb3e00700 1 -- 172.21.15.57:0/55958319 <== osd.11 v2:172.21.15.189:6824/368428802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ebc063600 con 0x7f9ea00aeeb0 2024-04-13T05:16:04.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] conn(0x7f9ea00aeeb0 msgr2=0x7f9ea00b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:04.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] conn(0x7f9ea00aeeb0 0x7f9ea00b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ea00a3d50 msgr2=0x7f9ea00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9ea00a3d50 0x7f9ea00a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00b5c0 tx=0x7f9eac005fd0 comp rx=0 tx=0).stop 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 msgr2=0x7f9ebc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 --2- 172.21.15.57:0/55958319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ebc108ac0 0x7f9ebc075830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4007d70 tx=0x7f9eb400a340 comp rx=0 tx=0).stop 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ebbe00700 1 -- 172.21.15.57:0/55958319 reap_dead start 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 shutdown_connections 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 >> 172.21.15.57:0/55958319 conn(0x7f9ebc079c10 msgr2=0x7f9ebc101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 shutdown_connections 2024-04-13T05:16:04.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:02.978+0000 7f9ec2600700 1 -- 172.21.15.57:0/55958319 wait complete. 2024-04-13T05:16:04.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:04.255 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:04.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:04.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:04.838 INFO:teuthology.orchestra.run.smithi057.stdout:313532612643 2024-04-13T05:16:04.839 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-13T05:16:05.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:05 smithi057 ceph-mon[32331]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:05 smithi083 ceph-mon[35133]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:05.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:05 smithi189 ceph-mon[35531]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:06.656 INFO:teuthology.orchestra.run.smithi057.stdout:107374182484 2024-04-13T05:16:06.657 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-13T05:16:07.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:07 smithi083 ceph-mon[35133]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:16:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:07 smithi057 ceph-mon[32331]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:16:07.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:07 smithi189 ceph-mon[35531]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:16:07.939 INFO:teuthology.orchestra.run.smithi057.stdout:347892351007 2024-04-13T05:16:07.940 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-13T05:16:08.468 INFO:teuthology.orchestra.run.smithi057.stdout:253403070515 2024-04-13T05:16:08.469 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-13T05:16:08.570 INFO:teuthology.orchestra.run.smithi057.stdout:287762808875 2024-04-13T05:16:08.571 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-13T05:16:08.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 -- 172.21.15.57:0/3218130004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f199010cfd0 msgr2=0x7f199010f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 --2- 172.21.15.57:0/3218130004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f199010cfd0 0x7f199010f490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f198800b0a0 tx=0x7f198800b3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 -- 172.21.15.57:0/3218130004 shutdown_connections 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 --2- 172.21.15.57:0/3218130004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f199010cfd0 0x7f199010f490 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 --2- 172.21.15.57:0/3218130004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990103ba0 0x7f1990103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 --2- 172.21.15.57:0/3218130004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901044c0 0x7f199010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 -- 172.21.15.57:0/3218130004 >> 172.21.15.57:0/3218130004 conn(0x7f19900fd3d0 msgr2=0x7f19900ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 -- 172.21.15.57:0/3218130004 shutdown_connections 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.768+0000 7f1996e00700 1 -- 172.21.15.57:0/3218130004 wait complete. 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.769+0000 7f1996e00700 1 Processor -- start 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 -- start start 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990103ba0 0x7f199019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19901044c0 0x7f199019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f199010cfd0 0x7f199019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112c30 con 0x7f19901044c0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112ab0 con 0x7f1990103ba0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1996e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112db0 con 0x7f199010cfd0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1995000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19901044c0 0x7f199019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.770+0000 7f1995000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19901044c0 0x7f199019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57730/0 (socket says 172.21.15.57:57730) 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 -- 172.21.15.57:0/437751098 learned_addr learned my addr 172.21.15.57:0/437751098 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995a00700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990103ba0 0x7f199019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 -- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f199010cfd0 msgr2=0x7f199019fbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f199010cfd0 0x7f199019fbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 -- 172.21.15.57:0/437751098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990103ba0 msgr2=0x7f199019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990103ba0 0x7f199019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 -- 172.21.15.57:0/437751098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19901a02f0 con 0x7f19901044c0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.771+0000 7f1995000700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19901044c0 0x7f199019b820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f197c00d720 tx=0x7f197c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:08.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.772+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c017070 con 0x7f19901044c0 2024-04-13T05:16:08.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.772+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901a05e0 con 0x7f19901044c0 2024-04-13T05:16:08.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.772+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f197c005020 con 0x7f19901044c0 2024-04-13T05:16:08.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.772+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c00fab0 con 0x7f19901044c0 2024-04-13T05:16:08.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.772+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901a7e60 con 0x7f19901044c0 2024-04-13T05:16:08.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.773+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1990063200 con 0x7f19901044c0 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.776+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f197c0051d0 con 0x7f19901044c0 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.777+0000 7f198e000700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f19780a3cd0 0x7f19780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.777+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f197c0d0420 con 0x7f19901044c0 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.777+0000 7f198e000700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] conn(0x7f19780aee30 0x7f19780b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.778+0000 7f1995a00700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f19780a3cd0 0x7f19780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.778+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 --> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19780b19a0 con 0x7f19780aee30 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.778+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f197c0d0870 con 0x7f19901044c0 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.778+0000 7f1996400700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] conn(0x7f19780aee30 0x7f19780b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.778+0000 7f1995a00700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f19780a3cd0 0x7f19780a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1984005950 tx=0x7f19840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.779+0000 7f1996400700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] conn(0x7f19780aee30 0x7f19780b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.781+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== osd.5 v2:172.21.15.189:6808/3029406488 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19780b19a0 con 0x7f19780aee30 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.800+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 --> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1990063680 con 0x7f19780aee30 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.801+0000 7f198e000700 1 -- 172.21.15.57:0/437751098 <== osd.5 v2:172.21.15.189:6808/3029406488 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1990063680 con 0x7f19780aee30 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] conn(0x7f19780aee30 msgr2=0x7f19780b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] conn(0x7f19780aee30 0x7f19780b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f19780a3cd0 msgr2=0x7f19780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f19780a3cd0 0x7f19780a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1984005950 tx=0x7f19840058e0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19901044c0 msgr2=0x7f199019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:08.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 --2- 172.21.15.57:0/437751098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19901044c0 0x7f199019b820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f197c00d720 tx=0x7f197c00dae0 comp rx=0 tx=0).stop 2024-04-13T05:16:08.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996400700 1 -- 172.21.15.57:0/437751098 reap_dead start 2024-04-13T05:16:08.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 shutdown_connections 2024-04-13T05:16:08.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 >> 172.21.15.57:0/437751098 conn(0x7f19900fd3d0 msgr2=0x7f19900fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:08.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 shutdown_connections 2024-04-13T05:16:08.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:05.802+0000 7f1996e00700 1 -- 172.21.15.57:0/437751098 wait complete. 2024-04-13T05:16:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:09 smithi057 ceph-mon[32331]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.342+0000 7fb25f200700 1 -- 172.21.15.57:0/612582489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb25806b0e0 msgr2=0x7fb25810f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.342+0000 7fb25f200700 1 --2- 172.21.15.57:0/612582489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb25806b0e0 0x7fb25810f020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb244009990 tx=0x7fb244009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 -- 172.21.15.57:0/612582489 shutdown_connections 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 --2- 172.21.15.57:0/612582489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb25806a7c0 0x7fb25806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 --2- 172.21.15.57:0/612582489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25810f560 0x7fb258111950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 --2- 172.21.15.57:0/612582489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb25806b0e0 0x7fb25810f020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 -- 172.21.15.57:0/612582489 >> 172.21.15.57:0/612582489 conn(0x7fb258078960 msgr2=0x7fb258078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 -- 172.21.15.57:0/612582489 shutdown_connections 2024-04-13T05:16:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.343+0000 7fb25f200700 1 -- 172.21.15.57:0/612582489 wait complete. 2024-04-13T05:16:09.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.345+0000 7fb25f200700 1 Processor -- start 2024-04-13T05:16:09.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.345+0000 7fb25f200700 1 -- start start 2024-04-13T05:16:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.346+0000 7fb25f200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25806a7c0 0x7fb258103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.346+0000 7fb25f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb25806b0e0 0x7fb2581041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.346+0000 7fb25f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb25810f560 0x7fb258104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.346+0000 7fb25f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb258114f10 con 0x7fb25806b0e0 2024-04-13T05:16:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.346+0000 7fb25f200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb258114d90 con 0x7fb25806a7c0 2024-04-13T05:16:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.346+0000 7fb25f200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb258115090 con 0x7fb25810f560 2024-04-13T05:16:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.347+0000 7fb257e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25806a7c0 0x7fb258103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.347+0000 7fb257e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25806a7c0 0x7fb258103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:38462/0 (socket says 172.21.15.57:38462) 2024-04-13T05:16:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.347+0000 7fb257e00700 1 -- 172.21.15.57:0/3083990629 learned_addr learned my addr 172.21.15.57:0/3083990629 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.348+0000 7fb257e00700 1 -- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb25810f560 msgr2=0x7fb258104710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-13T05:16:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.349+0000 7fb257e00700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb25810f560 0x7fb258104710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.349+0000 7fb257e00700 1 -- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb25806b0e0 msgr2=0x7fb2581041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.349+0000 7fb257e00700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb25806b0e0 0x7fb2581041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.349+0000 7fb257e00700 1 -- 172.21.15.57:0/3083990629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2581aba70 con 0x7fb25806a7c0 2024-04-13T05:16:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.350+0000 7fb257e00700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25806a7c0 0x7fb258103c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c920 tx=0x7fb24800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.350+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248007b00 con 0x7fb25806a7c0 2024-04-13T05:16:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.350+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb248004ce0 con 0x7fb25806a7c0 2024-04-13T05:16:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.350+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248005600 con 0x7fb25806a7c0 2024-04-13T05:16:09.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.351+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2581abd60 con 0x7fb25806a7c0 2024-04-13T05:16:09.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.351+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2581ac1f0 con 0x7fb25806a7c0 2024-04-13T05:16:09.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.352+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb258063240 con 0x7fb25806a7c0 2024-04-13T05:16:09.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.355+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb248004750 con 0x7fb25806a7c0 2024-04-13T05:16:09.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.356+0000 7fb254c00700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.357+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb2480d0920 con 0x7fb25806a7c0 2024-04-13T05:16:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.357+0000 7fb254c00700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] conn(0x7fb23c0aef20 0x7fb23c0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.357+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 --> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb23c0b1a90 con 0x7fb23c0aef20 2024-04-13T05:16:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.357+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb2480d0d10 con 0x7fb25806a7c0 2024-04-13T05:16:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.357+0000 7fb257400700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.358+0000 7fb25d200700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] conn(0x7fb23c0aef20 0x7fb23c0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.358+0000 7fb257400700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2440098e0 tx=0x7fb244009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:09.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.359+0000 7fb25d200700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] conn(0x7fb23c0aef20 0x7fb23c0b13c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:09.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.362+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== osd.2 v2:172.21.15.189:6800/2908933106 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb23c0b1a90 con 0x7fb23c0aef20 2024-04-13T05:16:09.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.392+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 --> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb258063640 con 0x7fb23c0aef20 2024-04-13T05:16:09.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.393+0000 7fb254c00700 1 -- 172.21.15.57:0/3083990629 <== osd.2 v2:172.21.15.189:6800/2908933106 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb258063640 con 0x7fb23c0aef20 2024-04-13T05:16:09.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] conn(0x7fb23c0aef20 msgr2=0x7fb23c0b13c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:09.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] conn(0x7fb23c0aef20 0x7fb23c0b13c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb23c0a3dc0 msgr2=0x7fb23c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb23c0a3dc0 0x7fb23c0a6280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2440098e0 tx=0x7fb244009ee0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25806a7c0 msgr2=0x7fb258103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 --2- 172.21.15.57:0/3083990629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb25806a7c0 0x7fb258103c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c920 tx=0x7fb24800cce0 comp rx=0 tx=0).stop 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25d200700 1 -- 172.21.15.57:0/3083990629 reap_dead start 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 shutdown_connections 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.394+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 >> 172.21.15.57:0/3083990629 conn(0x7fb258078960 msgr2=0x7fb2581109b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:09.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.395+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 shutdown_connections 2024-04-13T05:16:09.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.395+0000 7fb25f200700 1 -- 172.21.15.57:0/3083990629 wait complete. 2024-04-13T05:16:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:09 smithi083 ceph-mon[35133]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:09.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:09 smithi189 ceph-mon[35531]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:10.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 -- 172.21.15.57:0/819444259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 msgr2=0x7f6b00105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:10.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 --2- 172.21.15.57:0/819444259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b00105330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6af00099c0 tx=0x7f6af0009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 -- 172.21.15.57:0/819444259 shutdown_connections 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 --2- 172.21.15.57:0/819444259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b0010a030 0x7f6b00111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 --2- 172.21.15.57:0/819444259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b00105330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 --2- 172.21.15.57:0/819444259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b00105900 0x7f6b00109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 -- 172.21.15.57:0/819444259 >> 172.21.15.57:0/819444259 conn(0x7f6b001007e0 msgr2=0x7f6b00102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 -- 172.21.15.57:0/819444259 shutdown_connections 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.974+0000 7f6b06e00700 1 -- 172.21.15.57:0/819444259 wait complete. 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.975+0000 7f6b06e00700 1 Processor -- start 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.975+0000 7f6b06e00700 1 -- start start 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b0019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b00105900 0x7f6b0019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b0010a030 0x7f6b0019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00114f30 con 0x7f6b00104f50 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00114db0 con 0x7f6b0010a030 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6b001150b0 con 0x7f6b00105900 2024-04-13T05:16:10.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b0019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b0019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57754/0 (socket says 172.21.15.57:57754) 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 -- 172.21.15.57:0/1617097525 learned_addr learned my addr 172.21.15.57:0/1617097525 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6af7400700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b00105900 0x7f6b0019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 -- 172.21.15.57:0/1617097525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b00105900 msgr2=0x7f6b0019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b00105900 0x7f6b0019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b04e00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b0010a030 0x7f6b0019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 -- 172.21.15.57:0/1617097525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b0010a030 msgr2=0x7f6b0019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b0010a030 0x7f6b0019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 -- 172.21.15.57:0/1617097525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b001a0470 con 0x7f6b00104f50 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6affe00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b0019b460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005910 tx=0x7f6af0004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0026070 con 0x7f6b00104f50 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b001a0700 con 0x7f6b00104f50 2024-04-13T05:16:10.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6af0003c70 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af000fbc0 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.976+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b001a7fe0 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.978+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b00063140 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.980+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6af0029070 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.982+0000 7f6afd600700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ae00a3cd0 0x7f6ae00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.982+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6af00d19b0 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.982+0000 7f6af7400700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ae00a3cd0 0x7f6ae00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.982+0000 7f6afd600700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] conn(0x7f6ae00aed00 0x7f6ae00b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.982+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 --> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ae00b1870 con 0x7f6ae00aed00 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.982+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6af00d1d60 con 0x7f6b00104f50 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.983+0000 7f6b04e00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] conn(0x7f6ae00aed00 0x7f6ae00b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.983+0000 7f6af7400700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ae00a3cd0 0x7f6ae00a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6b0010b620 tx=0x7f6ae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.983+0000 7f6b04e00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] conn(0x7f6ae00aed00 0x7f6ae00b11a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:09.983+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== osd.0 v2:172.21.15.57:6802/3403486452 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ae00b1870 con 0x7f6ae00aed00 2024-04-13T05:16:10.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.003+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 --> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b00063540 con 0x7f6ae00aed00 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.003+0000 7f6afd600700 1 -- 172.21.15.57:0/1617097525 <== osd.0 v2:172.21.15.57:6802/3403486452 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6b00063540 con 0x7f6ae00aed00 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] conn(0x7f6ae00aed00 msgr2=0x7f6ae00b11a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] conn(0x7f6ae00aed00 0x7f6ae00b11a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ae00a3cd0 msgr2=0x7f6ae00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ae00a3cd0 0x7f6ae00a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6b0010b620 tx=0x7f6ae8009260 comp rx=0 tx=0).stop 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 msgr2=0x7f6b0019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 --2- 172.21.15.57:0/1617097525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b00104f50 0x7f6b0019b460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005910 tx=0x7f6af0004320 comp rx=0 tx=0).stop 2024-04-13T05:16:10.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b04e00700 1 -- 172.21.15.57:0/1617097525 reap_dead start 2024-04-13T05:16:10.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 shutdown_connections 2024-04-13T05:16:10.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 >> 172.21.15.57:0/1617097525 conn(0x7f6b001007e0 msgr2=0x7f6b00102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:10.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 shutdown_connections 2024-04-13T05:16:10.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.004+0000 7f6b06e00700 1 -- 172.21.15.57:0/1617097525 wait complete. 2024-04-13T05:16:11.242 INFO:teuthology.orchestra.run.smithi057.stdout:141733920848 2024-04-13T05:16:11.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-13T05:16:11.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:11 smithi057 ceph-mon[32331]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-13T05:16:11.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:11 smithi083 ceph-mon[35133]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-13T05:16:11.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:11 smithi189 ceph-mon[35531]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-13T05:16:11.687 INFO:teuthology.orchestra.run.smithi057.stdout:219043332155 2024-04-13T05:16:11.687 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-13T05:16:12.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 -- 172.21.15.57:0/1813575834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9168075600 msgr2=0x7f916810efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 --2- 172.21.15.57:0/1813575834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9168075600 0x7f916810efd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9154009990 tx=0x7f9154009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 -- 172.21.15.57:0/1813575834 shutdown_connections 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 --2- 172.21.15.57:0/1813575834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f916810f510 0x7f9168111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 --2- 172.21.15.57:0/1813575834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9168074ce0 0x7f91680750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 --2- 172.21.15.57:0/1813575834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9168075600 0x7f916810efd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 -- 172.21.15.57:0/1813575834 >> 172.21.15.57:0/1813575834 conn(0x7f9168079c10 msgr2=0x7f916807a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 -- 172.21.15.57:0/1813575834 shutdown_connections 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.817+0000 7f916e400700 1 -- 172.21.15.57:0/1813575834 wait complete. 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 Processor -- start 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 -- start start 2024-04-13T05:16:12.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9168074ce0 0x7f916819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9168075600 0x7f916819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f916810f510 0x7f916819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9168114ec0 con 0x7f916810f510 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.818+0000 7f916e400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9168114d40 con 0x7f9168075600 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f916e400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9168115040 con 0x7f9168074ce0 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f916810f510 0x7f916819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f916810f510 0x7f916819fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57780/0 (socket says 172.21.15.57:57780) 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 -- 172.21.15.57:0/3132984244 learned_addr learned my addr 172.21.15.57:0/3132984244 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9168074ce0 0x7f916819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9166a00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9168075600 0x7f916819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 -- 172.21.15.57:0/3132984244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9168074ce0 msgr2=0x7f916819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:12.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9168074ce0 0x7f916819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 -- 172.21.15.57:0/3132984244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9168075600 msgr2=0x7f916819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9168075600 0x7f916819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167e00700 1 -- 172.21.15.57:0/3132984244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91681a0380 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.819+0000 7f9167400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9168074ce0 0x7f916819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.820+0000 7f9167e00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f916810f510 0x7f916819fc40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9160007ee0 tx=0x7f916000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.820+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916000ddd0 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.820+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91681a0670 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.820+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9160004bb0 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.820+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160015400 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.820+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91681a7ef0 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.821+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9168101750 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.824+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9160004d60 con 0x7f916810f510 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.825+0000 7f915fe00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f914c0a3d50 0x7f914c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:12.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.825+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f91600d0e10 con 0x7f916810f510 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.825+0000 7f9167400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f914c0a3d50 0x7f914c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.826+0000 7f915fe00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] conn(0x7f914c0aeeb0 0x7f914c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.826+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 --> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f916000be50 con 0x7f914c0aeeb0 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.826+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f91600d11c0 con 0x7f916810f510 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.826+0000 7f9166a00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] conn(0x7f914c0aeeb0 0x7f914c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.826+0000 7f9167400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f914c0a3d50 0x7f914c0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9158005950 tx=0x7f91580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:12.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.827+0000 7f9166a00700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] conn(0x7f914c0aeeb0 0x7f914c0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.827+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== osd.6 v2:172.21.15.57:6818/830219588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f916000be50 con 0x7f914c0aeeb0 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.846+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 --> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91680763a0 con 0x7f914c0aeeb0 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.846+0000 7f915fe00700 1 -- 172.21.15.57:0/3132984244 <== osd.6 v2:172.21.15.57:6818/830219588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91680763a0 con 0x7f914c0aeeb0 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] conn(0x7f914c0aeeb0 msgr2=0x7f914c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] conn(0x7f914c0aeeb0 0x7f914c0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f914c0a3d50 msgr2=0x7f914c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f914c0a3d50 0x7f914c0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9158005950 tx=0x7f91580058e0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f916810f510 msgr2=0x7f916819fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 --2- 172.21.15.57:0/3132984244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f916810f510 0x7f916819fc40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9160007ee0 tx=0x7f916000d3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f9167e00700 1 -- 172.21.15.57:0/3132984244 reap_dead start 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 shutdown_connections 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.847+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 >> 172.21.15.57:0/3132984244 conn(0x7f9168079c10 msgr2=0x7f9168100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:12.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.848+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 shutdown_connections 2024-04-13T05:16:12.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:10.848+0000 7f916e400700 1 -- 172.21.15.57:0/3132984244 wait complete. 2024-04-13T05:16:12.384 INFO:teuthology.orchestra.run.smithi057.stdout:163208757321 2024-04-13T05:16:12.384 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-13T05:16:12.895 INFO:teuthology.orchestra.run.smithi057.stdout:197568495684 2024-04-13T05:16:12.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-13T05:16:13.223 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:13 smithi057 ceph-mon[32331]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:16:13.224 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-13T05:16:13.224 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8, 3, 9]}]: dispatch 2024-04-13T05:16:13.224 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-13T05:16:13.276 INFO:teuthology.orchestra.run.smithi057.stdout:34359738475 2024-04-13T05:16:13.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-13T05:16:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:13 smithi083 ceph-mon[35133]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:16:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-13T05:16:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8, 3, 9]}]: dispatch 2024-04-13T05:16:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-13T05:16:13.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:13 smithi189 ceph-mon[35531]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:16:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-13T05:16:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8, 3, 9]}]: dispatch 2024-04-13T05:16:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-13T05:16:13.676 INFO:teuthology.orchestra.run.smithi057.stdout:77309411423 2024-04-13T05:16:13.677 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-13T05:16:13.969 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:13.983 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:13.984 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:13.992 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:14.011 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:14.022 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:14.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-13T05:16:14.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8, 3, 9]}]': finished 2024-04-13T05:16:14.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-13T05:16:14.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:14 smithi057 ceph-mon[32331]: osdmap e93: 12 total, 12 up, 12 in 2024-04-13T05:16:14.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-13T05:16:14.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8, 3, 9]}]': finished 2024-04-13T05:16:14.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-13T05:16:14.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:14 smithi083 ceph-mon[35133]: osdmap e93: 12 total, 12 up, 12 in 2024-04-13T05:16:14.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-13T05:16:14.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8, 3, 9]}]': finished 2024-04-13T05:16:14.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-13T05:16:14.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:14 smithi189 ceph-mon[35531]: osdmap e93: 12 total, 12 up, 12 in 2024-04-13T05:16:15.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:15 smithi057 ceph-mon[32331]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-13T05:16:15.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:15 smithi057 ceph-mon[32331]: osdmap e94: 12 total, 12 up, 12 in 2024-04-13T05:16:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:15 smithi083 ceph-mon[35133]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-13T05:16:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:15 smithi083 ceph-mon[35133]: osdmap e94: 12 total, 12 up, 12 in 2024-04-13T05:16:15.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:15 smithi189 ceph-mon[35531]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-13T05:16:15.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:15 smithi189 ceph-mon[35531]: osdmap e94: 12 total, 12 up, 12 in 2024-04-13T05:16:17.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:17 smithi083 ceph-mon[35133]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:17.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:17 smithi057 ceph-mon[32331]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:17.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:17 smithi189 ceph-mon[35531]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:18.295 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:19.325 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:19 smithi057 ceph-mon[32331]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-13T05:16:19.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:19 smithi083 ceph-mon[35133]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-13T05:16:19.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:19 smithi189 ceph-mon[35531]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-13T05:16:21.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:21 smithi083 ceph-mon[35133]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-13T05:16:21.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:21 smithi057 ceph-mon[32331]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-13T05:16:21.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:21 smithi189 ceph-mon[35531]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-13T05:16:23.395 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:23 smithi057 ceph-mon[32331]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-13T05:16:23.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:23 smithi083 ceph-mon[35133]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-13T05:16:23.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:23 smithi189 ceph-mon[35531]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-13T05:16:25.279 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:25 smithi057 ceph-mon[32331]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-13T05:16:25.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:25 smithi083 ceph-mon[35133]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-13T05:16:25.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:25 smithi189 ceph-mon[35531]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-13T05:16:27.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:27 smithi057 ceph-mon[32331]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-13T05:16:27.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:27 smithi189 ceph-mon[35531]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-13T05:16:27.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:27 smithi083 ceph-mon[35133]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-13T05:16:28.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:28 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1729913644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-13T05:16:28.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:28 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1729913644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-13T05:16:28.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 -- 172.21.15.57:0/989667318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4109fe0 msgr2=0x7f47d4111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 --2- 172.21.15.57:0/989667318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4109fe0 0x7f47d4111bb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00b0d0 tx=0x7f47cc00b3e0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 -- 172.21.15.57:0/989667318 shutdown_connections 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 --2- 172.21.15.57:0/989667318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4109fe0 0x7f47d4111bb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 --2- 172.21.15.57:0/989667318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d4104f00 0x7f47d41052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 --2- 172.21.15.57:0/989667318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47d41058b0 0x7f47d4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.397+0000 7f47d9e00700 1 -- 172.21.15.57:0/989667318 >> 172.21.15.57:0/989667318 conn(0x7f47d4100790 msgr2=0x7f47d4102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.398+0000 7f47d9e00700 1 -- 172.21.15.57:0/989667318 shutdown_connections 2024-04-13T05:16:28.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.398+0000 7f47d9e00700 1 -- 172.21.15.57:0/989667318 wait complete. 2024-04-13T05:16:28.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 Processor -- start 2024-04-13T05:16:28.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 -- start start 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 0x7f47d419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d41058b0 0x7f47d419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47d4109fe0 0x7f47d419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4114ec0 con 0x7f47d4104f00 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4114d40 con 0x7f47d41058b0 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.399+0000 7f47d9e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4115040 con 0x7f47d4109fe0 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 0x7f47d419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 0x7f47d419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46146/0 (socket says 172.21.15.57:46146) 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 -- 172.21.15.57:0/1895876965 learned_addr learned my addr 172.21.15.57:0/1895876965 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47cbe00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d41058b0 0x7f47d419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:28.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d3400700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47d4109fe0 0x7f47d419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:28.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 -- 172.21.15.57:0/1895876965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47d4109fe0 msgr2=0x7f47d419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:28.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47d4109fe0 0x7f47d419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 -- 172.21.15.57:0/1895876965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d41058b0 msgr2=0x7f47d419b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:28.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d41058b0 0x7f47d419b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47d2a00700 1 -- 172.21.15.57:0/1895876965 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d41a0350 con 0x7f47d4104f00 2024-04-13T05:16:28.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.400+0000 7f47cbe00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d41058b0 0x7f47d419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.401+0000 7f47d2a00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 0x7f47d419b390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f47c400d950 tx=0x7f47c400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.401+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4016070 con 0x7f47d4104f00 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.401+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d41a0640 con 0x7f47d4104f00 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.401+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47c4010460 con 0x7f47d4104f00 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.401+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400f620 con 0x7f47d4104f00 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.401+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d41a7ec0 con 0x7f47d4104f00 2024-04-13T05:16:28.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.402+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d40697a0 con 0x7f47d4104f00 2024-04-13T05:16:28.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.405+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f47c4019070 con 0x7f47d4104f00 2024-04-13T05:16:28.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.407+0000 7f47d8a00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47b80a3cd0 0x7f47b80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:28.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.408+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f47c40d05d0 con 0x7f47d4104f00 2024-04-13T05:16:28.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.408+0000 7f47cbe00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47b80a3cd0 0x7f47b80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:28.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.409+0000 7f47cbe00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47b80a3cd0 0x7f47b80a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47d4100e30 tx=0x7f47bc005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:28.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.411+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47c409ca50 con 0x7f47d4104f00 2024-04-13T05:16:28.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.651+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f47d402dfa0 con 0x7f47d4104f00 2024-04-13T05:16:28.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.652+0000 7f47d8a00700 1 -- 172.21.15.57:0/1895876965 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f47c409c350 con 0x7f47d4104f00 2024-04-13T05:16:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:163208757325 2024-04-13T05:16:28.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47b80a3cd0 msgr2=0x7f47b80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:28.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47b80a3cd0 0x7f47b80a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47d4100e30 tx=0x7f47bc005d80 comp rx=0 tx=0).stop 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 msgr2=0x7f47d419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 0x7f47d419b390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f47c400d950 tx=0x7f47c400dd10 comp rx=0 tx=0).stop 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 shutdown_connections 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47d4109fe0 0x7f47d419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47b80a3cd0 0x7f47b80a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47d4104f00 0x7f47d419b390 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.655+0000 7f47d9e00700 1 --2- 172.21.15.57:0/1895876965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d41058b0 0x7f47d419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.656+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 >> 172.21.15.57:0/1895876965 conn(0x7f47d4100790 msgr2=0x7f47d4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.656+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 shutdown_connections 2024-04-13T05:16:28.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:28.656+0000 7f47d9e00700 1 -- 172.21.15.57:0/1895876965 wait complete. 2024-04-13T05:16:28.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:28 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1729913644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-13T05:16:29.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:29 smithi057 ceph-mon[32331]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:29.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:29 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1895876965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-13T05:16:29.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:29 smithi189 ceph-mon[35531]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:29.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:29 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1895876965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-13T05:16:29.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:29 smithi083 ceph-mon[35133]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:29.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:29 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1895876965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 -- 172.21.15.57:0/287664286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa46810a010 msgr2=0x7fa468111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 --2- 172.21.15.57:0/287664286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa46810a010 0x7fa468111be0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa46000b0d0 tx=0x7fa46000b3e0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 -- 172.21.15.57:0/287664286 shutdown_connections 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 --2- 172.21.15.57:0/287664286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa46810a010 0x7fa468111be0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 --2- 172.21.15.57:0/287664286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa468104f30 0x7fa468105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 --2- 172.21.15.57:0/287664286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4681058e0 0x7fa468109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.552+0000 7fa46f200700 1 -- 172.21.15.57:0/287664286 >> 172.21.15.57:0/287664286 conn(0x7fa4681007c0 msgr2=0x7fa468102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:29.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.553+0000 7fa46f200700 1 -- 172.21.15.57:0/287664286 shutdown_connections 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.553+0000 7fa46f200700 1 -- 172.21.15.57:0/287664286 wait complete. 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 Processor -- start 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 -- start start 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 0x7fa46819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4681058e0 0x7fa46819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa46810a010 0x7fa46819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa468114f20 con 0x7fa468104f30 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa468114da0 con 0x7fa4681058e0 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa46f200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681150a0 con 0x7fa46810a010 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.554+0000 7fa467e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 0x7fa46819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 0x7fa46819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46166/0 (socket says 172.21.15.57:46166) 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 -- 172.21.15.57:0/994366572 learned_addr learned my addr 172.21.15.57:0/994366572 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467400700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4681058e0 0x7fa46819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:29.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa46d200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa46810a010 0x7fa46819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 -- 172.21.15.57:0/994366572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa46810a010 msgr2=0x7fa46819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa46810a010 0x7fa46819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 -- 172.21.15.57:0/994366572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4681058e0 msgr2=0x7fa46819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4681058e0 0x7fa46819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 -- 172.21.15.57:0/994366572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4681a0340 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.555+0000 7fa467e00700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 0x7fa46819b330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d950 tx=0x7fa45800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.556+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458016070 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.556+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa458010460 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.556+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4681a0630 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.556+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45800f620 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.556+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681a7eb0 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.557+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4680697a0 con 0x7fa468104f30 2024-04-13T05:16:29.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.559+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa458019070 con 0x7fa468104f30 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.561+0000 7fa464c00700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa44c0a3d50 0x7fa44c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.562+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa4580d0550 con 0x7fa468104f30 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.562+0000 7fa467400700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa44c0a3d50 0x7fa44c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.562+0000 7fa467400700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa44c0a3d50 0x7fa44c0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa454006fd0 tx=0x7fa454008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.567+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa45809c9d0 con 0x7fa468104f30 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.824+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa4681a80b0 con 0x7fa468104f30 2024-04-13T05:16:29.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.825+0000 7fa464c00700 1 -- 172.21.15.57:0/994366572 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa45809c2d0 con 0x7fa468104f30 2024-04-13T05:16:29.827 INFO:teuthology.orchestra.run.smithi057.stdout:34359738479 2024-04-13T05:16:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.827+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa44c0a3d50 msgr2=0x7fa44c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.827+0000 7fa46f200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa44c0a3d50 0x7fa44c0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa454006fd0 tx=0x7fa454008040 comp rx=0 tx=0).stop 2024-04-13T05:16:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.827+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 msgr2=0x7fa46819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.827+0000 7fa46f200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 0x7fa46819b330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d950 tx=0x7fa45800dd10 comp rx=0 tx=0).stop 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 shutdown_connections 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa46810a010 0x7fa46819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa44c0a3d50 0x7fa44c0a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa468104f30 0x7fa46819b330 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 --2- 172.21.15.57:0/994366572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4681058e0 0x7fa46819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 >> 172.21.15.57:0/994366572 conn(0x7fa4681007c0 msgr2=0x7fa4681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 shutdown_connections 2024-04-13T05:16:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:29.828+0000 7fa46f200700 1 -- 172.21.15.57:0/994366572 wait complete. 2024-04-13T05:16:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 -- 172.21.15.57:0/2421630345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 msgr2=0x7f4318109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 --2- 172.21.15.57:0/2421630345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f4318109950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f43040099c0 tx=0x7f4304009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 -- 172.21.15.57:0/2421630345 shutdown_connections 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 --2- 172.21.15.57:0/2421630345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f431810a030 0x7f4318111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 --2- 172.21.15.57:0/2421630345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4318104f50 0x7f4318105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 --2- 172.21.15.57:0/2421630345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f4318109950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 -- 172.21.15.57:0/2421630345 >> 172.21.15.57:0/2421630345 conn(0x7f43181007e0 msgr2=0x7f4318102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 -- 172.21.15.57:0/2421630345 shutdown_connections 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.121+0000 7f431f600700 1 -- 172.21.15.57:0/2421630345 wait complete. 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.122+0000 7f431f600700 1 Processor -- start 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.122+0000 7f431f600700 1 -- start start 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431f600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4318104f50 0x7f431819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431f600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f431819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:30.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431f600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f431810a030 0x7f431819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431f600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4318114ef0 con 0x7f4318105900 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431f600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4318114d70 con 0x7f4318104f50 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431f600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4318115070 con 0x7f431810a030 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f4317e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f431819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f4317e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f431819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46178/0 (socket says 172.21.15.57:46178) 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f4317e00700 1 -- 172.21.15.57:0/3729995785 learned_addr learned my addr 172.21.15.57:0/3729995785 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431d600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f431810a030 0x7f431819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.123+0000 7f431cc00700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4318104f50 0x7f431819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4317e00700 1 -- 172.21.15.57:0/3729995785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f431810a030 msgr2=0x7f431819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4317e00700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f431810a030 0x7f431819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4317e00700 1 -- 172.21.15.57:0/3729995785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4318104f50 msgr2=0x7f431819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4317e00700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4318104f50 0x7f431819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4317e00700 1 -- 172.21.15.57:0/3729995785 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181a03d0 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4317e00700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f431819b900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f430400c010 tx=0x7f43040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304004830 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4304005020 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.124+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181a0660 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.125+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304003e20 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.125+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a7f40 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.126+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4318106430 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.128+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4304005540 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.129+0000 7f4315600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43000a3d50 0x7f43000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.129+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f43040d1880 con 0x7f4318105900 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.130+0000 7f431cc00700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43000a3d50 0x7f43000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.130+0000 7f431cc00700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43000a3d50 0x7f43000a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f430c005950 tx=0x7f430c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:30.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.132+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4304013070 con 0x7f4318105900 2024-04-13T05:16:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.370+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f431802dfa0 con 0x7f4318105900 2024-04-13T05:16:30.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.371+0000 7f4315600700 1 -- 172.21.15.57:0/3729995785 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f430409dd90 con 0x7f4318105900 2024-04-13T05:16:30.373 INFO:teuthology.orchestra.run.smithi057.stdout:107374182491 2024-04-13T05:16:30.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:30 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/994366572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-13T05:16:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43000a3d50 msgr2=0x7f43000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43000a3d50 0x7f43000a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f430c005950 tx=0x7f430c0058e0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 msgr2=0x7f431819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f431819b900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f430400c010 tx=0x7f43040047b0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 shutdown_connections 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f431810a030 0x7f431819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43000a3d50 0x7f43000a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4318104f50 0x7f431819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 --2- 172.21.15.57:0/3729995785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4318105900 0x7f431819b900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 >> 172.21.15.57:0/3729995785 conn(0x7f43181007e0 msgr2=0x7f4318101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 shutdown_connections 2024-04-13T05:16:30.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.375+0000 7f431f600700 1 -- 172.21.15.57:0/3729995785 wait complete. 2024-04-13T05:16:30.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:30 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/994366572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-13T05:16:30.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:30 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/994366572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-13T05:16:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 -- 172.21.15.57:0/3238991346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 msgr2=0x7f8d6406b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 --2- 172.21.15.57:0/3238991346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d6406b4b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009990 tx=0x7f8d50009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 -- 172.21.15.57:0/3238991346 shutdown_connections 2024-04-13T05:16:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 --2- 172.21.15.57:0/3238991346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d6406b9f0 0x7f8d6410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 --2- 172.21.15.57:0/3238991346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d64108a90 0x7f8d64108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 --2- 172.21.15.57:0/3238991346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d6406b4b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 -- 172.21.15.57:0/3238991346 >> 172.21.15.57:0/3238991346 conn(0x7f8d64078920 msgr2=0x7f8d64078d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 -- 172.21.15.57:0/3238991346 shutdown_connections 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.969+0000 7f8d6c000700 1 -- 172.21.15.57:0/3238991346 wait complete. 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.970+0000 7f8d6c000700 1 Processor -- start 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.970+0000 7f8d6c000700 1 -- start start 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d6c000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d64199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d6c000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6406b9f0 0x7f8d641996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d6c000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d64108a90 0x7f8d6419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d6c000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64112c70 con 0x7f8d6406b050 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d6c000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64112af0 con 0x7f8d6406b9f0 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d6c000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64112df0 con 0x7f8d64108a90 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d69600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d64199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d69600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d64199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46190/0 (socket says 172.21.15.57:46190) 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.971+0000 7f8d69600700 1 -- 172.21.15.57:0/2641569069 learned_addr learned my addr 172.21.15.57:0/2641569069 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d68c00700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6406b9f0 0x7f8d641996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d6a000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d64108a90 0x7f8d6419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d69600700 1 -- 172.21.15.57:0/2641569069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d64108a90 msgr2=0x7f8d6419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d69600700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d64108a90 0x7f8d6419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d69600700 1 -- 172.21.15.57:0/2641569069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6406b9f0 msgr2=0x7f8d641996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d69600700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6406b9f0 0x7f8d641996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d69600700 1 -- 172.21.15.57:0/2641569069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6419e190 con 0x7f8d6406b050 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d69600700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d64199180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c920 tx=0x7f8d5800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58007b00 con 0x7f8d6406b050 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.972+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d58004ce0 con 0x7f8d6406b050 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.973+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6419e480 con 0x7f8d6406b050 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.973+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58005600 con 0x7f8d6406b050 2024-04-13T05:16:31.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.973+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d641019a0 con 0x7f8d6406b050 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.974+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d640697a0 con 0x7f8d6406b050 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.976+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d58007ca0 con 0x7f8d6406b050 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.977+0000 7f8d62000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d4c0a3cd0 0x7f8d4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.977+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8d580d0910 con 0x7f8d6406b050 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.978+0000 7f8d68c00700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d4c0a3cd0 0x7f8d4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.978+0000 7f8d68c00700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d4c0a3cd0 0x7f8d4c0a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8d50006010 tx=0x7f8d5001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:31.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:30.981+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d58014030 con 0x7f8d6406b050 2024-04-13T05:16:31.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.225+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8d6419a4e0 con 0x7f8d6406b050 2024-04-13T05:16:31.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.226+0000 7f8d62000700 1 -- 172.21.15.57:0/2641569069 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8d5809cd90 con 0x7f8d6406b050 2024-04-13T05:16:31.228 INFO:teuthology.orchestra.run.smithi057.stdout:55834574953 2024-04-13T05:16:31.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d4c0a3cd0 msgr2=0x7f8d4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d4c0a3cd0 0x7f8d4c0a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8d50006010 tx=0x7f8d5001a040 comp rx=0 tx=0).stop 2024-04-13T05:16:31.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 msgr2=0x7f8d64199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d64199180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c920 tx=0x7f8d5800cce0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 shutdown_connections 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d64108a90 0x7f8d6419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8d4c0a3cd0 0x7f8d4c0a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d6406b050 0x7f8d64199180 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 --2- 172.21.15.57:0/2641569069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d6406b9f0 0x7f8d641996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.229+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 >> 172.21.15.57:0/2641569069 conn(0x7f8d64078920 msgr2=0x7f8d640ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.230+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 shutdown_connections 2024-04-13T05:16:31.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:31.230+0000 7f8d6c000700 1 -- 172.21.15.57:0/2641569069 wait complete. 2024-04-13T05:16:31.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:31 smithi057 ceph-mon[32331]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:31.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:31 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3729995785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-13T05:16:31.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:31 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2641569069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-13T05:16:31.559 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182484 got 107374182491 for osd.3 2024-04-13T05:16:31.559 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:31.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:31 smithi189 ceph-mon[35531]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:31.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:31 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3729995785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-13T05:16:31.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:31 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2641569069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-13T05:16:31.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 -- 172.21.15.57:0/2123405609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109cf0 msgr2=0x7f9d5810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 --2- 172.21.15.57:0/2123405609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109cf0 0x7f9d5810dd40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d440099c0 tx=0x7f9d44009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 -- 172.21.15.57:0/2123405609 shutdown_connections 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 --2- 172.21.15.57:0/2123405609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d58109340 0x7f9d58109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 --2- 172.21.15.57:0/2123405609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d5810e420 0x7f9d58115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 --2- 172.21.15.57:0/2123405609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109cf0 0x7f9d5810dd40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.527+0000 7f9d5f000700 1 -- 172.21.15.57:0/2123405609 >> 172.21.15.57:0/2123405609 conn(0x7f9d58076280 msgr2=0x7f9d581078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.528+0000 7f9d5f000700 1 -- 172.21.15.57:0/2123405609 shutdown_connections 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.528+0000 7f9d5f000700 1 -- 172.21.15.57:0/2123405609 wait complete. 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 Processor -- start 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 -- start start 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 0x7f9d581a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d58109cf0 0x7f9d581a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5810e420 0x7f9d5819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58119300 con 0x7f9d58109340 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58119180 con 0x7f9d58109cf0 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.529+0000 7f9d5f000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58119480 con 0x7f9d5810e420 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 0x7f9d581a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 0x7f9d581a5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46130/0 (socket says 172.21.15.57:46130) 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 -- 172.21.15.57:0/1729913644 learned_addr learned my addr 172.21.15.57:0/1729913644 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d5d000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5810e420 0x7f9d5819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57400700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d58109cf0 0x7f9d581a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 -- 172.21.15.57:0/1729913644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5810e420 msgr2=0x7f9d5819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5810e420 0x7f9d5819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 -- 172.21.15.57:0/1729913644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d58109cf0 msgr2=0x7f9d581a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d58109cf0 0x7f9d581a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57e00700 1 -- 172.21.15.57:0/1729913644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5819ff40 con 0x7f9d58109340 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d5d000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5810e420 0x7f9d5819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:16:31.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.530+0000 7f9d57400700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d58109cf0 0x7f9d581a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.531+0000 7f9d57e00700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 0x7f9d581a5590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800d980 tx=0x7f9d4800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.531+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48017070 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.531+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d48004830 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.531+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48005440 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.531+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d581a0230 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.531+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d581ac510 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.532+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5810a820 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.534+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d480055e0 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.536+0000 7f9d54c00700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d3c0a3cd0 0x7f9d3c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.537+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9d480d08e0 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.537+0000 7f9d57400700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d3c0a3cd0 0x7f9d3c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.537+0000 7f9d57400700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d3c0a3cd0 0x7f9d3c0a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c00 tx=0x7f9d44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.541+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d48020020 con 0x7f9d58109340 2024-04-13T05:16:31.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.800+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9d581a0460 con 0x7f9d58109340 2024-04-13T05:16:31.710 INFO:teuthology.orchestra.run.smithi057.stdout:253403070520 2024-04-13T05:16:31.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.801+0000 7f9d54c00700 1 -- 172.21.15.57:0/1729913644 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d4809cd60 con 0x7f9d58109340 2024-04-13T05:16:31.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d3c0a3cd0 msgr2=0x7f9d3c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d3c0a3cd0 0x7f9d3c0a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c00 tx=0x7f9d44005fb0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 msgr2=0x7f9d581a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 0x7f9d581a5590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800d980 tx=0x7f9d4800dd40 comp rx=0 tx=0).stop 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 shutdown_connections 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5810e420 0x7f9d5819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d3c0a3cd0 0x7f9d3c0a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d58109340 0x7f9d581a5590 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 --2- 172.21.15.57:0/1729913644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d58109cf0 0x7f9d581a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 >> 172.21.15.57:0/1729913644 conn(0x7f9d58076280 msgr2=0x7f9d58076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 shutdown_connections 2024-04-13T05:16:31.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:27.804+0000 7f9d5f000700 1 -- 172.21.15.57:0/1729913644 wait complete. 2024-04-13T05:16:31.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:31 smithi083 ceph-mon[35133]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:31.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:31 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3729995785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-13T05:16:31.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:31 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2641569069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-13T05:16:33.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:33 smithi057 ceph-mon[32331]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:33.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 -- 172.21.15.57:0/891914895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 msgr2=0x7f05f0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:33.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 --2- 172.21.15.57:0/891914895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f0108ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009990 tx=0x7f05e0009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 -- 172.21.15.57:0/891914895 shutdown_connections 2024-04-13T05:16:33.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 --2- 172.21.15.57:0/891914895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f01034c0 0x7f05f010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 --2- 172.21.15.57:0/891914895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f0108ea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 --2- 172.21.15.57:0/891914895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f0102b00 0x7f05f0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.528+0000 7f05f4a00700 1 -- 172.21.15.57:0/891914895 >> 172.21.15.57:0/891914895 conn(0x7f05f0079c10 msgr2=0x7f05f007a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.529+0000 7f05f4a00700 1 -- 172.21.15.57:0/891914895 shutdown_connections 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.529+0000 7f05f4a00700 1 -- 172.21.15.57:0/891914895 wait complete. 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.530+0000 7f05f4a00700 1 Processor -- start 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.530+0000 7f05f4a00700 1 -- start start 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.530+0000 7f05f4a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f0102b00 0x7f05f019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.530+0000 7f05f4a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f01034c0 0x7f05f019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.530+0000 7f05f4a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05f4a00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0112ca0 con 0x7f05f0108ac0 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05f4a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0112b20 con 0x7f05f0102b00 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05f4a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0112e20 con 0x7f05f01034c0 2024-04-13T05:16:33.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46200/0 (socket says 172.21.15.57:46200) 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 -- 172.21.15.57:0/1622371011 learned_addr learned my addr 172.21.15.57:0/1622371011 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ede00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f0102b00 0x7f05f019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ed400700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f01034c0 0x7f05f019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 -- 172.21.15.57:0/1622371011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f01034c0 msgr2=0x7f05f019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f01034c0 0x7f05f019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 -- 172.21.15.57:0/1622371011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f0102b00 msgr2=0x7f05f019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f0102b00 0x7f05f019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.531+0000 7f05ee800700 1 -- 172.21.15.57:0/1622371011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01a0480 con 0x7f05f0108ac0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.532+0000 7f05ee800700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f019fd40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f05e8007ee0 tx=0x7f05e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.532+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e800dbd0 con 0x7f05f0108ac0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.532+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e800dd70 con 0x7f05f0108ac0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.532+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01a0770 con 0x7f05f0108ac0 2024-04-13T05:16:33.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.532+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8005990 con 0x7f05f0108ac0 2024-04-13T05:16:33.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.532+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01a7ff0 con 0x7f05f0108ac0 2024-04-13T05:16:33.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.534+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f00697a0 con 0x7f05f0108ac0 2024-04-13T05:16:33.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.538+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f05e801a020 con 0x7f05f0108ac0 2024-04-13T05:16:33.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.539+0000 7f05e6a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05d40a3d50 0x7f05d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:33.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.539+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f05e80d0620 con 0x7f05f0108ac0 2024-04-13T05:16:33.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.539+0000 7f05ede00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05d40a3d50 0x7f05d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:33.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.540+0000 7f05ede00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05d40a3d50 0x7f05d40a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b5c0 tx=0x7f05e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:33.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.542+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e8010070 con 0x7f05f0108ac0 2024-04-13T05:16:33.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:33 smithi189 ceph-mon[35531]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:33.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:33 smithi083 ceph-mon[35133]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:33.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.785+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f05f002dfa0 con 0x7f05f0108ac0 2024-04-13T05:16:33.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.786+0000 7f05e6a00700 1 -- 172.21.15.57:0/1622371011 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f05e809caa0 con 0x7f05f0108ac0 2024-04-13T05:16:33.788 INFO:teuthology.orchestra.run.smithi057.stdout:313532612650 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05d40a3d50 msgr2=0x7f05d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05d40a3d50 0x7f05d40a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b5c0 tx=0x7f05e0005fd0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 msgr2=0x7f05f019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f019fd40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f05e8007ee0 tx=0x7f05e800d3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 shutdown_connections 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f05d40a3d50 0x7f05d40a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0108ac0 0x7f05f019fd40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f0102b00 0x7f05f019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 --2- 172.21.15.57:0/1622371011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f01034c0 0x7f05f019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:33.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.788+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 >> 172.21.15.57:0/1622371011 conn(0x7f05f0079c10 msgr2=0x7f05f010dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:33.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.789+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 shutdown_connections 2024-04-13T05:16:33.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.789+0000 7f05f4a00700 1 -- 172.21.15.57:0/1622371011 wait complete. 2024-04-13T05:16:33.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757321 got 163208757325 for osd.5 2024-04-13T05:16:33.973 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:34 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1622371011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-13T05:16:34.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:34 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3444990487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-13T05:16:34.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:34 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1622371011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-13T05:16:34.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:34 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3444990487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-13T05:16:34.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:34 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1622371011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-13T05:16:34.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:34 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3444990487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-13T05:16:35.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070515 got 253403070520 for osd.8 2024-04-13T05:16:35.274 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:35.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:35 smithi057 ceph-mon[32331]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:35.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:35 smithi189 ceph-mon[35531]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:35.685 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738475 got 34359738479 for osd.0 2024-04-13T05:16:35.685 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:35.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:35 smithi083 ceph-mon[35133]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:16:36.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 -- 172.21.15.57:0/3798345587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c1019e0 msgr2=0x7fcb7c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 --2- 172.21.15.57:0/3798345587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c1019e0 0x7fcb7c101dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009990 tx=0x7fcb6c009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 -- 172.21.15.57:0/3798345587 shutdown_connections 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 --2- 172.21.15.57:0/3798345587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7c106df0 0x7fcb7c1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 --2- 172.21.15.57:0/3798345587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c1019e0 0x7fcb7c101dc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 --2- 172.21.15.57:0/3798345587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7c102300 0x7fcb7c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 -- 172.21.15.57:0/3798345587 >> 172.21.15.57:0/3798345587 conn(0x7fcb7c0fd440 msgr2=0x7fcb7c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 -- 172.21.15.57:0/3798345587 shutdown_connections 2024-04-13T05:16:36.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.080+0000 7fcb81800700 1 -- 172.21.15.57:0/3798345587 wait complete. 2024-04-13T05:16:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.081+0000 7fcb81800700 1 Processor -- start 2024-04-13T05:16:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 -- start start 2024-04-13T05:16:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7c1019e0 0x7fcb7c1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 0x7fcb7c1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7c106df0 0x7fcb7c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c119240 con 0x7fcb7c102300 2024-04-13T05:16:36.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c1190c0 con 0x7fcb7c1019e0 2024-04-13T05:16:36.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb81800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c1193c0 con 0x7fcb7c106df0 2024-04-13T05:16:36.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.082+0000 7fcb7b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 0x7fcb7c1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 0x7fcb7c1a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36234/0 (socket says 172.21.15.57:36234) 2024-04-13T05:16:36.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 -- 172.21.15.57:0/3782919303 learned_addr learned my addr 172.21.15.57:0/3782919303 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:36.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7be00700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7c1019e0 0x7fcb7c1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb80e00700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7c106df0 0x7fcb7c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 -- 172.21.15.57:0/3782919303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7c106df0 msgr2=0x7fcb7c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7c106df0 0x7fcb7c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 -- 172.21.15.57:0/3782919303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7c1019e0 msgr2=0x7fcb7c1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7c1019e0 0x7fcb7c1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.083+0000 7fcb7b400700 1 -- 172.21.15.57:0/3782919303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c19fd40 con 0x7fcb7c102300 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.084+0000 7fcb7b400700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 0x7fcb7c1a5960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c970 tx=0x7fcb6800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.084+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68004a30 con 0x7fcb7c102300 2024-04-13T05:16:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.084+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb680054d0 con 0x7fcb7c102300 2024-04-13T05:16:36.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.084+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68007bc0 con 0x7fcb7c102300 2024-04-13T05:16:36.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.084+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c1a0030 con 0x7fcb7c102300 2024-04-13T05:16:36.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.084+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1ac2f0 con 0x7fcb7c102300 2024-04-13T05:16:36.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.085+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c0697a0 con 0x7fcb7c102300 2024-04-13T05:16:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.087+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb68004bd0 con 0x7fcb7c102300 2024-04-13T05:16:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.089+0000 7fcb78c00700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcb600a3d50 0x7fcb600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.089+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcb680d0430 con 0x7fcb7c102300 2024-04-13T05:16:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.089+0000 7fcb7be00700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcb600a3d50 0x7fcb600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.090+0000 7fcb7be00700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcb600a3d50 0x7fcb600a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00b5c0 tx=0x7fcb6c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:36.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.093+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb68014030 con 0x7fcb7c102300 2024-04-13T05:16:36.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.333+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcb7c1ac580 con 0x7fcb7c102300 2024-04-13T05:16:36.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.334+0000 7fcb78c00700 1 -- 172.21.15.57:0/3782919303 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcb6809c910 con 0x7fcb7c102300 2024-04-13T05:16:36.336 INFO:teuthology.orchestra.run.smithi057.stdout:347892351014 2024-04-13T05:16:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.336+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcb600a3d50 msgr2=0x7fcb600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.336+0000 7fcb81800700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcb600a3d50 0x7fcb600a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00b5c0 tx=0x7fcb6c005960 comp rx=0 tx=0).stop 2024-04-13T05:16:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.336+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 msgr2=0x7fcb7c1a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.336+0000 7fcb81800700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 0x7fcb7c1a5960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c970 tx=0x7fcb6800cc80 comp rx=0 tx=0).stop 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.336+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 shutdown_connections 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7c106df0 0x7fcb7c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcb600a3d50 0x7fcb600a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb7c1019e0 0x7fcb7c1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 --2- 172.21.15.57:0/3782919303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7c102300 0x7fcb7c1a5960 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 >> 172.21.15.57:0/3782919303 conn(0x7fcb7c0fd440 msgr2=0x7fcb7c1105b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 shutdown_connections 2024-04-13T05:16:36.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.337+0000 7fcb81800700 1 -- 172.21.15.57:0/3782919303 wait complete. 2024-04-13T05:16:36.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 -- 172.21.15.57:0/2261880275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 msgr2=0x7f49cc10f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 --2- 172.21.15.57:0/2261880275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc10f990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f49c400b0a0 tx=0x7f49c400b3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 -- 172.21.15.57:0/2261880275 shutdown_connections 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 --2- 172.21.15.57:0/2261880275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 0x7f49cc107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 --2- 172.21.15.57:0/2261880275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc10f990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 --2- 172.21.15.57:0/2261880275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 0x7f49cc102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 -- 172.21.15.57:0/2261880275 >> 172.21.15.57:0/2261880275 conn(0x7f49cc079c10 msgr2=0x7f49cc07a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 -- 172.21.15.57:0/2261880275 shutdown_connections 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.431+0000 7f49d1200700 1 -- 172.21.15.57:0/2261880275 wait complete. 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.432+0000 7f49d1200700 1 Processor -- start 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 -- start start 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 0x7f49cc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 0x7f49cc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc114f60 con 0x7f49cc103520 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc114de0 con 0x7f49cc102b60 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49d1200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1150e0 con 0x7f49cc107a40 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49c9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49c9400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36254/0 (socket says 172.21.15.57:36254) 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.433+0000 7f49c9400700 1 -- 172.21.15.57:0/3535416971 learned_addr learned my addr 172.21.15.57:0/3535416971 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9e00700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 0x7f49cc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49ca800700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 0x7f49cc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9400700 1 -- 172.21.15.57:0/3535416971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 msgr2=0x7f49cc19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9400700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 0x7f49cc19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9400700 1 -- 172.21.15.57:0/3535416971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 msgr2=0x7f49cc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9400700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 0x7f49cc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9400700 1 -- 172.21.15.57:0/3535416971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc1a0340 con 0x7f49cc103520 2024-04-13T05:16:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49ca800700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 0x7f49cc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9e00700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 0x7f49cc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c9400700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc19b870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d950 tx=0x7f49b800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.434+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8016070 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.435+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc1a0630 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.435+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49b8010460 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.435+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b800f620 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.435+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1a7eb0 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.436+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc0697a0 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.439+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f49b8019070 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.440+0000 7f49c2a00700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49b00a3d50 0x7f49b00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.440+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49b80d08f0 con 0x7f49cc103520 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.440+0000 7f49c9e00700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49b00a3d50 0x7f49b00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.441+0000 7f49c9e00700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49b00a3d50 0x7f49b00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005e10 tx=0x7f49bc005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:36.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.442+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49b809cd70 con 0x7f49cc103520 2024-04-13T05:16:36.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.682+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f49cc02dfa0 con 0x7f49cc103520 2024-04-13T05:16:36.709 INFO:teuthology.orchestra.run.smithi057.stdout:141733920855 2024-04-13T05:16:36.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.683+0000 7f49c2a00700 1 -- 172.21.15.57:0/3535416971 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49b809c670 con 0x7f49cc103520 2024-04-13T05:16:36.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49b00a3d50 msgr2=0x7f49b00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49b00a3d50 0x7f49b00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005e10 tx=0x7f49bc005d60 comp rx=0 tx=0).stop 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 msgr2=0x7f49cc19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc19b870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d950 tx=0x7f49b800dd10 comp rx=0 tx=0).stop 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 shutdown_connections 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49cc107a40 0x7f49cc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49b00a3d50 0x7f49b00a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49cc102b60 0x7f49cc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 --2- 172.21.15.57:0/3535416971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49cc103520 0x7f49cc19b870 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.686+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 >> 172.21.15.57:0/3535416971 conn(0x7f49cc079c10 msgr2=0x7f49cc100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.687+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 shutdown_connections 2024-04-13T05:16:36.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:36.687+0000 7f49d1200700 1 -- 172.21.15.57:0/3535416971 wait complete. 2024-04-13T05:16:37.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 -- 172.21.15.57:0/2494771747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4014104f30 msgr2=0x7f4014105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 --2- 172.21.15.57:0/2494771747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4014104f30 0x7f4014105310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f40040099c0 tx=0x7f4004009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 -- 172.21.15.57:0/2494771747 shutdown_connections 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 --2- 172.21.15.57:0/2494771747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f401410a010 0x7f4014111be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 --2- 172.21.15.57:0/2494771747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4014104f30 0x7f4014105310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 --2- 172.21.15.57:0/2494771747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40141058e0 0x7f4014109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 -- 172.21.15.57:0/2494771747 >> 172.21.15.57:0/2494771747 conn(0x7f40141007c0 msgr2=0x7f4014102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 -- 172.21.15.57:0/2494771747 shutdown_connections 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.140+0000 7f4019c00700 1 -- 172.21.15.57:0/2494771747 wait complete. 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.141+0000 7f4019c00700 1 Processor -- start 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.141+0000 7f4019c00700 1 -- start start 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4019c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4014104f30 0x7f401419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4019c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40141058e0 0x7f401419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4019c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 0x7f401419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4019c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4014114ef0 con 0x7f401410a010 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4019c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4014114d70 con 0x7f4014104f30 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4019c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4014115070 con 0x7f40141058e0 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4013400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 0x7f401419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4013400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 0x7f401419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36282/0 (socket says 172.21.15.57:36282) 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4013400700 1 -- 172.21.15.57:0/394686611 learned_addr learned my addr 172.21.15.57:0/394686611 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4012a00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4014104f30 0x7f401419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.142+0000 7f4012000700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40141058e0 0x7f401419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4013400700 1 -- 172.21.15.57:0/394686611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40141058e0 msgr2=0x7f401419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4013400700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40141058e0 0x7f401419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4013400700 1 -- 172.21.15.57:0/394686611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4014104f30 msgr2=0x7f401419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4013400700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4014104f30 0x7f401419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4013400700 1 -- 172.21.15.57:0/394686611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141a03d0 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4013400700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 0x7f401419fc90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f400c007ee0 tx=0x7f400c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c00dbd0 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f400c00dd70 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c005990 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.143+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a06c0 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.144+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a7f40 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.145+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40140697a0 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.147+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f400c01a020 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.149+0000 7f400b400700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3ff80a3d50 0x7f3ff80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.149+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f400c0cff90 con 0x7f401410a010 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.149+0000 7f4012a00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3ff80a3d50 0x7f3ff80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.150+0000 7f4012a00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3ff80a3d50 0x7f3ff80a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4004000c00 tx=0x7f4004005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:37.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.154+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f400c010070 con 0x7f401410a010 2024-04-13T05:16:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.394+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f401402e160 con 0x7f401410a010 2024-04-13T05:16:37.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.395+0000 7f400b400700 1 -- 172.21.15.57:0/394686611 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f400c09c410 con 0x7f401410a010 2024-04-13T05:16:37.398 INFO:teuthology.orchestra.run.smithi057.stdout:197568495689 2024-04-13T05:16:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3ff80a3d50 msgr2=0x7f3ff80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3ff80a3d50 0x7f3ff80a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4004000c00 tx=0x7f4004005fb0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 msgr2=0x7f401419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 0x7f401419fc90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f400c007ee0 tx=0x7f400c00d3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 shutdown_connections 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3ff80a3d50 0x7f3ff80a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f401410a010 0x7f401419fc90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4014104f30 0x7f401419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 --2- 172.21.15.57:0/394686611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40141058e0 0x7f401419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 >> 172.21.15.57:0/394686611 conn(0x7f40141007c0 msgr2=0x7f4014102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 shutdown_connections 2024-04-13T05:16:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:37.398+0000 7f4019c00700 1 -- 172.21.15.57:0/394686611 wait complete. 2024-04-13T05:16:37.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:37 smithi083 ceph-mon[35133]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:16:37.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:37 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3782919303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-13T05:16:37.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:37 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3535416971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-13T05:16:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:37 smithi057 ceph-mon[32331]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:16:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:37 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3782919303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-13T05:16:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:37 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3535416971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-13T05:16:37.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:37 smithi189 ceph-mon[35531]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:16:37.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:37 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3782919303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-13T05:16:37.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:37 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3535416971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-13T05:16:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 -- 172.21.15.57:0/2513839374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe68010f510 msgr2=0x7fe680111900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 --2- 172.21.15.57:0/2513839374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe68010f510 0x7fe680111900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b0a0 tx=0x7fe67800b3b0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 -- 172.21.15.57:0/2513839374 shutdown_connections 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 --2- 172.21.15.57:0/2513839374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe680074ce0 0x7fe6800750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 --2- 172.21.15.57:0/2513839374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe68010f510 0x7fe680111900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 --2- 172.21.15.57:0/2513839374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680075600 0x7fe68010efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.615+0000 7fe688000700 1 -- 172.21.15.57:0/2513839374 >> 172.21.15.57:0/2513839374 conn(0x7fe680079c10 msgr2=0x7fe68007a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.616+0000 7fe688000700 1 -- 172.21.15.57:0/2513839374 shutdown_connections 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.616+0000 7fe688000700 1 -- 172.21.15.57:0/2513839374 wait complete. 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 Processor -- start 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 -- start start 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe680074ce0 0x7fe68019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 0x7fe68019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68010f510 0x7fe68019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe680114ea0 con 0x7fe680075600 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe680114d20 con 0x7fe68010f510 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.617+0000 7fe688000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe680115020 con 0x7fe680074ce0 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 0x7fe68019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 0x7fe68019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46214/0 (socket says 172.21.15.57:46214) 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 -- 172.21.15.57:0/3444990487 learned_addr learned my addr 172.21.15.57:0/3444990487 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe686000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68010f510 0x7fe68019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe685600700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe680074ce0 0x7fe68019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 -- 172.21.15.57:0/3444990487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe680074ce0 msgr2=0x7fe68019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe680074ce0 0x7fe68019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 -- 172.21.15.57:0/3444990487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68010f510 msgr2=0x7fe68019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68010f510 0x7fe68019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 -- 172.21.15.57:0/3444990487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6801a02b0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.618+0000 7fe684c00700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 0x7fe68019b830 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00d980 tx=0x7fe66c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.619+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c017030 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.619+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe66c0044d0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.619+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6801a05a0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.619+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c003aa0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.619+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801a7e20 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.620+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6800697a0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.622+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe66c01a070 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.624+0000 7fe67e000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe6680a3cd0 0x7fe6680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.624+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe66c0d0350 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.624+0000 7fe685600700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe6680a3cd0 0x7fe6680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.625+0000 7fe685600700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe6680a3cd0 0x7fe6680a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe674005950 tx=0x7fe6740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.633+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe66c09c7d0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.892+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe68002dfa0 con 0x7fe680075600 2024-04-13T05:16:37.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.893+0000 7fe67e000700 1 -- 172.21.15.57:0/3444990487 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe66c09c0d0 con 0x7fe680075600 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stdout:287762808881 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe6680a3cd0 msgr2=0x7fe6680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe6680a3cd0 0x7fe6680a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe674005950 tx=0x7fe6740058e0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 msgr2=0x7fe68019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 0x7fe68019b830 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00d980 tx=0x7fe66c00dd40 comp rx=0 tx=0).stop 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 shutdown_connections 2024-04-13T05:16:37.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe680074ce0 0x7fe68019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe6680a3cd0 0x7fe6680a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68010f510 0x7fe68019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 --2- 172.21.15.57:0/3444990487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe680075600 0x7fe68019b830 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:37.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 >> 172.21.15.57:0/3444990487 conn(0x7fe680079c10 msgr2=0x7fe6800ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:37.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 shutdown_connections 2024-04-13T05:16:37.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:33.896+0000 7fe688000700 1 -- 172.21.15.57:0/3444990487 wait complete. 2024-04-13T05:16:38.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:38 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/394686611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-13T05:16:38.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:38 smithi057 ceph-mon[32331]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:38.689 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:38.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:38 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/394686611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-13T05:16:38.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:38 smithi083 ceph-mon[35133]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:38.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:38 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/394686611' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-13T05:16:38.832 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:38 smithi189 ceph-mon[35531]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:39.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351007 got 347892351014 for osd.11 2024-04-13T05:16:39.042 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:39.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 -- 172.21.15.57:0/345643005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 msgr2=0x7f568406b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:39.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 --2- 172.21.15.57:0/345643005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568406b640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f566c0099c0 tx=0x7f566c009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 -- 172.21.15.57:0/345643005 shutdown_connections 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 --2- 172.21.15.57:0/345643005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568406bb80 0x7f568410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 --2- 172.21.15.57:0/345643005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684108b50 0x7f5684108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 --2- 172.21.15.57:0/345643005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568406b640 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 -- 172.21.15.57:0/345643005 >> 172.21.15.57:0/345643005 conn(0x7f5684077a00 msgr2=0x7f5684077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 -- 172.21.15.57:0/345643005 shutdown_connections 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.282+0000 7f568b000700 1 -- 172.21.15.57:0/345643005 wait complete. 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.283+0000 7f568b000700 1 Processor -- start 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.283+0000 7f568b000700 1 -- start start 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f568b000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f568b000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568406bb80 0x7f568419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f568b000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684108b50 0x7f568419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f568b000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5684112ce0 con 0x7f568406b1c0 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f568b000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5684112b60 con 0x7f5684108b50 2024-04-13T05:16:39.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f568b000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5684112e60 con 0x7f568406bb80 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f5683e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f5683e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36304/0 (socket says 172.21.15.57:36304) 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f5683e00700 1 -- 172.21.15.57:0/1239137203 learned_addr learned my addr 172.21.15.57:0/1239137203 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.284+0000 7f567b400700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568406bb80 0x7f568419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5689000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684108b50 0x7f568419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5683e00700 1 -- 172.21.15.57:0/1239137203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568406bb80 msgr2=0x7f568419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5683e00700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568406bb80 0x7f568419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5683e00700 1 -- 172.21.15.57:0/1239137203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684108b50 msgr2=0x7f568419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5683e00700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684108b50 0x7f568419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5683e00700 1 -- 172.21.15.57:0/1239137203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56841a03d0 con 0x7f568406b1c0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5683e00700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568419b3c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56740060b0 tx=0x7f5674007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.285+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567401a070 con 0x7f568406b1c0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.286+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56841a06c0 con 0x7f568406b1c0 2024-04-13T05:16:39.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.286+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f567400de20 con 0x7f568406b1c0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.286+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841a7f40 con 0x7f568406b1c0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.286+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740146c0 con 0x7f568406b1c0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.287+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56840697a0 con 0x7f568406b1c0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.289+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f567401d070 con 0x7f568406b1c0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.290+0000 7f5681600700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f56680a3cd0 0x7f56680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.291+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f56740d03c0 con 0x7f568406b1c0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.291+0000 7f567b400700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f56680a3cd0 0x7f56680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.292+0000 7f567b400700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f56680a3cd0 0x7f56680a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5684103b50 tx=0x7f566c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:39.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.294+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5674010070 con 0x7f568406b1c0 2024-04-13T05:16:39.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.533+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f568419c720 con 0x7f568406b1c0 2024-04-13T05:16:39.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.534+0000 7f5681600700 1 -- 172.21.15.57:0/1239137203 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f567409c840 con 0x7f568406b1c0 2024-04-13T05:16:39.536 INFO:teuthology.orchestra.run.smithi057.stdout:219043332162 2024-04-13T05:16:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f56680a3cd0 msgr2=0x7f56680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:39.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f56680a3cd0 0x7f56680a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5684103b50 tx=0x7f566c005fb0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 msgr2=0x7f568419b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568419b3c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56740060b0 tx=0x7f5674007f90 comp rx=0 tx=0).stop 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 shutdown_connections 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f56680a3cd0 0x7f56680a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684108b50 0x7f568419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f568406b1c0 0x7f568419b3c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 --2- 172.21.15.57:0/1239137203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568406bb80 0x7f568419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.537+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 >> 172.21.15.57:0/1239137203 conn(0x7f5684077a00 msgr2=0x7f568410dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.538+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 shutdown_connections 2024-04-13T05:16:39.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:39.538+0000 7f568b000700 1 -- 172.21.15.57:0/1239137203 wait complete. 2024-04-13T05:16:40.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1239137203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-13T05:16:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1239137203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-13T05:16:40.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1239137203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-13T05:16:40.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612643 got 313532612650 for osd.10 2024-04-13T05:16:40.090 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:40.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:40 smithi057 ceph-mon[32331]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:40.924 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574948 got 55834574953 for osd.1 2024-04-13T05:16:40.924 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:41.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:40 smithi083 ceph-mon[35133]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:41.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:40 smithi189 ceph-mon[35531]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:16:41.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332155 got 219043332162 for osd.7 2024-04-13T05:16:41.552 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:42.217 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920848 got 141733920855 for osd.4 2024-04-13T05:16:42.217 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:42.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808875 got 287762808881 for osd.9 2024-04-13T05:16:42.959 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:43.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:42 smithi083 ceph-mon[35133]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:43.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:42 smithi057 ceph-mon[32331]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:43.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:42 smithi189 ceph-mon[35531]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:43.679 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495684 got 197568495689 for osd.6 2024-04-13T05:16:43.679 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:43.939 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:43 smithi057 ceph-mon[32331]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-13T05:16:44.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:43 smithi083 ceph-mon[35133]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-13T05:16:44.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:43 smithi189 ceph-mon[35531]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-13T05:16:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 -- 172.21.15.57:0/1881202280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 msgr2=0x7f7e14102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 --2- 172.21.15.57:0/1881202280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e14102f80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009990 tx=0x7f7e04009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 -- 172.21.15.57:0/1881202280 shutdown_connections 2024-04-13T05:16:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 --2- 172.21.15.57:0/1881202280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 0x7f7e14108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 --2- 172.21.15.57:0/1881202280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e141034c0 0x7f7e1410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 --2- 172.21.15.57:0/1881202280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e14102f80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 -- 172.21.15.57:0/1881202280 >> 172.21.15.57:0/1881202280 conn(0x7f7e14079c60 msgr2=0x7f7e1407a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 -- 172.21.15.57:0/1881202280 shutdown_connections 2024-04-13T05:16:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.268+0000 7f7e1c800700 1 -- 172.21.15.57:0/1881202280 wait complete. 2024-04-13T05:16:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 Processor -- start 2024-04-13T05:16:44.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 -- start start 2024-04-13T05:16:44.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e141990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:44.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e141034c0 0x7f7e141995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:44.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 0x7f7e1419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:44.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14112c90 con 0x7f7e14102b00 2024-04-13T05:16:44.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14112b10 con 0x7f7e141034c0 2024-04-13T05:16:44.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.270+0000 7f7e1c800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14112e10 con 0x7f7e14108b80 2024-04-13T05:16:44.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e141990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:44.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e141990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36316/0 (socket says 172.21.15.57:36316) 2024-04-13T05:16:44.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 -- 172.21.15.57:0/3444980080 learned_addr learned my addr 172.21.15.57:0/3444980080 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19400700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e141034c0 0x7f7e141995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e1a800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 0x7f7e1419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 -- 172.21.15.57:0/3444980080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 msgr2=0x7f7e1419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 0x7f7e1419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 -- 172.21.15.57:0/3444980080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e141034c0 msgr2=0x7f7e141995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e141034c0 0x7f7e141995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.271+0000 7f7e19e00700 1 -- 172.21.15.57:0/3444980080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1419e060 con 0x7f7e14102b00 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e1a800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 0x7f7e1419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:44.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e19e00700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e141990a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c9b0 tx=0x7f7e0800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:44.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08007b00 con 0x7f7e14102b00 2024-04-13T05:16:44.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1419e350 con 0x7f7e14102b00 2024-04-13T05:16:44.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e08007ca0 con 0x7f7e14102b00 2024-04-13T05:16:44.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08018660 con 0x7f7e14102b00 2024-04-13T05:16:44.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.272+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e141a5bd0 con 0x7f7e14102b00 2024-04-13T05:16:44.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.274+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e140697a0 con 0x7f7e14102b00 2024-04-13T05:16:44.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.276+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e0800f450 con 0x7f7e14102b00 2024-04-13T05:16:44.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.277+0000 7f7e0ea00700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:44.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.278+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7e080d09e0 con 0x7f7e14102b00 2024-04-13T05:16:44.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.278+0000 7f7e19400700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:44.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.279+0000 7f7e19400700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400b5c0 tx=0x7f7e04005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:44.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.281+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e08014030 con 0x7f7e14102b00 2024-04-13T05:16:44.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.527+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7e1419a3b0 con 0x7f7e14102b00 2024-04-13T05:16:44.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.528+0000 7f7e0ea00700 1 -- 172.21.15.57:0/3444980080 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e0809d050 con 0x7f7e14102b00 2024-04-13T05:16:44.530 INFO:teuthology.orchestra.run.smithi057.stdout:77309411429 2024-04-13T05:16:44.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.530+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7dfc0a3c40 msgr2=0x7f7dfc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.530+0000 7f7e1c800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400b5c0 tx=0x7f7e04005960 comp rx=0 tx=0).stop 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 msgr2=0x7f7e141990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e141990a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c9b0 tx=0x7f7e0800ccc0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 shutdown_connections 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e14108b80 0x7f7e1419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e14102b00 0x7f7e141990a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 --2- 172.21.15.57:0/3444980080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e141034c0 0x7f7e141995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 >> 172.21.15.57:0/3444980080 conn(0x7f7e14079c60 msgr2=0x7f7e14101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 shutdown_connections 2024-04-13T05:16:44.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:44.531+0000 7f7e1c800700 1 -- 172.21.15.57:0/3444980080 wait complete. 2024-04-13T05:16:44.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:44 smithi057 ceph-mon[32331]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-13T05:16:44.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:44 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3444980080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-13T05:16:44.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:44.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:44 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:45.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:44 smithi189 ceph-mon[35531]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-13T05:16:45.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:44 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3444980080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-13T05:16:45.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:45.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:44 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:44 smithi083 ceph-mon[35133]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-13T05:16:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:44 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3444980080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-13T05:16:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:44 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:45.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411423 got 77309411429 for osd.2 2024-04-13T05:16:45.371 DEBUG:teuthology.parallel:result is None 2024-04-13T05:16:45.371 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-13T05:16:45.372 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph pg dump --format=json 2024-04-13T05:16:45.828 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:46 smithi057 ceph-mon[32331]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-13T05:16:47.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:16:47.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:16:47.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:46 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:47.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:46 smithi189 ceph-mon[35531]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-13T05:16:47.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:16:47.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:16:47.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:46 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:46 smithi083 ceph-mon[35133]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-13T05:16:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:16:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:16:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:46 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 -- 172.21.15.57:0/2702327916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 msgr2=0x7f5734111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 --2- 172.21.15.57:0/2702327916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f5734111be0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f572c00b0d0 tx=0x7f572c00b3e0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 -- 172.21.15.57:0/2702327916 shutdown_connections 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 --2- 172.21.15.57:0/2702327916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5734104f30 0x7f5734105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 --2- 172.21.15.57:0/2702327916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f5734111be0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 --2- 172.21.15.57:0/2702327916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 0x7f5734109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 -- 172.21.15.57:0/2702327916 >> 172.21.15.57:0/2702327916 conn(0x7f57341007e0 msgr2=0x7f5734102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 -- 172.21.15.57:0/2702327916 shutdown_connections 2024-04-13T05:16:48.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.056+0000 7f5739600700 1 -- 172.21.15.57:0/2702327916 wait complete. 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 Processor -- start 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 -- start start 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5734104f30 0x7f573419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 0x7f573419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f573419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5734114f00 con 0x7f573410a010 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5734114d80 con 0x7f57341058e0 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.058+0000 7f5739600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5734115080 con 0x7f5734104f30 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f573419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f573419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42268/0 (socket says 172.21.15.57:42268) 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 -- 172.21.15.57:0/4161347338 learned_addr learned my addr 172.21.15.57:0/4161347338 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5732a00700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5734104f30 0x7f573419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5732000700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 0x7f573419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:48.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 -- 172.21.15.57:0/4161347338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5734104f30 msgr2=0x7f573419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5734104f30 0x7f573419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 -- 172.21.15.57:0/4161347338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 msgr2=0x7f573419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 0x7f573419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5733400700 1 -- 172.21.15.57:0/4161347338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341a0390 con 0x7f573410a010 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.059+0000 7f5732000700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 0x7f573419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.060+0000 7f5733400700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f573419fc50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d010 tx=0x7f572c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.060+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c01f070 con 0x7f573410a010 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.060+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f572c003c00 con 0x7f573410a010 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.060+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c01b6a0 con 0x7f573410a010 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.060+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341a0620 con 0x7f573410a010 2024-04-13T05:16:48.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.060+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a7f00 con 0x7f573410a010 2024-04-13T05:16:48.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.062+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57340697a0 con 0x7f573410a010 2024-04-13T05:16:48.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.065+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f572c022070 con 0x7f573410a010 2024-04-13T05:16:48.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.067+0000 7f572b400700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57180a3cc0 0x7f57180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:48.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.067+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f572c0d18c0 con 0x7f573410a010 2024-04-13T05:16:48.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.067+0000 7f5732a00700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57180a3cc0 0x7f57180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:48.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.068+0000 7f5732a00700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57180a3cc0 0x7f57180a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5724005950 tx=0x7f572400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:48.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.069+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f572c09df30 con 0x7f573410a010 2024-04-13T05:16:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.300+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f573419c6e0 con 0x7f57180a3cc0 2024-04-13T05:16:48.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.332+0000 7f572b400700 1 -- 172.21.15.57:0/4161347338 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340639 (secure 0 0 0) 0x7f573419c6e0 con 0x7f57180a3cc0 2024-04-13T05:16:48.336 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:16:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.340+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57180a3cc0 msgr2=0x7f57180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.340+0000 7f5739600700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57180a3cc0 0x7f57180a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5724005950 tx=0x7f572400b410 comp rx=0 tx=0).stop 2024-04-13T05:16:48.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.340+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 msgr2=0x7f573419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.340+0000 7f5739600700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f573419fc50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d010 tx=0x7f572c0079a0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 shutdown_connections 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5734104f30 0x7f573419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57180a3cc0 0x7f57180a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f573410a010 0x7f573419fc50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 --2- 172.21.15.57:0/4161347338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57341058e0 0x7f573419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 >> 172.21.15.57:0/4161347338 conn(0x7f57341007e0 msgr2=0x7f5734102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 shutdown_connections 2024-04-13T05:16:48.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:48.341+0000 7f5739600700 1 -- 172.21.15.57:0/4161347338 wait complete. 2024-04-13T05:16:48.345 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:16:49.013 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:48 smithi057 ceph-mon[32331]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:16:49.013 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:48 smithi057 ceph-mon[32331]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:16:49.083 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-13T05:16:47.644216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657618,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":646,"num_read_kb":443,"num_write":352,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367936,"kb_used_data":14624,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573888,"statfs":{"total":1151940427776,"available":1151563661312,"internally_reserved":0,"allocated":14974976,"data_stored":8264540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":33,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005391"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600603+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:16:41.600603+0000","last_peered":"2024-04-13T05:16:41.600603+0000","last_clean":"2024-04-13T05:16:41.600603+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:16:41.600603+0000","last_undegraded":"2024-04-13T05:16:41.600603+0000","last_fullsized":"2024-04-13T05:16:41.600603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593168+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:16:41.593168+0000","last_peered":"2024-04-13T05:16:41.593168+0000","last_clean":"2024-04-13T05:16:41.593168+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:16:41.593168+0000","last_undegraded":"2024-04-13T05:16:41.593168+0000","last_fullsized":"2024-04-13T05:16:41.593168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600943+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:16:41.600943+0000","last_peered":"2024-04-13T05:16:41.600943+0000","last_clean":"2024-04-13T05:16:41.600943+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:16:41.600943+0000","last_undegraded":"2024-04-13T05:16:41.600943+0000","last_fullsized":"2024-04-13T05:16:41.600943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.573728+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:16:41.573728+0000","last_peered":"2024-04-13T05:16:41.573728+0000","last_clean":"2024-04-13T05:16:41.573728+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:16:41.573728+0000","last_undegraded":"2024-04-13T05:16:41.573728+0000","last_fullsized":"2024-04-13T05:16:41.573728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.580106+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:16:41.580106+0000","last_peered":"2024-04-13T05:16:41.580106+0000","last_clean":"2024-04-13T05:16:41.580106+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:16:41.580106+0000","last_undegraded":"2024-04-13T05:16:41.580106+0000","last_fullsized":"2024-04-13T05:16:41.580106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591744+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:16:41.591744+0000","last_peered":"2024-04-13T05:16:41.591744+0000","last_clean":"2024-04-13T05:16:41.591744+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:16:41.591744+0000","last_undegraded":"2024-04-13T05:16:41.591744+0000","last_fullsized":"2024-04-13T05:16:41.591744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593200+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:16:41.593200+0000","last_peered":"2024-04-13T05:16:41.593200+0000","last_clean":"2024-04-13T05:16:41.593200+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:16:41.593200+0000","last_undegraded":"2024-04-13T05:16:41.593200+0000","last_fullsized":"2024-04-13T05:16:41.593200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596818+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:16:41.596818+0000","last_peered":"2024-04-13T05:16:41.596818+0000","last_clean":"2024-04-13T05:16:41.596818+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:16:41.596818+0000","last_undegraded":"2024-04-13T05:16:41.596818+0000","last_fullsized":"2024-04-13T05:16:41.596818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600243+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:16:41.600243+0000","last_peered":"2024-04-13T05:16:41.600243+0000","last_clean":"2024-04-13T05:16:41.600243+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:16:41.600243+0000","last_undegraded":"2024-04-13T05:16:41.600243+0000","last_fullsized":"2024-04-13T05:16:41.600243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595627+0000","last_change":"2024-04-13T05:16:14.217489+0000","last_active":"2024-04-13T05:16:41.595627+0000","last_peered":"2024-04-13T05:16:41.595627+0000","last_clean":"2024-04-13T05:16:41.595627+0000","last_became_active":"2024-04-13T05:16:14.216651+0000","last_became_peered":"2024-04-13T05:16:14.216651+0000","last_unstale":"2024-04-13T05:16:41.595627+0000","last_undegraded":"2024-04-13T05:16:41.595627+0000","last_fullsized":"2024-04-13T05:16:41.595627+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:55:10.808460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601686+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:16:41.601686+0000","last_peered":"2024-04-13T05:16:41.601686+0000","last_clean":"2024-04-13T05:16:41.601686+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:16:41.601686+0000","last_undegraded":"2024-04-13T05:16:41.601686+0000","last_fullsized":"2024-04-13T05:16:41.601686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585211+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:16:41.585211+0000","last_peered":"2024-04-13T05:16:41.585211+0000","last_clean":"2024-04-13T05:16:41.585211+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:16:41.585211+0000","last_undegraded":"2024-04-13T05:16:41.585211+0000","last_fullsized":"2024-04-13T05:16:41.585211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590222+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:16:41.590222+0000","last_peered":"2024-04-13T05:16:41.590222+0000","last_clean":"2024-04-13T05:16:41.590222+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:16:41.590222+0000","last_undegraded":"2024-04-13T05:16:41.590222+0000","last_fullsized":"2024-04-13T05:16:41.590222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595919+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:16:41.595919+0000","last_peered":"2024-04-13T05:16:41.595919+0000","last_clean":"2024-04-13T05:16:41.595919+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:16:41.595919+0000","last_undegraded":"2024-04-13T05:16:41.595919+0000","last_fullsized":"2024-04-13T05:16:41.595919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591659+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:16:41.591659+0000","last_peered":"2024-04-13T05:16:41.591659+0000","last_clean":"2024-04-13T05:16:41.591659+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:16:41.591659+0000","last_undegraded":"2024-04-13T05:16:41.591659+0000","last_fullsized":"2024-04-13T05:16:41.591659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596919+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:16:41.596919+0000","last_peered":"2024-04-13T05:16:41.596919+0000","last_clean":"2024-04-13T05:16:41.596919+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:16:41.596919+0000","last_undegraded":"2024-04-13T05:16:41.596919+0000","last_fullsized":"2024-04-13T05:16:41.596919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600704+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:16:41.600704+0000","last_peered":"2024-04-13T05:16:41.600704+0000","last_clean":"2024-04-13T05:16:41.600704+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:16:41.600704+0000","last_undegraded":"2024-04-13T05:16:41.600704+0000","last_fullsized":"2024-04-13T05:16:41.600704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589080+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:16:41.589080+0000","last_peered":"2024-04-13T05:16:41.589080+0000","last_clean":"2024-04-13T05:16:41.589080+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:16:41.589080+0000","last_undegraded":"2024-04-13T05:16:41.589080+0000","last_fullsized":"2024-04-13T05:16:41.589080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574707+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:16:41.574707+0000","last_peered":"2024-04-13T05:16:41.574707+0000","last_clean":"2024-04-13T05:16:41.574707+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:16:41.574707+0000","last_undegraded":"2024-04-13T05:16:41.574707+0000","last_fullsized":"2024-04-13T05:16:41.574707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207182+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:16:14.207182+0000","last_peered":"2024-04-13T05:16:14.207182+0000","last_clean":"2024-04-13T05:16:14.207182+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:16:14.207182+0000","last_undegraded":"2024-04-13T05:16:14.207182+0000","last_fullsized":"2024-04-13T05:16:14.207182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208504+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:16:14.208504+0000","last_peered":"2024-04-13T05:16:14.208504+0000","last_clean":"2024-04-13T05:16:14.208504+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:16:14.208504+0000","last_undegraded":"2024-04-13T05:16:14.208504+0000","last_fullsized":"2024-04-13T05:16:14.208504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596334+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:16:41.596334+0000","last_peered":"2024-04-13T05:16:41.596334+0000","last_clean":"2024-04-13T05:16:41.596334+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:16:41.596334+0000","last_undegraded":"2024-04-13T05:16:41.596334+0000","last_fullsized":"2024-04-13T05:16:41.596334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206864+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:16:14.206864+0000","last_peered":"2024-04-13T05:16:14.206864+0000","last_clean":"2024-04-13T05:16:14.206864+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:16:14.206864+0000","last_undegraded":"2024-04-13T05:16:14.206864+0000","last_fullsized":"2024-04-13T05:16:14.206864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206137+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:16:14.206137+0000","last_peered":"2024-04-13T05:16:14.206137+0000","last_clean":"2024-04-13T05:16:14.206137+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:16:14.206137+0000","last_undegraded":"2024-04-13T05:16:14.206137+0000","last_fullsized":"2024-04-13T05:16:14.206137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600848+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:16:41.600848+0000","last_peered":"2024-04-13T05:16:41.600848+0000","last_clean":"2024-04-13T05:16:41.600848+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:16:41.600848+0000","last_undegraded":"2024-04-13T05:16:41.600848+0000","last_fullsized":"2024-04-13T05:16:41.600848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207216+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:16:14.207216+0000","last_peered":"2024-04-13T05:16:14.207216+0000","last_clean":"2024-04-13T05:16:14.207216+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:16:14.207216+0000","last_undegraded":"2024-04-13T05:16:14.207216+0000","last_fullsized":"2024-04-13T05:16:14.207216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215001+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:16:14.215001+0000","last_peered":"2024-04-13T05:16:14.215001+0000","last_clean":"2024-04-13T05:16:14.215001+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:16:14.215001+0000","last_undegraded":"2024-04-13T05:16:14.215001+0000","last_fullsized":"2024-04-13T05:16:14.215001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599872+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:16:41.599872+0000","last_peered":"2024-04-13T05:16:41.599872+0000","last_clean":"2024-04-13T05:16:41.599872+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:16:41.599872+0000","last_undegraded":"2024-04-13T05:16:41.599872+0000","last_fullsized":"2024-04-13T05:16:41.599872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206958+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:16:14.206958+0000","last_peered":"2024-04-13T05:16:14.206958+0000","last_clean":"2024-04-13T05:16:14.206958+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:16:14.206958+0000","last_undegraded":"2024-04-13T05:16:14.206958+0000","last_fullsized":"2024-04-13T05:16:14.206958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711513+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:16:14.711513+0000","last_peered":"2024-04-13T05:16:14.711513+0000","last_clean":"2024-04-13T05:16:14.711513+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:16:14.711513+0000","last_undegraded":"2024-04-13T05:16:14.711513+0000","last_fullsized":"2024-04-13T05:16:14.711513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601014+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:16:41.601014+0000","last_peered":"2024-04-13T05:16:41.601014+0000","last_clean":"2024-04-13T05:16:41.601014+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:16:41.601014+0000","last_undegraded":"2024-04-13T05:16:41.601014+0000","last_fullsized":"2024-04-13T05:16:41.601014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207933+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:16:14.207933+0000","last_peered":"2024-04-13T05:16:14.207933+0000","last_clean":"2024-04-13T05:16:14.207933+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:16:14.207933+0000","last_undegraded":"2024-04-13T05:16:14.207933+0000","last_fullsized":"2024-04-13T05:16:14.207933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":119,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.588573+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:16:41.588573+0000","last_peered":"2024-04-13T05:16:41.588573+0000","last_clean":"2024-04-13T05:16:41.588573+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:16:41.588573+0000","last_undegraded":"2024-04-13T05:16:41.588573+0000","last_fullsized":"2024-04-13T05:16:41.588573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590566+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:16:41.590566+0000","last_peered":"2024-04-13T05:16:41.590566+0000","last_clean":"2024-04-13T05:16:41.590566+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:16:41.590566+0000","last_undegraded":"2024-04-13T05:16:41.590566+0000","last_fullsized":"2024-04-13T05:16:41.590566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207515+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:16:14.207515+0000","last_peered":"2024-04-13T05:16:14.207515+0000","last_clean":"2024-04-13T05:16:14.207515+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:16:14.207515+0000","last_undegraded":"2024-04-13T05:16:14.207515+0000","last_fullsized":"2024-04-13T05:16:14.207515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.212620+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:16:14.212620+0000","last_peered":"2024-04-13T05:16:14.212620+0000","last_clean":"2024-04-13T05:16:14.212620+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:16:14.212620+0000","last_undegraded":"2024-04-13T05:16:14.212620+0000","last_fullsized":"2024-04-13T05:16:14.212620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597026+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:16:41.597026+0000","last_peered":"2024-04-13T05:16:41.597026+0000","last_clean":"2024-04-13T05:16:41.597026+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:16:41.597026+0000","last_undegraded":"2024-04-13T05:16:41.597026+0000","last_fullsized":"2024-04-13T05:16:41.597026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215333+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:16:14.215333+0000","last_peered":"2024-04-13T05:16:14.215333+0000","last_clean":"2024-04-13T05:16:14.215333+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:16:14.215333+0000","last_undegraded":"2024-04-13T05:16:14.215333+0000","last_fullsized":"2024-04-13T05:16:14.215333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.500669+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:16:14.500669+0000","last_peered":"2024-04-13T05:16:14.500669+0000","last_clean":"2024-04-13T05:16:14.500669+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:16:14.500669+0000","last_undegraded":"2024-04-13T05:16:14.500669+0000","last_fullsized":"2024-04-13T05:16:14.500669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584165+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:16:41.584165+0000","last_peered":"2024-04-13T05:16:41.584165+0000","last_clean":"2024-04-13T05:16:41.584165+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:16:41.584165+0000","last_undegraded":"2024-04-13T05:16:41.584165+0000","last_fullsized":"2024-04-13T05:16:41.584165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548845+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:16:14.548845+0000","last_peered":"2024-04-13T05:16:14.548845+0000","last_clean":"2024-04-13T05:16:14.548845+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:16:14.548845+0000","last_undegraded":"2024-04-13T05:16:14.548845+0000","last_fullsized":"2024-04-13T05:16:14.548845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488119+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:16:14.488119+0000","last_peered":"2024-04-13T05:16:14.488119+0000","last_clean":"2024-04-13T05:16:14.488119+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:16:14.488119+0000","last_undegraded":"2024-04-13T05:16:14.488119+0000","last_fullsized":"2024-04-13T05:16:14.488119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595176+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:16:41.595176+0000","last_peered":"2024-04-13T05:16:41.595176+0000","last_clean":"2024-04-13T05:16:41.595176+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:16:41.595176+0000","last_undegraded":"2024-04-13T05:16:41.595176+0000","last_fullsized":"2024-04-13T05:16:41.595176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205299+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:16:14.205299+0000","last_peered":"2024-04-13T05:16:14.205299+0000","last_clean":"2024-04-13T05:16:14.205299+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:16:14.205299+0000","last_undegraded":"2024-04-13T05:16:14.205299+0000","last_fullsized":"2024-04-13T05:16:14.205299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":116,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:42.590317+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:16:42.590317+0000","last_peered":"2024-04-13T05:16:42.590317+0000","last_clean":"2024-04-13T05:16:42.590317+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:16:42.590317+0000","last_undegraded":"2024-04-13T05:16:42.590317+0000","last_fullsized":"2024-04-13T05:16:42.590317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597354+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:16:41.597354+0000","last_peered":"2024-04-13T05:16:41.597354+0000","last_clean":"2024-04-13T05:16:41.597354+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:16:41.597354+0000","last_undegraded":"2024-04-13T05:16:41.597354+0000","last_fullsized":"2024-04-13T05:16:41.597354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207805+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:16:14.207805+0000","last_peered":"2024-04-13T05:16:14.207805+0000","last_clean":"2024-04-13T05:16:14.207805+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:16:14.207805+0000","last_undegraded":"2024-04-13T05:16:14.207805+0000","last_fullsized":"2024-04-13T05:16:14.207805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712209+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:16:14.712209+0000","last_peered":"2024-04-13T05:16:14.712209+0000","last_clean":"2024-04-13T05:16:14.712209+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:16:14.712209+0000","last_undegraded":"2024-04-13T05:16:14.712209+0000","last_fullsized":"2024-04-13T05:16:14.712209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589834+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:16:41.589834+0000","last_peered":"2024-04-13T05:16:41.589834+0000","last_clean":"2024-04-13T05:16:41.589834+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:16:41.589834+0000","last_undegraded":"2024-04-13T05:16:41.589834+0000","last_fullsized":"2024-04-13T05:16:41.589834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207816+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:16:14.207816+0000","last_peered":"2024-04-13T05:16:14.207816+0000","last_clean":"2024-04-13T05:16:14.207816+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:16:14.207816+0000","last_undegraded":"2024-04-13T05:16:14.207816+0000","last_fullsized":"2024-04-13T05:16:14.207816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489176+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:16:14.489176+0000","last_peered":"2024-04-13T05:16:14.489176+0000","last_clean":"2024-04-13T05:16:14.489176+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:16:14.489176+0000","last_undegraded":"2024-04-13T05:16:14.489176+0000","last_fullsized":"2024-04-13T05:16:14.489176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601329+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:16:41.601329+0000","last_peered":"2024-04-13T05:16:41.601329+0000","last_clean":"2024-04-13T05:16:41.601329+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:16:41.601329+0000","last_undegraded":"2024-04-13T05:16:41.601329+0000","last_fullsized":"2024-04-13T05:16:41.601329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208034+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:16:14.208034+0000","last_peered":"2024-04-13T05:16:14.208034+0000","last_clean":"2024-04-13T05:16:14.208034+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:16:14.208034+0000","last_undegraded":"2024-04-13T05:16:14.208034+0000","last_fullsized":"2024-04-13T05:16:14.208034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":240,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:43.603869+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:16:43.603869+0000","last_peered":"2024-04-13T05:16:43.603869+0000","last_clean":"2024-04-13T05:16:43.603869+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:16:43.603869+0000","last_undegraded":"2024-04-13T05:16:43.603869+0000","last_fullsized":"2024-04-13T05:16:43.603869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":174,"num_read_kb":150,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595634+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:16:41.595634+0000","last_peered":"2024-04-13T05:16:41.595634+0000","last_clean":"2024-04-13T05:16:41.595634+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:16:41.595634+0000","last_undegraded":"2024-04-13T05:16:41.595634+0000","last_fullsized":"2024-04-13T05:16:41.595634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489985+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:16:14.489985+0000","last_peered":"2024-04-13T05:16:14.489985+0000","last_clean":"2024-04-13T05:16:14.489985+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:16:14.489985+0000","last_undegraded":"2024-04-13T05:16:14.489985+0000","last_fullsized":"2024-04-13T05:16:14.489985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207679+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:16:14.207679+0000","last_peered":"2024-04-13T05:16:14.207679+0000","last_clean":"2024-04-13T05:16:14.207679+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:16:14.207679+0000","last_undegraded":"2024-04-13T05:16:14.207679+0000","last_fullsized":"2024-04-13T05:16:14.207679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590833+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:16:41.590833+0000","last_peered":"2024-04-13T05:16:41.590833+0000","last_clean":"2024-04-13T05:16:41.590833+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:16:41.590833+0000","last_undegraded":"2024-04-13T05:16:41.590833+0000","last_fullsized":"2024-04-13T05:16:41.590833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214342+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:16:14.214342+0000","last_peered":"2024-04-13T05:16:14.214342+0000","last_clean":"2024-04-13T05:16:14.214342+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:16:14.214342+0000","last_undegraded":"2024-04-13T05:16:14.214342+0000","last_fullsized":"2024-04-13T05:16:14.214342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488741+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:16:14.488741+0000","last_peered":"2024-04-13T05:16:14.488741+0000","last_clean":"2024-04-13T05:16:14.488741+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:16:14.488741+0000","last_undegraded":"2024-04-13T05:16:14.488741+0000","last_fullsized":"2024-04-13T05:16:14.488741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596455+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:16:41.596455+0000","last_peered":"2024-04-13T05:16:41.596455+0000","last_clean":"2024-04-13T05:16:41.596455+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:16:41.596455+0000","last_undegraded":"2024-04-13T05:16:41.596455+0000","last_fullsized":"2024-04-13T05:16:41.596455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208654+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:16:14.208654+0000","last_peered":"2024-04-13T05:16:14.208654+0000","last_clean":"2024-04-13T05:16:14.208654+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:16:14.208654+0000","last_undegraded":"2024-04-13T05:16:14.208654+0000","last_fullsized":"2024-04-13T05:16:14.208654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":121,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.598448+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:16:41.598448+0000","last_peered":"2024-04-13T05:16:41.598448+0000","last_clean":"2024-04-13T05:16:41.598448+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:16:41.598448+0000","last_undegraded":"2024-04-13T05:16:41.598448+0000","last_fullsized":"2024-04-13T05:16:41.598448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593508+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:16:41.593508+0000","last_peered":"2024-04-13T05:16:41.593508+0000","last_clean":"2024-04-13T05:16:41.593508+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:16:41.593508+0000","last_undegraded":"2024-04-13T05:16:41.593508+0000","last_fullsized":"2024-04-13T05:16:41.593508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501984+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:16:14.501984+0000","last_peered":"2024-04-13T05:16:14.501984+0000","last_clean":"2024-04-13T05:16:14.501984+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:16:14.501984+0000","last_undegraded":"2024-04-13T05:16:14.501984+0000","last_fullsized":"2024-04-13T05:16:14.501984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709675+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:16:14.709675+0000","last_peered":"2024-04-13T05:16:14.709675+0000","last_clean":"2024-04-13T05:16:14.709675+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:16:14.709675+0000","last_undegraded":"2024-04-13T05:16:14.709675+0000","last_fullsized":"2024-04-13T05:16:14.709675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600435+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:16:41.600435+0000","last_peered":"2024-04-13T05:16:41.600435+0000","last_clean":"2024-04-13T05:16:41.600435+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:16:41.600435+0000","last_undegraded":"2024-04-13T05:16:41.600435+0000","last_fullsized":"2024-04-13T05:16:41.600435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488267+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:16:14.488267+0000","last_peered":"2024-04-13T05:16:14.488267+0000","last_clean":"2024-04-13T05:16:14.488267+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:16:14.488267+0000","last_undegraded":"2024-04-13T05:16:14.488267+0000","last_fullsized":"2024-04-13T05:16:14.488267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710548+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:16:14.710548+0000","last_peered":"2024-04-13T05:16:14.710548+0000","last_clean":"2024-04-13T05:16:14.710548+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:16:14.710548+0000","last_undegraded":"2024-04-13T05:16:14.710548+0000","last_fullsized":"2024-04-13T05:16:14.710548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.583945+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:16:41.583945+0000","last_peered":"2024-04-13T05:16:41.583945+0000","last_clean":"2024-04-13T05:16:41.583945+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:16:41.583945+0000","last_undegraded":"2024-04-13T05:16:41.583945+0000","last_fullsized":"2024-04-13T05:16:41.583945+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711126+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:16:14.711126+0000","last_peered":"2024-04-13T05:16:14.711126+0000","last_clean":"2024-04-13T05:16:14.711126+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:16:14.711126+0000","last_undegraded":"2024-04-13T05:16:14.711126+0000","last_fullsized":"2024-04-13T05:16:14.711126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208216+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:16:14.208216+0000","last_peered":"2024-04-13T05:16:14.208216+0000","last_clean":"2024-04-13T05:16:14.208216+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:16:14.208216+0000","last_undegraded":"2024-04-13T05:16:14.208216+0000","last_fullsized":"2024-04-13T05:16:14.208216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596822+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:16:41.596822+0000","last_peered":"2024-04-13T05:16:41.596822+0000","last_clean":"2024-04-13T05:16:41.596822+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:16:41.596822+0000","last_undegraded":"2024-04-13T05:16:41.596822+0000","last_fullsized":"2024-04-13T05:16:41.596822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501136+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:16:14.501136+0000","last_peered":"2024-04-13T05:16:14.501136+0000","last_clean":"2024-04-13T05:16:14.501136+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:16:14.501136+0000","last_undegraded":"2024-04-13T05:16:14.501136+0000","last_fullsized":"2024-04-13T05:16:14.501136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711674+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:16:14.711674+0000","last_peered":"2024-04-13T05:16:14.711674+0000","last_clean":"2024-04-13T05:16:14.711674+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:16:14.711674+0000","last_undegraded":"2024-04-13T05:16:14.711674+0000","last_fullsized":"2024-04-13T05:16:14.711674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591381+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:16:41.591381+0000","last_peered":"2024-04-13T05:16:41.591381+0000","last_clean":"2024-04-13T05:16:41.591381+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:16:41.591381+0000","last_undegraded":"2024-04-13T05:16:41.591381+0000","last_fullsized":"2024-04-13T05:16:41.591381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501697+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:16:14.501697+0000","last_peered":"2024-04-13T05:16:14.501697+0000","last_clean":"2024-04-13T05:16:14.501697+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:16:14.501697+0000","last_undegraded":"2024-04-13T05:16:14.501697+0000","last_fullsized":"2024-04-13T05:16:14.501697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586794+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:16:41.586794+0000","last_peered":"2024-04-13T05:16:41.586794+0000","last_clean":"2024-04-13T05:16:41.586794+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:16:41.586794+0000","last_undegraded":"2024-04-13T05:16:41.586794+0000","last_fullsized":"2024-04-13T05:16:41.586794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":118,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:42.600028+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:16:42.600028+0000","last_peered":"2024-04-13T05:16:42.600028+0000","last_clean":"2024-04-13T05:16:42.600028+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:16:42.600028+0000","last_undegraded":"2024-04-13T05:16:42.600028+0000","last_fullsized":"2024-04-13T05:16:42.600028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712433+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:16:14.712433+0000","last_peered":"2024-04-13T05:16:14.712433+0000","last_clean":"2024-04-13T05:16:14.712433+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:16:14.712433+0000","last_undegraded":"2024-04-13T05:16:14.712433+0000","last_fullsized":"2024-04-13T05:16:14.712433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601957+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:16:41.601957+0000","last_peered":"2024-04-13T05:16:41.601957+0000","last_clean":"2024-04-13T05:16:41.601957+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:16:41.601957+0000","last_undegraded":"2024-04-13T05:16:41.601957+0000","last_fullsized":"2024-04-13T05:16:41.601957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207216+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:16:14.207216+0000","last_peered":"2024-04-13T05:16:14.207216+0000","last_clean":"2024-04-13T05:16:14.207216+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:16:14.207216+0000","last_undegraded":"2024-04-13T05:16:14.207216+0000","last_fullsized":"2024-04-13T05:16:14.207216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710982+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:16:14.710982+0000","last_peered":"2024-04-13T05:16:14.710982+0000","last_clean":"2024-04-13T05:16:14.710982+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:16:14.710982+0000","last_undegraded":"2024-04-13T05:16:14.710982+0000","last_fullsized":"2024-04-13T05:16:14.710982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489979+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:16:14.489979+0000","last_peered":"2024-04-13T05:16:14.489979+0000","last_clean":"2024-04-13T05:16:14.489979+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:16:14.489979+0000","last_undegraded":"2024-04-13T05:16:14.489979+0000","last_fullsized":"2024-04-13T05:16:14.489979+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574284+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:16:41.574284+0000","last_peered":"2024-04-13T05:16:41.574284+0000","last_clean":"2024-04-13T05:16:41.574284+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:16:41.574284+0000","last_undegraded":"2024-04-13T05:16:41.574284+0000","last_fullsized":"2024-04-13T05:16:41.574284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711264+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:16:14.711264+0000","last_peered":"2024-04-13T05:16:14.711264+0000","last_clean":"2024-04-13T05:16:14.711264+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:16:14.711264+0000","last_undegraded":"2024-04-13T05:16:14.711264+0000","last_fullsized":"2024-04-13T05:16:14.711264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214529+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:16:14.214529+0000","last_peered":"2024-04-13T05:16:14.214529+0000","last_clean":"2024-04-13T05:16:14.214529+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:16:14.214529+0000","last_undegraded":"2024-04-13T05:16:14.214529+0000","last_fullsized":"2024-04-13T05:16:14.214529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710784+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:16:14.710784+0000","last_peered":"2024-04-13T05:16:14.710784+0000","last_clean":"2024-04-13T05:16:14.710784+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:16:14.710784+0000","last_undegraded":"2024-04-13T05:16:14.710784+0000","last_fullsized":"2024-04-13T05:16:14.710784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574646+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:16:41.574646+0000","last_peered":"2024-04-13T05:16:41.574646+0000","last_clean":"2024-04-13T05:16:41.574646+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:16:41.574646+0000","last_undegraded":"2024-04-13T05:16:41.574646+0000","last_fullsized":"2024-04-13T05:16:41.574646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205157+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:16:14.205157+0000","last_peered":"2024-04-13T05:16:14.205157+0000","last_clean":"2024-04-13T05:16:14.205157+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:16:14.205157+0000","last_undegraded":"2024-04-13T05:16:14.205157+0000","last_fullsized":"2024-04-13T05:16:14.205157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208389+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:16:14.208389+0000","last_peered":"2024-04-13T05:16:14.208389+0000","last_clean":"2024-04-13T05:16:14.208389+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:16:14.208389+0000","last_undegraded":"2024-04-13T05:16:14.208389+0000","last_fullsized":"2024-04-13T05:16:14.208389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594139+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:16:41.594139+0000","last_peered":"2024-04-13T05:16:41.594139+0000","last_clean":"2024-04-13T05:16:41.594139+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:16:41.594139+0000","last_undegraded":"2024-04-13T05:16:41.594139+0000","last_fullsized":"2024-04-13T05:16:41.594139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206843+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:16:14.206843+0000","last_peered":"2024-04-13T05:16:14.206843+0000","last_clean":"2024-04-13T05:16:14.206843+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:16:14.206843+0000","last_undegraded":"2024-04-13T05:16:14.206843+0000","last_fullsized":"2024-04-13T05:16:14.206843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208960+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:16:14.208960+0000","last_peered":"2024-04-13T05:16:14.208960+0000","last_clean":"2024-04-13T05:16:14.208960+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:16:14.208960+0000","last_undegraded":"2024-04-13T05:16:14.208960+0000","last_fullsized":"2024-04-13T05:16:14.208960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595936+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:16:41.595936+0000","last_peered":"2024-04-13T05:16:41.595936+0000","last_clean":"2024-04-13T05:16:41.595936+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:16:41.595936+0000","last_undegraded":"2024-04-13T05:16:41.595936+0000","last_fullsized":"2024-04-13T05:16:41.595936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208468+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:16:14.208468+0000","last_peered":"2024-04-13T05:16:14.208468+0000","last_clean":"2024-04-13T05:16:14.208468+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:16:14.208468+0000","last_undegraded":"2024-04-13T05:16:14.208468+0000","last_fullsized":"2024-04-13T05:16:14.208468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709332+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:16:14.709332+0000","last_peered":"2024-04-13T05:16:14.709332+0000","last_clean":"2024-04-13T05:16:14.709332+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:16:14.709332+0000","last_undegraded":"2024-04-13T05:16:14.709332+0000","last_fullsized":"2024-04-13T05:16:14.709332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.532767+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:16:41.532767+0000","last_peered":"2024-04-13T05:16:41.532767+0000","last_clean":"2024-04-13T05:16:41.532767+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:16:41.532767+0000","last_undegraded":"2024-04-13T05:16:41.532767+0000","last_fullsized":"2024-04-13T05:16:41.532767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208551+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:16:14.208551+0000","last_peered":"2024-04-13T05:16:14.208551+0000","last_clean":"2024-04-13T05:16:14.208551+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:16:14.208551+0000","last_undegraded":"2024-04-13T05:16:14.208551+0000","last_fullsized":"2024-04-13T05:16:14.208551+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205417+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:16:14.205417+0000","last_peered":"2024-04-13T05:16:14.205417+0000","last_clean":"2024-04-13T05:16:14.205417+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:16:14.205417+0000","last_undegraded":"2024-04-13T05:16:14.205417+0000","last_fullsized":"2024-04-13T05:16:14.205417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591171+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:16:41.591171+0000","last_peered":"2024-04-13T05:16:41.591171+0000","last_clean":"2024-04-13T05:16:41.591171+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:16:41.591171+0000","last_undegraded":"2024-04-13T05:16:41.591171+0000","last_fullsized":"2024-04-13T05:16:41.591171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207020+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:16:14.207020+0000","last_peered":"2024-04-13T05:16:14.207020+0000","last_clean":"2024-04-13T05:16:14.207020+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:16:14.207020+0000","last_undegraded":"2024-04-13T05:16:14.207020+0000","last_fullsized":"2024-04-13T05:16:14.207020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214036+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:16:14.214036+0000","last_peered":"2024-04-13T05:16:14.214036+0000","last_clean":"2024-04-13T05:16:14.214036+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:16:14.214036+0000","last_undegraded":"2024-04-13T05:16:14.214036+0000","last_fullsized":"2024-04-13T05:16:14.214036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597636+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:16:41.597636+0000","last_peered":"2024-04-13T05:16:41.597636+0000","last_clean":"2024-04-13T05:16:41.597636+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:16:41.597636+0000","last_undegraded":"2024-04-13T05:16:41.597636+0000","last_fullsized":"2024-04-13T05:16:41.597636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208102+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:16:14.208102+0000","last_peered":"2024-04-13T05:16:14.208102+0000","last_clean":"2024-04-13T05:16:14.208102+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:16:14.208102+0000","last_undegraded":"2024-04-13T05:16:14.208102+0000","last_fullsized":"2024-04-13T05:16:14.208102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710990+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:16:14.710990+0000","last_peered":"2024-04-13T05:16:14.710990+0000","last_clean":"2024-04-13T05:16:14.710990+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:16:14.710990+0000","last_undegraded":"2024-04-13T05:16:14.710990+0000","last_fullsized":"2024-04-13T05:16:14.710990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601865+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:16:41.601865+0000","last_peered":"2024-04-13T05:16:41.601865+0000","last_clean":"2024-04-13T05:16:41.601865+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:16:41.601865+0000","last_undegraded":"2024-04-13T05:16:41.601865+0000","last_fullsized":"2024-04-13T05:16:41.601865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709076+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:16:14.709076+0000","last_peered":"2024-04-13T05:16:14.709076+0000","last_clean":"2024-04-13T05:16:14.709076+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:16:14.709076+0000","last_undegraded":"2024-04-13T05:16:14.709076+0000","last_fullsized":"2024-04-13T05:16:14.709076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207927+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:16:14.207927+0000","last_peered":"2024-04-13T05:16:14.207927+0000","last_clean":"2024-04-13T05:16:14.207927+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:16:14.207927+0000","last_undegraded":"2024-04-13T05:16:14.207927+0000","last_fullsized":"2024-04-13T05:16:14.207927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601679+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:16:41.601679+0000","last_peered":"2024-04-13T05:16:41.601679+0000","last_clean":"2024-04-13T05:16:41.601679+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:16:41.601679+0000","last_undegraded":"2024-04-13T05:16:41.601679+0000","last_fullsized":"2024-04-13T05:16:41.601679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548084+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:16:14.548084+0000","last_peered":"2024-04-13T05:16:14.548084+0000","last_clean":"2024-04-13T05:16:14.548084+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:16:14.548084+0000","last_undegraded":"2024-04-13T05:16:14.548084+0000","last_fullsized":"2024-04-13T05:16:14.548084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489530+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:16:14.489530+0000","last_peered":"2024-04-13T05:16:14.489530+0000","last_clean":"2024-04-13T05:16:14.489530+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:16:14.489530+0000","last_undegraded":"2024-04-13T05:16:14.489530+0000","last_fullsized":"2024-04-13T05:16:14.489530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585748+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:16:41.585748+0000","last_peered":"2024-04-13T05:16:41.585748+0000","last_clean":"2024-04-13T05:16:41.585748+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:16:41.585748+0000","last_undegraded":"2024-04-13T05:16:41.585748+0000","last_fullsized":"2024-04-13T05:16:41.585748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.487768+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:16:14.487768+0000","last_peered":"2024-04-13T05:16:14.487768+0000","last_clean":"2024-04-13T05:16:14.487768+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:16:14.487768+0000","last_undegraded":"2024-04-13T05:16:14.487768+0000","last_fullsized":"2024-04-13T05:16:14.487768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501741+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:16:14.501741+0000","last_peered":"2024-04-13T05:16:14.501741+0000","last_clean":"2024-04-13T05:16:14.501741+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:16:14.501741+0000","last_undegraded":"2024-04-13T05:16:14.501741+0000","last_fullsized":"2024-04-13T05:16:14.501741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594599+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:16:41.594599+0000","last_peered":"2024-04-13T05:16:41.594599+0000","last_clean":"2024-04-13T05:16:41.594599+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:16:41.594599+0000","last_undegraded":"2024-04-13T05:16:41.594599+0000","last_fullsized":"2024-04-13T05:16:41.594599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584232+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:16:41.584232+0000","last_peered":"2024-04-13T05:16:41.584232+0000","last_clean":"2024-04-13T05:16:41.584232+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:16:41.584232+0000","last_undegraded":"2024-04-13T05:16:41.584232+0000","last_fullsized":"2024-04-13T05:16:41.584232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.583656+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:16:41.583656+0000","last_peered":"2024-04-13T05:16:41.583656+0000","last_clean":"2024-04-13T05:16:41.583656+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:16:41.583656+0000","last_undegraded":"2024-04-13T05:16:41.583656+0000","last_fullsized":"2024-04-13T05:16:41.583656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591341+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:16:41.591341+0000","last_peered":"2024-04-13T05:16:41.591341+0000","last_clean":"2024-04-13T05:16:41.591341+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:16:41.591341+0000","last_undegraded":"2024-04-13T05:16:41.591341+0000","last_fullsized":"2024-04-13T05:16:41.591341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589861+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:16:41.589861+0000","last_peered":"2024-04-13T05:16:41.589861+0000","last_clean":"2024-04-13T05:16:41.589861+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:16:41.589861+0000","last_undegraded":"2024-04-13T05:16:41.589861+0000","last_fullsized":"2024-04-13T05:16:41.589861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585560+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:16:41.585560+0000","last_peered":"2024-04-13T05:16:41.585560+0000","last_clean":"2024-04-13T05:16:41.585560+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:16:41.585560+0000","last_undegraded":"2024-04-13T05:16:41.585560+0000","last_fullsized":"2024-04-13T05:16:41.585560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571227+0000","last_change":"2024-04-13T05:16:14.714804+0000","last_active":"2024-04-13T05:16:41.571227+0000","last_peered":"2024-04-13T05:16:41.571227+0000","last_clean":"2024-04-13T05:16:41.571227+0000","last_became_active":"2024-04-13T05:16:14.713542+0000","last_became_peered":"2024-04-13T05:16:14.713542+0000","last_unstale":"2024-04-13T05:16:41.571227+0000","last_undegraded":"2024-04-13T05:16:41.571227+0000","last_fullsized":"2024-04-13T05:16:41.571227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:21:57.719452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589295+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:16:41.589295+0000","last_peered":"2024-04-13T05:16:41.589295+0000","last_clean":"2024-04-13T05:16:41.589295+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:16:41.589295+0000","last_undegraded":"2024-04-13T05:16:41.589295+0000","last_fullsized":"2024-04-13T05:16:41.589295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.570115+0000","last_change":"2024-04-13T05:16:14.713320+0000","last_active":"2024-04-13T05:16:41.570115+0000","last_peered":"2024-04-13T05:16:41.570115+0000","last_clean":"2024-04-13T05:16:41.570115+0000","last_became_active":"2024-04-13T05:16:14.712507+0000","last_became_peered":"2024-04-13T05:16:14.712507+0000","last_unstale":"2024-04-13T05:16:41.570115+0000","last_undegraded":"2024-04-13T05:16:41.570115+0000","last_fullsized":"2024-04-13T05:16:41.570115+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:07.206767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.598046+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:16:41.598046+0000","last_peered":"2024-04-13T05:16:41.598046+0000","last_clean":"2024-04-13T05:16:41.598046+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:16:41.598046+0000","last_undegraded":"2024-04-13T05:16:41.598046+0000","last_fullsized":"2024-04-13T05:16:41.598046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593528+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:16:41.593528+0000","last_peered":"2024-04-13T05:16:41.593528+0000","last_clean":"2024-04-13T05:16:41.593528+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:16:41.593528+0000","last_undegraded":"2024-04-13T05:16:41.593528+0000","last_fullsized":"2024-04-13T05:16:41.593528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599870+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:16:41.599870+0000","last_peered":"2024-04-13T05:16:41.599870+0000","last_clean":"2024-04-13T05:16:41.599870+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:16:41.599870+0000","last_undegraded":"2024-04-13T05:16:41.599870+0000","last_fullsized":"2024-04-13T05:16:41.599870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593751+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:16:41.593751+0000","last_peered":"2024-04-13T05:16:41.593751+0000","last_clean":"2024-04-13T05:16:41.593751+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:16:41.593751+0000","last_undegraded":"2024-04-13T05:16:41.593751+0000","last_fullsized":"2024-04-13T05:16:41.593751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571491+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:16:41.571491+0000","last_peered":"2024-04-13T05:16:41.571491+0000","last_clean":"2024-04-13T05:16:41.571491+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:16:41.571491+0000","last_undegraded":"2024-04-13T05:16:41.571491+0000","last_fullsized":"2024-04-13T05:16:41.571491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67016,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":520,"num_read_kb":334,"num_write":169,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32896,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712256,"statfs":{"total":95995035648,"available":95961350144,"internally_reserved":0,"allocated":1761280,"data_stored":1176987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":530645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070524,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":525887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332164,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1122519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":584621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":523113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":585747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":527489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757329,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1117575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:16:49.089 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph pg dump --format=json 2024-04-13T05:16:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:48 smithi189 ceph-mon[35531]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:16:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:48 smithi189 ceph-mon[35531]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:16:49.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:48 smithi083 ceph-mon[35133]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:16:49.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:48 smithi083 ceph-mon[35133]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:16:49.596 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:51.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:50 smithi057 ceph-mon[32331]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-13T05:16:51.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:50 smithi189 ceph-mon[35531]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-13T05:16:51.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:50 smithi083 ceph-mon[35133]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-13T05:16:51.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 -- 172.21.15.57:0/2855390148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 msgr2=0x7f131410ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:51.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 --2- 172.21.15.57:0/2855390148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131410ef70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009990 tx=0x7f12fc009ca0 comp rx=0 tx=0).stop 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 -- 172.21.15.57:0/2855390148 shutdown_connections 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 --2- 172.21.15.57:0/2855390148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1314074ce0 0x7f13140750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 --2- 172.21.15.57:0/2855390148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f131410f4b0 0x7f13141118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 --2- 172.21.15.57:0/2855390148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131410ef70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 -- 172.21.15.57:0/2855390148 >> 172.21.15.57:0/2855390148 conn(0x7f1314079c10 msgr2=0x7f131407a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 -- 172.21.15.57:0/2855390148 shutdown_connections 2024-04-13T05:16:51.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.882+0000 7f131b200700 1 -- 172.21.15.57:0/2855390148 wait complete. 2024-04-13T05:16:51.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.883+0000 7f131b200700 1 Processor -- start 2024-04-13T05:16:51.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.883+0000 7f131b200700 1 -- start start 2024-04-13T05:16:51.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f131b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314074ce0 0x7f131419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:51.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f131b200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f131b200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131410f4b0 0x7f131419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f131b200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1314114e70 con 0x7f1314075600 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f131b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1314114cf0 con 0x7f1314074ce0 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f131b200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1314114ff0 con 0x7f131410f4b0 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f130b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f130b400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42280/0 (socket says 172.21.15.57:42280) 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f130b400700 1 -- 172.21.15.57:0/988712366 learned_addr learned my addr 172.21.15.57:0/988712366 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f1319200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131410f4b0 0x7f131419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:51.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.884+0000 7f1313e00700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314074ce0 0x7f131419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.885+0000 7f130b400700 1 -- 172.21.15.57:0/988712366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131410f4b0 msgr2=0x7f131419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.885+0000 7f130b400700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131410f4b0 0x7f131419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.885+0000 7f130b400700 1 -- 172.21.15.57:0/988712366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314074ce0 msgr2=0x7f131419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:51.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.885+0000 7f130b400700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314074ce0 0x7f131419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.885+0000 7f130b400700 1 -- 172.21.15.57:0/988712366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13141a02b0 con 0x7f1314075600 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.885+0000 7f130b400700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131419b7e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12fc000c00 tx=0x7f12fc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.886+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc027070 con 0x7f1314075600 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.886+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12fc004010 con 0x7f1314075600 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.886+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13141a0540 con 0x7f1314075600 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.886+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc022c60 con 0x7f1314075600 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.886+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13141a7e20 con 0x7f1314075600 2024-04-13T05:16:51.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.887+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13140697a0 con 0x7f1314075600 2024-04-13T05:16:51.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.889+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f12fc02a070 con 0x7f1314075600 2024-04-13T05:16:51.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.891+0000 7f1311600700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f12f40a3d40 0x7f12f40a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:51.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.891+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f12fc022e00 con 0x7f1314075600 2024-04-13T05:16:51.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.891+0000 7f1313e00700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f12f40a3d40 0x7f12f40a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:51.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.892+0000 7f1313e00700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f12f40a3d40 0x7f12f40a6200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1304005950 tx=0x7f130400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:51.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:51.895+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12fc05eff0 con 0x7f1314075600 2024-04-13T05:16:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.119+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1314062ef0 con 0x7f12f40a3d40 2024-04-13T05:16:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.149+0000 7f1311600700 1 -- 172.21.15.57:0/988712366 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7f1314062ef0 con 0x7f12f40a3d40 2024-04-13T05:16:52.152 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:16:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f12f40a3d40 msgr2=0x7f12f40a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f12f40a3d40 0x7f12f40a6200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1304005950 tx=0x7f130400a300 comp rx=0 tx=0).stop 2024-04-13T05:16:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 msgr2=0x7f131419b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131419b7e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12fc000c00 tx=0x7f12fc0036f0 comp rx=0 tx=0).stop 2024-04-13T05:16:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 shutdown_connections 2024-04-13T05:16:52.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131410f4b0 0x7f131419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f12f40a3d40 0x7f12f40a6200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1314074ce0 0x7f131419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 --2- 172.21.15.57:0/988712366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1314075600 0x7f131419b7e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.157+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 >> 172.21.15.57:0/988712366 conn(0x7f1314079c10 msgr2=0x7f1314100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.158+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 shutdown_connections 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:52.158+0000 7f131b200700 1 -- 172.21.15.57:0/988712366 wait complete. 2024-04-13T05:16:52.161 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:16:53.008 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:52 smithi057 ceph-mon[32331]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:16:53.008 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:52 smithi057 ceph-mon[32331]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:16:53.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:52 smithi189 ceph-mon[35531]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:16:53.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:52 smithi189 ceph-mon[35531]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:16:53.085 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-04-13T05:16:51.645869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657618,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":676,"num_read_kb":458,"num_write":352,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367936,"kb_used_data":14624,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573888,"statfs":{"total":1151940427776,"available":1151563661312,"internally_reserved":0,"allocated":14974976,"data_stored":8264540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":33,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005117"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600603+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:16:41.600603+0000","last_peered":"2024-04-13T05:16:41.600603+0000","last_clean":"2024-04-13T05:16:41.600603+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:16:41.600603+0000","last_undegraded":"2024-04-13T05:16:41.600603+0000","last_fullsized":"2024-04-13T05:16:41.600603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593168+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:16:41.593168+0000","last_peered":"2024-04-13T05:16:41.593168+0000","last_clean":"2024-04-13T05:16:41.593168+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:16:41.593168+0000","last_undegraded":"2024-04-13T05:16:41.593168+0000","last_fullsized":"2024-04-13T05:16:41.593168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600943+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:16:41.600943+0000","last_peered":"2024-04-13T05:16:41.600943+0000","last_clean":"2024-04-13T05:16:41.600943+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:16:41.600943+0000","last_undegraded":"2024-04-13T05:16:41.600943+0000","last_fullsized":"2024-04-13T05:16:41.600943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.573728+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:16:41.573728+0000","last_peered":"2024-04-13T05:16:41.573728+0000","last_clean":"2024-04-13T05:16:41.573728+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:16:41.573728+0000","last_undegraded":"2024-04-13T05:16:41.573728+0000","last_fullsized":"2024-04-13T05:16:41.573728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.580106+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:16:41.580106+0000","last_peered":"2024-04-13T05:16:41.580106+0000","last_clean":"2024-04-13T05:16:41.580106+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:16:41.580106+0000","last_undegraded":"2024-04-13T05:16:41.580106+0000","last_fullsized":"2024-04-13T05:16:41.580106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591744+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:16:41.591744+0000","last_peered":"2024-04-13T05:16:41.591744+0000","last_clean":"2024-04-13T05:16:41.591744+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:16:41.591744+0000","last_undegraded":"2024-04-13T05:16:41.591744+0000","last_fullsized":"2024-04-13T05:16:41.591744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593200+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:16:41.593200+0000","last_peered":"2024-04-13T05:16:41.593200+0000","last_clean":"2024-04-13T05:16:41.593200+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:16:41.593200+0000","last_undegraded":"2024-04-13T05:16:41.593200+0000","last_fullsized":"2024-04-13T05:16:41.593200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596818+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:16:41.596818+0000","last_peered":"2024-04-13T05:16:41.596818+0000","last_clean":"2024-04-13T05:16:41.596818+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:16:41.596818+0000","last_undegraded":"2024-04-13T05:16:41.596818+0000","last_fullsized":"2024-04-13T05:16:41.596818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600243+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:16:41.600243+0000","last_peered":"2024-04-13T05:16:41.600243+0000","last_clean":"2024-04-13T05:16:41.600243+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:16:41.600243+0000","last_undegraded":"2024-04-13T05:16:41.600243+0000","last_fullsized":"2024-04-13T05:16:41.600243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595627+0000","last_change":"2024-04-13T05:16:14.217489+0000","last_active":"2024-04-13T05:16:41.595627+0000","last_peered":"2024-04-13T05:16:41.595627+0000","last_clean":"2024-04-13T05:16:41.595627+0000","last_became_active":"2024-04-13T05:16:14.216651+0000","last_became_peered":"2024-04-13T05:16:14.216651+0000","last_unstale":"2024-04-13T05:16:41.595627+0000","last_undegraded":"2024-04-13T05:16:41.595627+0000","last_fullsized":"2024-04-13T05:16:41.595627+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:55:10.808460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601686+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:16:41.601686+0000","last_peered":"2024-04-13T05:16:41.601686+0000","last_clean":"2024-04-13T05:16:41.601686+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:16:41.601686+0000","last_undegraded":"2024-04-13T05:16:41.601686+0000","last_fullsized":"2024-04-13T05:16:41.601686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585211+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:16:41.585211+0000","last_peered":"2024-04-13T05:16:41.585211+0000","last_clean":"2024-04-13T05:16:41.585211+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:16:41.585211+0000","last_undegraded":"2024-04-13T05:16:41.585211+0000","last_fullsized":"2024-04-13T05:16:41.585211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590222+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:16:41.590222+0000","last_peered":"2024-04-13T05:16:41.590222+0000","last_clean":"2024-04-13T05:16:41.590222+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:16:41.590222+0000","last_undegraded":"2024-04-13T05:16:41.590222+0000","last_fullsized":"2024-04-13T05:16:41.590222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595919+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:16:41.595919+0000","last_peered":"2024-04-13T05:16:41.595919+0000","last_clean":"2024-04-13T05:16:41.595919+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:16:41.595919+0000","last_undegraded":"2024-04-13T05:16:41.595919+0000","last_fullsized":"2024-04-13T05:16:41.595919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591659+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:16:41.591659+0000","last_peered":"2024-04-13T05:16:41.591659+0000","last_clean":"2024-04-13T05:16:41.591659+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:16:41.591659+0000","last_undegraded":"2024-04-13T05:16:41.591659+0000","last_fullsized":"2024-04-13T05:16:41.591659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596919+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:16:41.596919+0000","last_peered":"2024-04-13T05:16:41.596919+0000","last_clean":"2024-04-13T05:16:41.596919+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:16:41.596919+0000","last_undegraded":"2024-04-13T05:16:41.596919+0000","last_fullsized":"2024-04-13T05:16:41.596919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600704+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:16:41.600704+0000","last_peered":"2024-04-13T05:16:41.600704+0000","last_clean":"2024-04-13T05:16:41.600704+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:16:41.600704+0000","last_undegraded":"2024-04-13T05:16:41.600704+0000","last_fullsized":"2024-04-13T05:16:41.600704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589080+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:16:41.589080+0000","last_peered":"2024-04-13T05:16:41.589080+0000","last_clean":"2024-04-13T05:16:41.589080+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:16:41.589080+0000","last_undegraded":"2024-04-13T05:16:41.589080+0000","last_fullsized":"2024-04-13T05:16:41.589080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574707+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:16:41.574707+0000","last_peered":"2024-04-13T05:16:41.574707+0000","last_clean":"2024-04-13T05:16:41.574707+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:16:41.574707+0000","last_undegraded":"2024-04-13T05:16:41.574707+0000","last_fullsized":"2024-04-13T05:16:41.574707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207182+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:16:14.207182+0000","last_peered":"2024-04-13T05:16:14.207182+0000","last_clean":"2024-04-13T05:16:14.207182+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:16:14.207182+0000","last_undegraded":"2024-04-13T05:16:14.207182+0000","last_fullsized":"2024-04-13T05:16:14.207182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208504+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:16:14.208504+0000","last_peered":"2024-04-13T05:16:14.208504+0000","last_clean":"2024-04-13T05:16:14.208504+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:16:14.208504+0000","last_undegraded":"2024-04-13T05:16:14.208504+0000","last_fullsized":"2024-04-13T05:16:14.208504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596334+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:16:41.596334+0000","last_peered":"2024-04-13T05:16:41.596334+0000","last_clean":"2024-04-13T05:16:41.596334+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:16:41.596334+0000","last_undegraded":"2024-04-13T05:16:41.596334+0000","last_fullsized":"2024-04-13T05:16:41.596334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206864+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:16:14.206864+0000","last_peered":"2024-04-13T05:16:14.206864+0000","last_clean":"2024-04-13T05:16:14.206864+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:16:14.206864+0000","last_undegraded":"2024-04-13T05:16:14.206864+0000","last_fullsized":"2024-04-13T05:16:14.206864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206137+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:16:14.206137+0000","last_peered":"2024-04-13T05:16:14.206137+0000","last_clean":"2024-04-13T05:16:14.206137+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:16:14.206137+0000","last_undegraded":"2024-04-13T05:16:14.206137+0000","last_fullsized":"2024-04-13T05:16:14.206137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600848+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:16:41.600848+0000","last_peered":"2024-04-13T05:16:41.600848+0000","last_clean":"2024-04-13T05:16:41.600848+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:16:41.600848+0000","last_undegraded":"2024-04-13T05:16:41.600848+0000","last_fullsized":"2024-04-13T05:16:41.600848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207216+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:16:14.207216+0000","last_peered":"2024-04-13T05:16:14.207216+0000","last_clean":"2024-04-13T05:16:14.207216+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:16:14.207216+0000","last_undegraded":"2024-04-13T05:16:14.207216+0000","last_fullsized":"2024-04-13T05:16:14.207216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215001+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:16:14.215001+0000","last_peered":"2024-04-13T05:16:14.215001+0000","last_clean":"2024-04-13T05:16:14.215001+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:16:14.215001+0000","last_undegraded":"2024-04-13T05:16:14.215001+0000","last_fullsized":"2024-04-13T05:16:14.215001+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599872+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:16:41.599872+0000","last_peered":"2024-04-13T05:16:41.599872+0000","last_clean":"2024-04-13T05:16:41.599872+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:16:41.599872+0000","last_undegraded":"2024-04-13T05:16:41.599872+0000","last_fullsized":"2024-04-13T05:16:41.599872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206958+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:16:14.206958+0000","last_peered":"2024-04-13T05:16:14.206958+0000","last_clean":"2024-04-13T05:16:14.206958+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:16:14.206958+0000","last_undegraded":"2024-04-13T05:16:14.206958+0000","last_fullsized":"2024-04-13T05:16:14.206958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711513+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:16:14.711513+0000","last_peered":"2024-04-13T05:16:14.711513+0000","last_clean":"2024-04-13T05:16:14.711513+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:16:14.711513+0000","last_undegraded":"2024-04-13T05:16:14.711513+0000","last_fullsized":"2024-04-13T05:16:14.711513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601014+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:16:41.601014+0000","last_peered":"2024-04-13T05:16:41.601014+0000","last_clean":"2024-04-13T05:16:41.601014+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:16:41.601014+0000","last_undegraded":"2024-04-13T05:16:41.601014+0000","last_fullsized":"2024-04-13T05:16:41.601014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207933+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:16:14.207933+0000","last_peered":"2024-04-13T05:16:14.207933+0000","last_clean":"2024-04-13T05:16:14.207933+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:16:14.207933+0000","last_undegraded":"2024-04-13T05:16:14.207933+0000","last_fullsized":"2024-04-13T05:16:14.207933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":124,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:46.603873+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:16:46.603873+0000","last_peered":"2024-04-13T05:16:46.603873+0000","last_clean":"2024-04-13T05:16:46.603873+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:16:46.603873+0000","last_undegraded":"2024-04-13T05:16:46.603873+0000","last_fullsized":"2024-04-13T05:16:46.603873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590566+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:16:41.590566+0000","last_peered":"2024-04-13T05:16:41.590566+0000","last_clean":"2024-04-13T05:16:41.590566+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:16:41.590566+0000","last_undegraded":"2024-04-13T05:16:41.590566+0000","last_fullsized":"2024-04-13T05:16:41.590566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207515+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:16:14.207515+0000","last_peered":"2024-04-13T05:16:14.207515+0000","last_clean":"2024-04-13T05:16:14.207515+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:16:14.207515+0000","last_undegraded":"2024-04-13T05:16:14.207515+0000","last_fullsized":"2024-04-13T05:16:14.207515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.212620+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:16:14.212620+0000","last_peered":"2024-04-13T05:16:14.212620+0000","last_clean":"2024-04-13T05:16:14.212620+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:16:14.212620+0000","last_undegraded":"2024-04-13T05:16:14.212620+0000","last_fullsized":"2024-04-13T05:16:14.212620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597026+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:16:41.597026+0000","last_peered":"2024-04-13T05:16:41.597026+0000","last_clean":"2024-04-13T05:16:41.597026+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:16:41.597026+0000","last_undegraded":"2024-04-13T05:16:41.597026+0000","last_fullsized":"2024-04-13T05:16:41.597026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215333+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:16:14.215333+0000","last_peered":"2024-04-13T05:16:14.215333+0000","last_clean":"2024-04-13T05:16:14.215333+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:16:14.215333+0000","last_undegraded":"2024-04-13T05:16:14.215333+0000","last_fullsized":"2024-04-13T05:16:14.215333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.500669+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:16:14.500669+0000","last_peered":"2024-04-13T05:16:14.500669+0000","last_clean":"2024-04-13T05:16:14.500669+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:16:14.500669+0000","last_undegraded":"2024-04-13T05:16:14.500669+0000","last_fullsized":"2024-04-13T05:16:14.500669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584165+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:16:41.584165+0000","last_peered":"2024-04-13T05:16:41.584165+0000","last_clean":"2024-04-13T05:16:41.584165+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:16:41.584165+0000","last_undegraded":"2024-04-13T05:16:41.584165+0000","last_fullsized":"2024-04-13T05:16:41.584165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548845+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:16:14.548845+0000","last_peered":"2024-04-13T05:16:14.548845+0000","last_clean":"2024-04-13T05:16:14.548845+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:16:14.548845+0000","last_undegraded":"2024-04-13T05:16:14.548845+0000","last_fullsized":"2024-04-13T05:16:14.548845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488119+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:16:14.488119+0000","last_peered":"2024-04-13T05:16:14.488119+0000","last_clean":"2024-04-13T05:16:14.488119+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:16:14.488119+0000","last_undegraded":"2024-04-13T05:16:14.488119+0000","last_fullsized":"2024-04-13T05:16:14.488119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595176+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:16:41.595176+0000","last_peered":"2024-04-13T05:16:41.595176+0000","last_clean":"2024-04-13T05:16:41.595176+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:16:41.595176+0000","last_undegraded":"2024-04-13T05:16:41.595176+0000","last_fullsized":"2024-04-13T05:16:41.595176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205299+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:16:14.205299+0000","last_peered":"2024-04-13T05:16:14.205299+0000","last_clean":"2024-04-13T05:16:14.205299+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:16:14.205299+0000","last_undegraded":"2024-04-13T05:16:14.205299+0000","last_fullsized":"2024-04-13T05:16:14.205299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":121,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:47.605581+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:16:47.605581+0000","last_peered":"2024-04-13T05:16:47.605581+0000","last_clean":"2024-04-13T05:16:47.605581+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:16:47.605581+0000","last_undegraded":"2024-04-13T05:16:47.605581+0000","last_fullsized":"2024-04-13T05:16:47.605581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597354+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:16:41.597354+0000","last_peered":"2024-04-13T05:16:41.597354+0000","last_clean":"2024-04-13T05:16:41.597354+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:16:41.597354+0000","last_undegraded":"2024-04-13T05:16:41.597354+0000","last_fullsized":"2024-04-13T05:16:41.597354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207805+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:16:14.207805+0000","last_peered":"2024-04-13T05:16:14.207805+0000","last_clean":"2024-04-13T05:16:14.207805+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:16:14.207805+0000","last_undegraded":"2024-04-13T05:16:14.207805+0000","last_fullsized":"2024-04-13T05:16:14.207805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712209+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:16:14.712209+0000","last_peered":"2024-04-13T05:16:14.712209+0000","last_clean":"2024-04-13T05:16:14.712209+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:16:14.712209+0000","last_undegraded":"2024-04-13T05:16:14.712209+0000","last_fullsized":"2024-04-13T05:16:14.712209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589834+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:16:41.589834+0000","last_peered":"2024-04-13T05:16:41.589834+0000","last_clean":"2024-04-13T05:16:41.589834+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:16:41.589834+0000","last_undegraded":"2024-04-13T05:16:41.589834+0000","last_fullsized":"2024-04-13T05:16:41.589834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207816+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:16:14.207816+0000","last_peered":"2024-04-13T05:16:14.207816+0000","last_clean":"2024-04-13T05:16:14.207816+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:16:14.207816+0000","last_undegraded":"2024-04-13T05:16:14.207816+0000","last_fullsized":"2024-04-13T05:16:14.207816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489176+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:16:14.489176+0000","last_peered":"2024-04-13T05:16:14.489176+0000","last_clean":"2024-04-13T05:16:14.489176+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:16:14.489176+0000","last_undegraded":"2024-04-13T05:16:14.489176+0000","last_fullsized":"2024-04-13T05:16:14.489176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601329+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:16:41.601329+0000","last_peered":"2024-04-13T05:16:41.601329+0000","last_clean":"2024-04-13T05:16:41.601329+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:16:41.601329+0000","last_undegraded":"2024-04-13T05:16:41.601329+0000","last_fullsized":"2024-04-13T05:16:41.601329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208034+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:16:14.208034+0000","last_peered":"2024-04-13T05:16:14.208034+0000","last_clean":"2024-04-13T05:16:14.208034+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:16:14.208034+0000","last_undegraded":"2024-04-13T05:16:14.208034+0000","last_fullsized":"2024-04-13T05:16:14.208034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":250,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:48.621166+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:16:48.621166+0000","last_peered":"2024-04-13T05:16:48.621166+0000","last_clean":"2024-04-13T05:16:48.621166+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:16:48.621166+0000","last_undegraded":"2024-04-13T05:16:48.621166+0000","last_fullsized":"2024-04-13T05:16:48.621166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":184,"num_read_kb":155,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595634+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:16:41.595634+0000","last_peered":"2024-04-13T05:16:41.595634+0000","last_clean":"2024-04-13T05:16:41.595634+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:16:41.595634+0000","last_undegraded":"2024-04-13T05:16:41.595634+0000","last_fullsized":"2024-04-13T05:16:41.595634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489985+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:16:14.489985+0000","last_peered":"2024-04-13T05:16:14.489985+0000","last_clean":"2024-04-13T05:16:14.489985+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:16:14.489985+0000","last_undegraded":"2024-04-13T05:16:14.489985+0000","last_fullsized":"2024-04-13T05:16:14.489985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207679+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:16:14.207679+0000","last_peered":"2024-04-13T05:16:14.207679+0000","last_clean":"2024-04-13T05:16:14.207679+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:16:14.207679+0000","last_undegraded":"2024-04-13T05:16:14.207679+0000","last_fullsized":"2024-04-13T05:16:14.207679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590833+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:16:41.590833+0000","last_peered":"2024-04-13T05:16:41.590833+0000","last_clean":"2024-04-13T05:16:41.590833+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:16:41.590833+0000","last_undegraded":"2024-04-13T05:16:41.590833+0000","last_fullsized":"2024-04-13T05:16:41.590833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214342+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:16:14.214342+0000","last_peered":"2024-04-13T05:16:14.214342+0000","last_clean":"2024-04-13T05:16:14.214342+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:16:14.214342+0000","last_undegraded":"2024-04-13T05:16:14.214342+0000","last_fullsized":"2024-04-13T05:16:14.214342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488741+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:16:14.488741+0000","last_peered":"2024-04-13T05:16:14.488741+0000","last_clean":"2024-04-13T05:16:14.488741+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:16:14.488741+0000","last_undegraded":"2024-04-13T05:16:14.488741+0000","last_fullsized":"2024-04-13T05:16:14.488741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596455+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:16:41.596455+0000","last_peered":"2024-04-13T05:16:41.596455+0000","last_clean":"2024-04-13T05:16:41.596455+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:16:41.596455+0000","last_undegraded":"2024-04-13T05:16:41.596455+0000","last_fullsized":"2024-04-13T05:16:41.596455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208654+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:16:14.208654+0000","last_peered":"2024-04-13T05:16:14.208654+0000","last_clean":"2024-04-13T05:16:14.208654+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:16:14.208654+0000","last_undegraded":"2024-04-13T05:16:14.208654+0000","last_fullsized":"2024-04-13T05:16:14.208654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":126,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:46.613603+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:16:46.613603+0000","last_peered":"2024-04-13T05:16:46.613603+0000","last_clean":"2024-04-13T05:16:46.613603+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:16:46.613603+0000","last_undegraded":"2024-04-13T05:16:46.613603+0000","last_fullsized":"2024-04-13T05:16:46.613603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593508+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:16:41.593508+0000","last_peered":"2024-04-13T05:16:41.593508+0000","last_clean":"2024-04-13T05:16:41.593508+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:16:41.593508+0000","last_undegraded":"2024-04-13T05:16:41.593508+0000","last_fullsized":"2024-04-13T05:16:41.593508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501984+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:16:14.501984+0000","last_peered":"2024-04-13T05:16:14.501984+0000","last_clean":"2024-04-13T05:16:14.501984+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:16:14.501984+0000","last_undegraded":"2024-04-13T05:16:14.501984+0000","last_fullsized":"2024-04-13T05:16:14.501984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709675+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:16:14.709675+0000","last_peered":"2024-04-13T05:16:14.709675+0000","last_clean":"2024-04-13T05:16:14.709675+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:16:14.709675+0000","last_undegraded":"2024-04-13T05:16:14.709675+0000","last_fullsized":"2024-04-13T05:16:14.709675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600435+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:16:41.600435+0000","last_peered":"2024-04-13T05:16:41.600435+0000","last_clean":"2024-04-13T05:16:41.600435+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:16:41.600435+0000","last_undegraded":"2024-04-13T05:16:41.600435+0000","last_fullsized":"2024-04-13T05:16:41.600435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488267+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:16:14.488267+0000","last_peered":"2024-04-13T05:16:14.488267+0000","last_clean":"2024-04-13T05:16:14.488267+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:16:14.488267+0000","last_undegraded":"2024-04-13T05:16:14.488267+0000","last_fullsized":"2024-04-13T05:16:14.488267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710548+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:16:14.710548+0000","last_peered":"2024-04-13T05:16:14.710548+0000","last_clean":"2024-04-13T05:16:14.710548+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:16:14.710548+0000","last_undegraded":"2024-04-13T05:16:14.710548+0000","last_fullsized":"2024-04-13T05:16:14.710548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.583945+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:16:41.583945+0000","last_peered":"2024-04-13T05:16:41.583945+0000","last_clean":"2024-04-13T05:16:41.583945+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:16:41.583945+0000","last_undegraded":"2024-04-13T05:16:41.583945+0000","last_fullsized":"2024-04-13T05:16:41.583945+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711126+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:16:14.711126+0000","last_peered":"2024-04-13T05:16:14.711126+0000","last_clean":"2024-04-13T05:16:14.711126+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:16:14.711126+0000","last_undegraded":"2024-04-13T05:16:14.711126+0000","last_fullsized":"2024-04-13T05:16:14.711126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208216+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:16:14.208216+0000","last_peered":"2024-04-13T05:16:14.208216+0000","last_clean":"2024-04-13T05:16:14.208216+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:16:14.208216+0000","last_undegraded":"2024-04-13T05:16:14.208216+0000","last_fullsized":"2024-04-13T05:16:14.208216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596822+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:16:41.596822+0000","last_peered":"2024-04-13T05:16:41.596822+0000","last_clean":"2024-04-13T05:16:41.596822+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:16:41.596822+0000","last_undegraded":"2024-04-13T05:16:41.596822+0000","last_fullsized":"2024-04-13T05:16:41.596822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501136+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:16:14.501136+0000","last_peered":"2024-04-13T05:16:14.501136+0000","last_clean":"2024-04-13T05:16:14.501136+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:16:14.501136+0000","last_undegraded":"2024-04-13T05:16:14.501136+0000","last_fullsized":"2024-04-13T05:16:14.501136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711674+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:16:14.711674+0000","last_peered":"2024-04-13T05:16:14.711674+0000","last_clean":"2024-04-13T05:16:14.711674+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:16:14.711674+0000","last_undegraded":"2024-04-13T05:16:14.711674+0000","last_fullsized":"2024-04-13T05:16:14.711674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591381+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:16:41.591381+0000","last_peered":"2024-04-13T05:16:41.591381+0000","last_clean":"2024-04-13T05:16:41.591381+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:16:41.591381+0000","last_undegraded":"2024-04-13T05:16:41.591381+0000","last_fullsized":"2024-04-13T05:16:41.591381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501697+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:16:14.501697+0000","last_peered":"2024-04-13T05:16:14.501697+0000","last_clean":"2024-04-13T05:16:14.501697+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:16:14.501697+0000","last_undegraded":"2024-04-13T05:16:14.501697+0000","last_fullsized":"2024-04-13T05:16:14.501697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586794+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:16:41.586794+0000","last_peered":"2024-04-13T05:16:41.586794+0000","last_clean":"2024-04-13T05:16:41.586794+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:16:41.586794+0000","last_undegraded":"2024-04-13T05:16:41.586794+0000","last_fullsized":"2024-04-13T05:16:41.586794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":123,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:47.615547+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:16:47.615547+0000","last_peered":"2024-04-13T05:16:47.615547+0000","last_clean":"2024-04-13T05:16:47.615547+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:16:47.615547+0000","last_undegraded":"2024-04-13T05:16:47.615547+0000","last_fullsized":"2024-04-13T05:16:47.615547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712433+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:16:14.712433+0000","last_peered":"2024-04-13T05:16:14.712433+0000","last_clean":"2024-04-13T05:16:14.712433+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:16:14.712433+0000","last_undegraded":"2024-04-13T05:16:14.712433+0000","last_fullsized":"2024-04-13T05:16:14.712433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601957+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:16:41.601957+0000","last_peered":"2024-04-13T05:16:41.601957+0000","last_clean":"2024-04-13T05:16:41.601957+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:16:41.601957+0000","last_undegraded":"2024-04-13T05:16:41.601957+0000","last_fullsized":"2024-04-13T05:16:41.601957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207216+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:16:14.207216+0000","last_peered":"2024-04-13T05:16:14.207216+0000","last_clean":"2024-04-13T05:16:14.207216+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:16:14.207216+0000","last_undegraded":"2024-04-13T05:16:14.207216+0000","last_fullsized":"2024-04-13T05:16:14.207216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710982+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:16:14.710982+0000","last_peered":"2024-04-13T05:16:14.710982+0000","last_clean":"2024-04-13T05:16:14.710982+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:16:14.710982+0000","last_undegraded":"2024-04-13T05:16:14.710982+0000","last_fullsized":"2024-04-13T05:16:14.710982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489979+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:16:14.489979+0000","last_peered":"2024-04-13T05:16:14.489979+0000","last_clean":"2024-04-13T05:16:14.489979+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:16:14.489979+0000","last_undegraded":"2024-04-13T05:16:14.489979+0000","last_fullsized":"2024-04-13T05:16:14.489979+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574284+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:16:41.574284+0000","last_peered":"2024-04-13T05:16:41.574284+0000","last_clean":"2024-04-13T05:16:41.574284+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:16:41.574284+0000","last_undegraded":"2024-04-13T05:16:41.574284+0000","last_fullsized":"2024-04-13T05:16:41.574284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711264+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:16:14.711264+0000","last_peered":"2024-04-13T05:16:14.711264+0000","last_clean":"2024-04-13T05:16:14.711264+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:16:14.711264+0000","last_undegraded":"2024-04-13T05:16:14.711264+0000","last_fullsized":"2024-04-13T05:16:14.711264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214529+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:16:14.214529+0000","last_peered":"2024-04-13T05:16:14.214529+0000","last_clean":"2024-04-13T05:16:14.214529+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:16:14.214529+0000","last_undegraded":"2024-04-13T05:16:14.214529+0000","last_fullsized":"2024-04-13T05:16:14.214529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710784+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:16:14.710784+0000","last_peered":"2024-04-13T05:16:14.710784+0000","last_clean":"2024-04-13T05:16:14.710784+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:16:14.710784+0000","last_undegraded":"2024-04-13T05:16:14.710784+0000","last_fullsized":"2024-04-13T05:16:14.710784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574646+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:16:41.574646+0000","last_peered":"2024-04-13T05:16:41.574646+0000","last_clean":"2024-04-13T05:16:41.574646+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:16:41.574646+0000","last_undegraded":"2024-04-13T05:16:41.574646+0000","last_fullsized":"2024-04-13T05:16:41.574646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205157+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:16:14.205157+0000","last_peered":"2024-04-13T05:16:14.205157+0000","last_clean":"2024-04-13T05:16:14.205157+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:16:14.205157+0000","last_undegraded":"2024-04-13T05:16:14.205157+0000","last_fullsized":"2024-04-13T05:16:14.205157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208389+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:16:14.208389+0000","last_peered":"2024-04-13T05:16:14.208389+0000","last_clean":"2024-04-13T05:16:14.208389+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:16:14.208389+0000","last_undegraded":"2024-04-13T05:16:14.208389+0000","last_fullsized":"2024-04-13T05:16:14.208389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594139+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:16:41.594139+0000","last_peered":"2024-04-13T05:16:41.594139+0000","last_clean":"2024-04-13T05:16:41.594139+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:16:41.594139+0000","last_undegraded":"2024-04-13T05:16:41.594139+0000","last_fullsized":"2024-04-13T05:16:41.594139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206843+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:16:14.206843+0000","last_peered":"2024-04-13T05:16:14.206843+0000","last_clean":"2024-04-13T05:16:14.206843+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:16:14.206843+0000","last_undegraded":"2024-04-13T05:16:14.206843+0000","last_fullsized":"2024-04-13T05:16:14.206843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208960+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:16:14.208960+0000","last_peered":"2024-04-13T05:16:14.208960+0000","last_clean":"2024-04-13T05:16:14.208960+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:16:14.208960+0000","last_undegraded":"2024-04-13T05:16:14.208960+0000","last_fullsized":"2024-04-13T05:16:14.208960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595936+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:16:41.595936+0000","last_peered":"2024-04-13T05:16:41.595936+0000","last_clean":"2024-04-13T05:16:41.595936+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:16:41.595936+0000","last_undegraded":"2024-04-13T05:16:41.595936+0000","last_fullsized":"2024-04-13T05:16:41.595936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208468+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:16:14.208468+0000","last_peered":"2024-04-13T05:16:14.208468+0000","last_clean":"2024-04-13T05:16:14.208468+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:16:14.208468+0000","last_undegraded":"2024-04-13T05:16:14.208468+0000","last_fullsized":"2024-04-13T05:16:14.208468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709332+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:16:14.709332+0000","last_peered":"2024-04-13T05:16:14.709332+0000","last_clean":"2024-04-13T05:16:14.709332+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:16:14.709332+0000","last_undegraded":"2024-04-13T05:16:14.709332+0000","last_fullsized":"2024-04-13T05:16:14.709332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.532767+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:16:41.532767+0000","last_peered":"2024-04-13T05:16:41.532767+0000","last_clean":"2024-04-13T05:16:41.532767+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:16:41.532767+0000","last_undegraded":"2024-04-13T05:16:41.532767+0000","last_fullsized":"2024-04-13T05:16:41.532767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208551+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:16:14.208551+0000","last_peered":"2024-04-13T05:16:14.208551+0000","last_clean":"2024-04-13T05:16:14.208551+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:16:14.208551+0000","last_undegraded":"2024-04-13T05:16:14.208551+0000","last_fullsized":"2024-04-13T05:16:14.208551+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205417+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:16:14.205417+0000","last_peered":"2024-04-13T05:16:14.205417+0000","last_clean":"2024-04-13T05:16:14.205417+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:16:14.205417+0000","last_undegraded":"2024-04-13T05:16:14.205417+0000","last_fullsized":"2024-04-13T05:16:14.205417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591171+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:16:41.591171+0000","last_peered":"2024-04-13T05:16:41.591171+0000","last_clean":"2024-04-13T05:16:41.591171+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:16:41.591171+0000","last_undegraded":"2024-04-13T05:16:41.591171+0000","last_fullsized":"2024-04-13T05:16:41.591171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207020+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:16:14.207020+0000","last_peered":"2024-04-13T05:16:14.207020+0000","last_clean":"2024-04-13T05:16:14.207020+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:16:14.207020+0000","last_undegraded":"2024-04-13T05:16:14.207020+0000","last_fullsized":"2024-04-13T05:16:14.207020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214036+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:16:14.214036+0000","last_peered":"2024-04-13T05:16:14.214036+0000","last_clean":"2024-04-13T05:16:14.214036+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:16:14.214036+0000","last_undegraded":"2024-04-13T05:16:14.214036+0000","last_fullsized":"2024-04-13T05:16:14.214036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597636+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:16:41.597636+0000","last_peered":"2024-04-13T05:16:41.597636+0000","last_clean":"2024-04-13T05:16:41.597636+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:16:41.597636+0000","last_undegraded":"2024-04-13T05:16:41.597636+0000","last_fullsized":"2024-04-13T05:16:41.597636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208102+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:16:14.208102+0000","last_peered":"2024-04-13T05:16:14.208102+0000","last_clean":"2024-04-13T05:16:14.208102+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:16:14.208102+0000","last_undegraded":"2024-04-13T05:16:14.208102+0000","last_fullsized":"2024-04-13T05:16:14.208102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710990+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:16:14.710990+0000","last_peered":"2024-04-13T05:16:14.710990+0000","last_clean":"2024-04-13T05:16:14.710990+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:16:14.710990+0000","last_undegraded":"2024-04-13T05:16:14.710990+0000","last_fullsized":"2024-04-13T05:16:14.710990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601865+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:16:41.601865+0000","last_peered":"2024-04-13T05:16:41.601865+0000","last_clean":"2024-04-13T05:16:41.601865+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:16:41.601865+0000","last_undegraded":"2024-04-13T05:16:41.601865+0000","last_fullsized":"2024-04-13T05:16:41.601865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709076+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:16:14.709076+0000","last_peered":"2024-04-13T05:16:14.709076+0000","last_clean":"2024-04-13T05:16:14.709076+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:16:14.709076+0000","last_undegraded":"2024-04-13T05:16:14.709076+0000","last_fullsized":"2024-04-13T05:16:14.709076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207927+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:16:14.207927+0000","last_peered":"2024-04-13T05:16:14.207927+0000","last_clean":"2024-04-13T05:16:14.207927+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:16:14.207927+0000","last_undegraded":"2024-04-13T05:16:14.207927+0000","last_fullsized":"2024-04-13T05:16:14.207927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601679+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:16:41.601679+0000","last_peered":"2024-04-13T05:16:41.601679+0000","last_clean":"2024-04-13T05:16:41.601679+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:16:41.601679+0000","last_undegraded":"2024-04-13T05:16:41.601679+0000","last_fullsized":"2024-04-13T05:16:41.601679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548084+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:16:14.548084+0000","last_peered":"2024-04-13T05:16:14.548084+0000","last_clean":"2024-04-13T05:16:14.548084+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:16:14.548084+0000","last_undegraded":"2024-04-13T05:16:14.548084+0000","last_fullsized":"2024-04-13T05:16:14.548084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489530+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:16:14.489530+0000","last_peered":"2024-04-13T05:16:14.489530+0000","last_clean":"2024-04-13T05:16:14.489530+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:16:14.489530+0000","last_undegraded":"2024-04-13T05:16:14.489530+0000","last_fullsized":"2024-04-13T05:16:14.489530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585748+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:16:41.585748+0000","last_peered":"2024-04-13T05:16:41.585748+0000","last_clean":"2024-04-13T05:16:41.585748+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:16:41.585748+0000","last_undegraded":"2024-04-13T05:16:41.585748+0000","last_fullsized":"2024-04-13T05:16:41.585748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.487768+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:16:14.487768+0000","last_peered":"2024-04-13T05:16:14.487768+0000","last_clean":"2024-04-13T05:16:14.487768+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:16:14.487768+0000","last_undegraded":"2024-04-13T05:16:14.487768+0000","last_fullsized":"2024-04-13T05:16:14.487768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501741+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:16:14.501741+0000","last_peered":"2024-04-13T05:16:14.501741+0000","last_clean":"2024-04-13T05:16:14.501741+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:16:14.501741+0000","last_undegraded":"2024-04-13T05:16:14.501741+0000","last_fullsized":"2024-04-13T05:16:14.501741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594599+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:16:41.594599+0000","last_peered":"2024-04-13T05:16:41.594599+0000","last_clean":"2024-04-13T05:16:41.594599+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:16:41.594599+0000","last_undegraded":"2024-04-13T05:16:41.594599+0000","last_fullsized":"2024-04-13T05:16:41.594599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584232+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:16:41.584232+0000","last_peered":"2024-04-13T05:16:41.584232+0000","last_clean":"2024-04-13T05:16:41.584232+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:16:41.584232+0000","last_undegraded":"2024-04-13T05:16:41.584232+0000","last_fullsized":"2024-04-13T05:16:41.584232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.583656+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:16:41.583656+0000","last_peered":"2024-04-13T05:16:41.583656+0000","last_clean":"2024-04-13T05:16:41.583656+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:16:41.583656+0000","last_undegraded":"2024-04-13T05:16:41.583656+0000","last_fullsized":"2024-04-13T05:16:41.583656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591341+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:16:41.591341+0000","last_peered":"2024-04-13T05:16:41.591341+0000","last_clean":"2024-04-13T05:16:41.591341+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:16:41.591341+0000","last_undegraded":"2024-04-13T05:16:41.591341+0000","last_fullsized":"2024-04-13T05:16:41.591341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589861+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:16:41.589861+0000","last_peered":"2024-04-13T05:16:41.589861+0000","last_clean":"2024-04-13T05:16:41.589861+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:16:41.589861+0000","last_undegraded":"2024-04-13T05:16:41.589861+0000","last_fullsized":"2024-04-13T05:16:41.589861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585560+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:16:41.585560+0000","last_peered":"2024-04-13T05:16:41.585560+0000","last_clean":"2024-04-13T05:16:41.585560+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:16:41.585560+0000","last_undegraded":"2024-04-13T05:16:41.585560+0000","last_fullsized":"2024-04-13T05:16:41.585560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571227+0000","last_change":"2024-04-13T05:16:14.714804+0000","last_active":"2024-04-13T05:16:41.571227+0000","last_peered":"2024-04-13T05:16:41.571227+0000","last_clean":"2024-04-13T05:16:41.571227+0000","last_became_active":"2024-04-13T05:16:14.713542+0000","last_became_peered":"2024-04-13T05:16:14.713542+0000","last_unstale":"2024-04-13T05:16:41.571227+0000","last_undegraded":"2024-04-13T05:16:41.571227+0000","last_fullsized":"2024-04-13T05:16:41.571227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:21:57.719452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.589295+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:16:41.589295+0000","last_peered":"2024-04-13T05:16:41.589295+0000","last_clean":"2024-04-13T05:16:41.589295+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:16:41.589295+0000","last_undegraded":"2024-04-13T05:16:41.589295+0000","last_fullsized":"2024-04-13T05:16:41.589295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.570115+0000","last_change":"2024-04-13T05:16:14.713320+0000","last_active":"2024-04-13T05:16:41.570115+0000","last_peered":"2024-04-13T05:16:41.570115+0000","last_clean":"2024-04-13T05:16:41.570115+0000","last_became_active":"2024-04-13T05:16:14.712507+0000","last_became_peered":"2024-04-13T05:16:14.712507+0000","last_unstale":"2024-04-13T05:16:41.570115+0000","last_undegraded":"2024-04-13T05:16:41.570115+0000","last_fullsized":"2024-04-13T05:16:41.570115+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:07.206767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.598046+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:16:41.598046+0000","last_peered":"2024-04-13T05:16:41.598046+0000","last_clean":"2024-04-13T05:16:41.598046+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:16:41.598046+0000","last_undegraded":"2024-04-13T05:16:41.598046+0000","last_fullsized":"2024-04-13T05:16:41.598046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593528+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:16:41.593528+0000","last_peered":"2024-04-13T05:16:41.593528+0000","last_clean":"2024-04-13T05:16:41.593528+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:16:41.593528+0000","last_undegraded":"2024-04-13T05:16:41.593528+0000","last_fullsized":"2024-04-13T05:16:41.593528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599870+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:16:41.599870+0000","last_peered":"2024-04-13T05:16:41.599870+0000","last_clean":"2024-04-13T05:16:41.599870+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:16:41.599870+0000","last_undegraded":"2024-04-13T05:16:41.599870+0000","last_fullsized":"2024-04-13T05:16:41.599870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593751+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:16:41.593751+0000","last_peered":"2024-04-13T05:16:41.593751+0000","last_clean":"2024-04-13T05:16:41.593751+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:16:41.593751+0000","last_undegraded":"2024-04-13T05:16:41.593751+0000","last_fullsized":"2024-04-13T05:16:41.593751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571491+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:16:41.571491+0000","last_peered":"2024-04-13T05:16:41.571491+0000","last_clean":"2024-04-13T05:16:41.571491+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:16:41.571491+0000","last_undegraded":"2024-04-13T05:16:41.571491+0000","last_fullsized":"2024-04-13T05:16:41.571491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67016,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":550,"num_read_kb":349,"num_write":169,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32896,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712256,"statfs":{"total":95995035648,"available":95961350144,"internally_reserved":0,"allocated":1761280,"data_stored":1176987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":530645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":525887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1122519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":584621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":523113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":585747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":527489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1117575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:16:53.090 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-13T05:16:53.090 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-13T05:16:53.090 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-13T05:16:53.090 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph health --format=json 2024-04-13T05:16:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:52 smithi083 ceph-mon[35133]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:16:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:52 smithi083 ceph-mon[35133]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:16:53.614 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:16:55.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:54 smithi057 ceph-mon[32331]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-13T05:16:55.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:54 smithi189 ceph-mon[35531]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-13T05:16:55.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:54 smithi083 ceph-mon[35133]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-13T05:16:56.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 -- 172.21.15.57:0/3836770631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 msgr2=0x7f8ca4111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:56.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 --2- 172.21.15.57:0/3836770631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca4111be0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099c0 tx=0x7f8c9c009cd0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 -- 172.21.15.57:0/3836770631 shutdown_connections 2024-04-13T05:16:56.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 --2- 172.21.15.57:0/3836770631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca4104f30 0x7f8ca4105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 --2- 172.21.15.57:0/3836770631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca4111be0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 --2- 172.21.15.57:0/3836770631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca41058e0 0x7f8ca4109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.059+0000 7f8caba00700 1 -- 172.21.15.57:0/3836770631 >> 172.21.15.57:0/3836770631 conn(0x7f8ca41007c0 msgr2=0x7f8ca4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:56.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.060+0000 7f8caba00700 1 -- 172.21.15.57:0/3836770631 shutdown_connections 2024-04-13T05:16:56.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.060+0000 7f8caba00700 1 -- 172.21.15.57:0/3836770631 wait complete. 2024-04-13T05:16:56.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 Processor -- start 2024-04-13T05:16:56.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 -- start start 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca4104f30 0x7f8ca419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca41058e0 0x7f8ca419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca4114f00 con 0x7f8ca410a010 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca4114d80 con 0x7f8ca4104f30 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.061+0000 7f8caba00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca4115080 con 0x7f8ca41058e0 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51782/0 (socket says 172.21.15.57:51782) 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 -- 172.21.15.57:0/3726981036 learned_addr learned my addr 172.21.15.57:0/3726981036 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9000700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca4104f30 0x7f8ca419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:56.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca3e00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca41058e0 0x7f8ca419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 -- 172.21.15.57:0/3726981036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca41058e0 msgr2=0x7f8ca419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca41058e0 0x7f8ca419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 -- 172.21.15.57:0/3726981036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca4104f30 msgr2=0x7f8ca419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca4104f30 0x7f8ca419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.062+0000 7f8ca9a00700 1 -- 172.21.15.57:0/3726981036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca41a0370 con 0x7f8ca410a010 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.063+0000 7f8ca9a00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca419fc30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0045e0 tx=0x7f8c9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.063+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c009e60 con 0x7f8ca410a010 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.063+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c9c004830 con 0x7f8ca410a010 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.063+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca41a0600 con 0x7f8ca410a010 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.063+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c005280 con 0x7f8ca410a010 2024-04-13T05:16:56.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.063+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41a7ee0 con 0x7f8ca410a010 2024-04-13T05:16:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.064+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca4106410 con 0x7f8ca410a010 2024-04-13T05:16:56.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.066+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8c9c0049e0 con 0x7f8ca410a010 2024-04-13T05:16:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.068+0000 7f8ca1600700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8c8c0a3c40 0x7f8c8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:56.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.068+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8c9c0d1520 con 0x7f8ca410a010 2024-04-13T05:16:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.068+0000 7f8ca9000700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8c8c0a3c40 0x7f8c8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:56.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.069+0000 7f8ca9000700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8c8c0a3c40 0x7f8c8c0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009cf0 tx=0x7f8c98009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:56.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.072+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c9c013070 con 0x7f8ca410a010 2024-04-13T05:16:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.342+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ca419c6c0 con 0x7f8ca410a010 2024-04-13T05:16:56.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.343+0000 7f8ca1600700 1 -- 172.21.15.57:0/3726981036 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8c9c09db90 con 0x7f8ca410a010 2024-04-13T05:16:56.345 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:16:56.345 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-13T05:16:56.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.345+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8c8c0a3c40 msgr2=0x7f8c8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:56.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.345+0000 7f8caba00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8c8c0a3c40 0x7f8c8c0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009cf0 tx=0x7f8c98009040 comp rx=0 tx=0).stop 2024-04-13T05:16:56.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.345+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 msgr2=0x7f8ca419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:56.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.345+0000 7f8caba00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca419fc30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0045e0 tx=0x7f8c9c0046c0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 shutdown_connections 2024-04-13T05:16:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8c8c0a3c40 0x7f8c8c0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca410a010 0x7f8ca419fc30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ca4104f30 0x7f8ca419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 --2- 172.21.15.57:0/3726981036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca41058e0 0x7f8ca419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:56.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 >> 172.21.15.57:0/3726981036 conn(0x7f8ca41007c0 msgr2=0x7f8ca4101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:56.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 shutdown_connections 2024-04-13T05:16:56.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:56.346+0000 7f8caba00700 1 -- 172.21.15.57:0/3726981036 wait complete. 2024-04-13T05:16:57.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:56 smithi057 ceph-mon[32331]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-13T05:16:57.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:56 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3726981036' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-13T05:16:57.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:56 smithi189 ceph-mon[35531]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-13T05:16:57.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:56 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3726981036' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-13T05:16:57.175 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-13T05:16:57.176 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:16:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:56 smithi083 ceph-mon[35133]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-13T05:16:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:56 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3726981036' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-13T05:16:57.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 -- 172.21.15.57:0/3319231518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 msgr2=0x7f7194110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 --2- 172.21.15.57:0/3319231518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f7194110880 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f718c009ca0 tx=0x7f718c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 -- 172.21.15.57:0/3319231518 shutdown_connections 2024-04-13T05:16:57.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 --2- 172.21.15.57:0/3319231518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194109630 0x7f7194109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 --2- 172.21.15.57:0/3319231518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f7194110880 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 --2- 172.21.15.57:0/3319231518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194108440 0x7f7194108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 -- 172.21.15.57:0/3319231518 >> 172.21.15.57:0/3319231518 conn(0x7f7194077400 msgr2=0x7f7194079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:57.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 -- 172.21.15.57:0/3319231518 shutdown_connections 2024-04-13T05:16:57.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.407+0000 7f7193e00640 1 -- 172.21.15.57:0/3319231518 wait complete. 2024-04-13T05:16:57.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.408+0000 7f7193e00640 1 Processor -- start 2024-04-13T05:16:57.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 -- start start 2024-04-13T05:16:57.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194108440 0x7f719419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194109630 0x7f719419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f71941a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7194113f80 con 0x7f7194109ff0 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7194113e00 con 0x7f7194108440 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7194114100 con 0x7f7194109630 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f71941a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f71941a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51808/0 (socket says 172.21.15.57:51808) 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7193400640 1 -- 172.21.15.57:0/663705540 learned_addr learned my addr 172.21.15.57:0/663705540 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7192a00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194108440 0x7f719419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.409+0000 7f7192000640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194109630 0x7f719419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193400640 1 -- 172.21.15.57:0/663705540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194109630 msgr2=0x7f719419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193400640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194109630 0x7f719419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193400640 1 -- 172.21.15.57:0/663705540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194108440 msgr2=0x7f719419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193400640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194108440 0x7f719419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193400640 1 -- 172.21.15.57:0/663705540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71941a6210 con 0x7f7194109ff0 2024-04-13T05:16:57.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193400640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f71941a5ba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f718c02c700 tx=0x7f718c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:57.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c041020 con 0x7f7194109ff0 2024-04-13T05:16:57.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.410+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71941a64a0 con 0x7f7194109ff0 2024-04-13T05:16:57.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.411+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f718c003e20 con 0x7f7194109ff0 2024-04-13T05:16:57.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.411+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c039040 con 0x7f7194109ff0 2024-04-13T05:16:57.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.411+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71941a69e0 con 0x7f7194109ff0 2024-04-13T05:16:57.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.412+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719410f760 con 0x7f7194109ff0 2024-04-13T05:16:57.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.414+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f718c044020 con 0x7f7194109ff0 2024-04-13T05:16:57.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.415+0000 7f718b400640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71780a3bb0 0x7f71780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.415+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f718c0f2cf0 con 0x7f7194109ff0 2024-04-13T05:16:57.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.416+0000 7f7192a00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71780a3bb0 0x7f71780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.416+0000 7f7192a00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71780a3bb0 0x7f71780a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7184005fd0 tx=0x7f7184005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:57.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.420+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f718c04b020 con 0x7f7194109ff0 2024-04-13T05:16:57.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.629+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7194062bc0 con 0x7f7194109ff0 2024-04-13T05:16:57.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.634+0000 7f718b400640 1 -- 172.21.15.57:0/663705540 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f718c03b2f0 con 0x7f7194109ff0 2024-04-13T05:16:57.636 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:16:57.636 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:16:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71780a3bb0 msgr2=0x7f71780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71780a3bb0 0x7f71780a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7184005fd0 tx=0x7f7184005850 comp rx=0 tx=0).stop 2024-04-13T05:16:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 msgr2=0x7f71941a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f71941a5ba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f718c02c700 tx=0x7f718c0328a0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 shutdown_connections 2024-04-13T05:16:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71780a3bb0 0x7f71780a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7194109ff0 0x7f71941a5ba0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194109630 0x7f719419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 --2- 172.21.15.57:0/663705540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194108440 0x7f719419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 >> 172.21.15.57:0/663705540 conn(0x7f7194077400 msgr2=0x7f719410c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.637+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 shutdown_connections 2024-04-13T05:16:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.638+0000 7f7193e00640 1 -- 172.21.15.57:0/663705540 wait complete. 2024-04-13T05:16:57.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:16:57.661 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:16:57.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 -- 172.21.15.57:0/3180050059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 msgr2=0x7f4890078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 --2- 172.21.15.57:0/3180050059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f4890078e00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4888009900 tx=0x7f488802bc30 comp rx=0 tx=0).stop 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 -- 172.21.15.57:0/3180050059 shutdown_connections 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 --2- 172.21.15.57:0/3180050059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f4890078e00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 --2- 172.21.15.57:0/3180050059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890077bc0 0x7f4890078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 --2- 172.21.15.57:0/3180050059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4890076980 0x7f4890076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.883+0000 7f4895a00640 1 -- 172.21.15.57:0/3180050059 >> 172.21.15.57:0/3180050059 conn(0x7f48900ff7b0 msgr2=0x7f4890101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.884+0000 7f4895a00640 1 -- 172.21.15.57:0/3180050059 shutdown_connections 2024-04-13T05:16:57.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.884+0000 7f4895a00640 1 -- 172.21.15.57:0/3180050059 wait complete. 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.884+0000 7f4895a00640 1 Processor -- start 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 -- start start 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890076980 0x7f489019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4890077bc0 0x7f489019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f48901a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f48901141a0 con 0x7f4890078560 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4890114020 con 0x7f4890077bc0 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f4895a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4890114320 con 0x7f4890076980 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f488f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f48901a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f488f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f48901a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51830/0 (socket says 172.21.15.57:51830) 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.885+0000 7f488f400640 1 -- 172.21.15.57:0/772590656 learned_addr learned my addr 172.21.15.57:0/772590656 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488e000640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4890077bc0 0x7f489019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488ea00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890076980 0x7f489019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488f400640 1 -- 172.21.15.57:0/772590656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890076980 msgr2=0x7f489019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488f400640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890076980 0x7f489019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488f400640 1 -- 172.21.15.57:0/772590656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4890077bc0 msgr2=0x7f489019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488f400640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4890077bc0 0x7f489019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488f400640 1 -- 172.21.15.57:0/772590656 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48901a63e0 con 0x7f4890078560 2024-04-13T05:16:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.886+0000 7f488f400640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f48901a5d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4888036040 tx=0x7f4888004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.887+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888043020 con 0x7f4890078560 2024-04-13T05:16:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.887+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4888004490 con 0x7f4890078560 2024-04-13T05:16:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.887+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888003d10 con 0x7f4890078560 2024-04-13T05:16:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.887+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48901a6670 con 0x7f4890078560 2024-04-13T05:16:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.887+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a6b70 con 0x7f4890078560 2024-04-13T05:16:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.888+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4890110b90 con 0x7f4890078560 2024-04-13T05:16:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.890+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4888046020 con 0x7f4890078560 2024-04-13T05:16:57.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.892+0000 7f4887400640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48740a3c30 0x7f48740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:57.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.893+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f48880f2b40 con 0x7f4890078560 2024-04-13T05:16:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.893+0000 7f488ea00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48740a3c30 0x7f48740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.894+0000 7f488ea00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48740a3c30 0x7f48740a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4880005fd0 tx=0x7f4880009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:57.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:57.897+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f488804d020 con 0x7f4890078560 2024-04-13T05:16:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:57 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/663705540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:16:58.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:57 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/663705540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:16:58.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.103+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4890062bc0 con 0x7f4890078560 2024-04-13T05:16:58.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.104+0000 7f4887400640 1 -- 172.21.15.57:0/772590656 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f48880bf180 con 0x7f4890078560 2024-04-13T05:16:58.106 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:16:58.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48740a3c30 msgr2=0x7f48740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48740a3c30 0x7f48740a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4880005fd0 tx=0x7f4880009870 comp rx=0 tx=0).stop 2024-04-13T05:16:58.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 msgr2=0x7f48901a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f48901a5d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4888036040 tx=0x7f4888004270 comp rx=0 tx=0).stop 2024-04-13T05:16:58.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 shutdown_connections 2024-04-13T05:16:58.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48740a3c30 0x7f48740a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4890077bc0 0x7f489019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4890078560 0x7f48901a5d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 --2- 172.21.15.57:0/772590656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890076980 0x7f489019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 >> 172.21.15.57:0/772590656 conn(0x7f48900ff7b0 msgr2=0x7f48901012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:58.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.107+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 shutdown_connections 2024-04-13T05:16:58.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.108+0000 7f4895a00640 1 -- 172.21.15.57:0/772590656 wait complete. 2024-04-13T05:16:58.132 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:16:58.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:57 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/663705540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:16:58.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.353+0000 7f95aa600640 1 -- 172.21.15.57:0/542617353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 msgr2=0x7f95a4076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.353+0000 7f95aa600640 1 --2- 172.21.15.57:0/542617353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a4076da0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9594009900 tx=0x7f959402bc40 comp rx=0 tx=0).stop 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 -- 172.21.15.57:0/542617353 shutdown_connections 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 --2- 172.21.15.57:0/542617353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4078560 0x7f95a4078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 --2- 172.21.15.57:0/542617353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95a4077bc0 0x7f95a4078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 --2- 172.21.15.57:0/542617353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a4076da0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 -- 172.21.15.57:0/542617353 >> 172.21.15.57:0/542617353 conn(0x7f95a40ff670 msgr2=0x7f95a4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 -- 172.21.15.57:0/542617353 shutdown_connections 2024-04-13T05:16:58.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.354+0000 7f95aa600640 1 -- 172.21.15.57:0/542617353 wait complete. 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 Processor -- start 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 -- start start 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95a4077bc0 0x7f95a419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4078560 0x7f95a41a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114140 con 0x7f95a4076980 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4113fc0 con 0x7f95a4078560 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.355+0000 7f95aa600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41142c0 con 0x7f95a4077bc0 2024-04-13T05:16:58.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51854/0 (socket says 172.21.15.57:51854) 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 -- 172.21.15.57:0/2761938554 learned_addr learned my addr 172.21.15.57:0/2761938554 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 -- 172.21.15.57:0/2761938554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95a4077bc0 msgr2=0x7f95a419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a3e00640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95a4077bc0 0x7f95a419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9c00640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4078560 0x7f95a41a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95a4077bc0 0x7f95a419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 -- 172.21.15.57:0/2761938554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4078560 msgr2=0x7f95a41a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4078560 0x7f95a41a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 -- 172.21.15.57:0/2761938554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a6490 con 0x7f95a4076980 2024-04-13T05:16:58.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.356+0000 7f95a9200640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a419e870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f95940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:58.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.357+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594043020 con 0x7f95a4076980 2024-04-13T05:16:58.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.357+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a6720 con 0x7f95a4076980 2024-04-13T05:16:58.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.357+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95940054c0 con 0x7f95a4076980 2024-04-13T05:16:58.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.357+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594003d80 con 0x7f95a4076980 2024-04-13T05:16:58.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.357+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a6d40 con 0x7f95a4076980 2024-04-13T05:16:58.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.358+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4110d00 con 0x7f95a4076980 2024-04-13T05:16:58.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.360+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9594046020 con 0x7f95a4076980 2024-04-13T05:16:58.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.362+0000 7f95a1600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95880a3c30 0x7f95880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.362+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f95940f2c10 con 0x7f95a4076980 2024-04-13T05:16:58.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.362+0000 7f95a3e00640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95880a3c30 0x7f95880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.363+0000 7f95a3e00640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95880a3c30 0x7f95880a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f95a419fd80 tx=0x7f95900072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:58.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.367+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f959404d020 con 0x7f95a4076980 2024-04-13T05:16:58.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.575+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95a4062bc0 con 0x7f95a4076980 2024-04-13T05:16:58.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.576+0000 7f95a1600640 1 -- 172.21.15.57:0/2761938554 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f95940bf1f0 con 0x7f95a4076980 2024-04-13T05:16:58.578 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:16:58.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95880a3c30 msgr2=0x7f95880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95880a3c30 0x7f95880a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f95a419fd80 tx=0x7f95900072f0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 msgr2=0x7f95a419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a419e870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f95940042f0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 shutdown_connections 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95880a3c30 0x7f95880a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4078560 0x7f95a41a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95a4077bc0 0x7f95a419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 --2- 172.21.15.57:0/2761938554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a4076980 0x7f95a419e870 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.578+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 >> 172.21.15.57:0/2761938554 conn(0x7f95a40ff670 msgr2=0x7f95a4100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.579+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 shutdown_connections 2024-04-13T05:16:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.579+0000 7f95aa600640 1 -- 172.21.15.57:0/2761938554 wait complete. 2024-04-13T05:16:58.601 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 -- 172.21.15.57:0/474544328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 msgr2=0x7f9ab410a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/474544328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab410a3b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009ca0 tx=0x7f9aac02c100 comp rx=0 tx=0).stop 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 -- 172.21.15.57:0/474544328 shutdown_connections 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/474544328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab41030d0 0x7f9ab4103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/474544328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab410a3b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/474544328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab4101f00 0x7f9ab4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 -- 172.21.15.57:0/474544328 >> 172.21.15.57:0/474544328 conn(0x7f9ab40fd4c0 msgr2=0x7f9ab40ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 -- 172.21.15.57:0/474544328 shutdown_connections 2024-04-13T05:16:58.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.827+0000 7f9ab8a00640 1 -- 172.21.15.57:0/474544328 wait complete. 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.828+0000 7f9ab8a00640 1 Processor -- start 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.828+0000 7f9ab8a00640 1 -- start start 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.828+0000 7f9ab8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab4101f00 0x7f9ab419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.828+0000 7f9ab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab41030d0 0x7f9ab419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.828+0000 7f9ab8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab8a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab410da00 con 0x7f9ab4103a90 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab410d880 con 0x7f9ab41030d0 2024-04-13T05:16:58.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab410db80 con 0x7f9ab4101f00 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab41a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51878/0 (socket says 172.21.15.57:51878) 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 -- 172.21.15.57:0/688555406 learned_addr learned my addr 172.21.15.57:0/688555406 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3400640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab4101f00 0x7f9ab419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab2a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab41030d0 0x7f9ab419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 -- 172.21.15.57:0/688555406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab4101f00 msgr2=0x7f9ab419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab4101f00 0x7f9ab419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 -- 172.21.15.57:0/688555406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab41030d0 msgr2=0x7f9ab419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab41030d0 0x7f9ab419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:58.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.829+0000 7f9ab3e00640 1 -- 172.21.15.57:0/688555406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab41a1e40 con 0x7f9ab4103a90 2024-04-13T05:16:58.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.830+0000 7f9ab3e00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab41a17d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9aac032520 tx=0x7f9aac034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:58.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.831+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac039040 con 0x7f9ab4103a90 2024-04-13T05:16:58.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.831+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aac044070 con 0x7f9ab4103a90 2024-04-13T05:16:58.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.831+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0043c0 con 0x7f9ab4103a90 2024-04-13T05:16:58.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.831+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab41a20d0 con 0x7f9ab4103a90 2024-04-13T05:16:58.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.831+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab41a2610 con 0x7f9ab4103a90 2024-04-13T05:16:58.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.832+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab41a2930 con 0x7f9ab4103a90 2024-04-13T05:16:58.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.834+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9aac004560 con 0x7f9ab4103a90 2024-04-13T05:16:58.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.835+0000 7f9aabe00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a980a3c30 0x7f9a980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:58.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.835+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9aac0f2cb0 con 0x7f9ab4103a90 2024-04-13T05:16:58.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.836+0000 7f9ab3400640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a980a3c30 0x7f9a980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:58.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.836+0000 7f9ab3400640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a980a3c30 0x7f9a980a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4005fd0 tx=0x7f9aa4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:58.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:58.840+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9aac04e020 con 0x7f9ab4103a90 2024-04-13T05:16:59.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:58 smithi057 ceph-mon[32331]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:59.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/772590656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:16:59.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2761938554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:16:59.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.052+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ab41a2b00 con 0x7f9ab4103a90 2024-04-13T05:16:59.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.053+0000 7f9aabe00640 1 -- 172.21.15.57:0/688555406 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f9aac0333a0 con 0x7f9ab4103a90 2024-04-13T05:16:59.055 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a980a3c30 msgr2=0x7f9a980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a980a3c30 0x7f9a980a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4005fd0 tx=0x7f9aa4005850 comp rx=0 tx=0).stop 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 msgr2=0x7f9ab41a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab41a17d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9aac032520 tx=0x7f9aac034880 comp rx=0 tx=0).stop 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 shutdown_connections 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a980a3c30 0x7f9a980a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab41030d0 0x7f9ab419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ab4103a90 0x7f9ab41a17d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 --2- 172.21.15.57:0/688555406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab4101f00 0x7f9ab419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 >> 172.21.15.57:0/688555406 conn(0x7f9ab40fd4c0 msgr2=0x7f9ab40741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:59.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 shutdown_connections 2024-04-13T05:16:59.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.056+0000 7f9ab8a00640 1 -- 172.21.15.57:0/688555406 wait complete. 2024-04-13T05:16:59.079 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:16:59.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:58 smithi189 ceph-mon[35531]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:59.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/772590656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:16:59.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2761938554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:16:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:58 smithi083 ceph-mon[35133]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:16:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/772590656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:16:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2761938554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:16:59.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 -- 172.21.15.57:0/2690209087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58101a40 msgr2=0x7f8a58101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 --2- 172.21.15.57:0/2690209087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58101a40 0x7f8a58101ec0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009ca0 tx=0x7f8a5002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 -- 172.21.15.57:0/2690209087 shutdown_connections 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 --2- 172.21.15.57:0/2690209087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a581010a0 0x7f8a58101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 --2- 172.21.15.57:0/2690209087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58101a40 0x7f8a58101ec0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 --2- 172.21.15.57:0/2690209087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a58100740 0x7f8a58100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.309+0000 7f8a57200640 1 -- 172.21.15.57:0/2690209087 >> 172.21.15.57:0/2690209087 conn(0x7f8a580fc360 msgr2=0x7f8a580fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.310+0000 7f8a57200640 1 -- 172.21.15.57:0/2690209087 shutdown_connections 2024-04-13T05:16:59.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.310+0000 7f8a57200640 1 -- 172.21.15.57:0/2690209087 wait complete. 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 Processor -- start 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 -- start start 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 0x7f8a5819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a581010a0 0x7f8a5819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58101a40 0x7f8a581a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5810fc20 con 0x7f8a58100740 2024-04-13T05:16:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5810faa0 con 0x7f8a58101a40 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a57200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5810fda0 con 0x7f8a581010a0 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.311+0000 7f8a55e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 0x7f8a5819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 0x7f8a5819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51888/0 (socket says 172.21.15.57:51888) 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 -- 172.21.15.57:0/1477093400 learned_addr learned my addr 172.21.15.57:0/1477093400 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a56800640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58101a40 0x7f8a581a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55400640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a581010a0 0x7f8a5819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 -- 172.21.15.57:0/1477093400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a581010a0 msgr2=0x7f8a5819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a581010a0 0x7f8a5819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 -- 172.21.15.57:0/1477093400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58101a40 msgr2=0x7f8a581a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58101a40 0x7f8a581a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 -- 172.21.15.57:0/1477093400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581a1f10 con 0x7f8a58100740 2024-04-13T05:16:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.312+0000 7f8a55e00640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 0x7f8a5819a2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e9f0 tx=0x7f8a4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.313+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800cd80 con 0x7f8a58100740 2024-04-13T05:16:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.313+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a480054d0 con 0x7f8a58100740 2024-04-13T05:16:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.313+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48010490 con 0x7f8a58100740 2024-04-13T05:16:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.313+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581a2200 con 0x7f8a58100740 2024-04-13T05:16:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.313+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581a2750 con 0x7f8a58100740 2024-04-13T05:16:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.314+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5810b490 con 0x7f8a58100740 2024-04-13T05:16:59.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.316+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a48005020 con 0x7f8a58100740 2024-04-13T05:16:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.318+0000 7f8a4ea00640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a3c0a3c30 0x7f8a3c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.318+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8a480d0c50 con 0x7f8a58100740 2024-04-13T05:16:59.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.318+0000 7f8a55400640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a3c0a3c30 0x7f8a3c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.319+0000 7f8a55400640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a3c0a3c30 0x7f8a3c0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a5819b800 tx=0x7f8a44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.322+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a4809d230 con 0x7f8a58100740 2024-04-13T05:16:59.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.528+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8a58062bc0 con 0x7f8a58100740 2024-04-13T05:16:59.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.529+0000 7f8a4ea00640 1 -- 172.21.15.57:0/1477093400 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f8a4809cb30 con 0x7f8a58100740 2024-04-13T05:16:59.531 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a3c0a3c30 msgr2=0x7f8a3c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a3c0a3c30 0x7f8a3c0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a5819b800 tx=0x7f8a44005f90 comp rx=0 tx=0).stop 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 msgr2=0x7f8a5819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 0x7f8a5819a2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e9f0 tx=0x7f8a4800edf0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 shutdown_connections 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a3c0a3c30 0x7f8a3c0a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a58101a40 0x7f8a581a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a581010a0 0x7f8a5819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 --2- 172.21.15.57:0/1477093400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a58100740 0x7f8a5819a2f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 >> 172.21.15.57:0/1477093400 conn(0x7f8a580fc360 msgr2=0x7f8a580fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:59.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 shutdown_connections 2024-04-13T05:16:59.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.532+0000 7f8a57200640 1 -- 172.21.15.57:0/1477093400 wait complete. 2024-04-13T05:16:59.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:16:59.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 -- 172.21.15.57:0/2910049286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 msgr2=0x7f4c5c110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2910049286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c110a60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009ca0 tx=0x7f4c5402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 -- 172.21.15.57:0/2910049286 shutdown_connections 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2910049286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c109810 0x7f4c5c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2910049286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c110a60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2910049286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c5c108620 0x7f4c5c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 -- 172.21.15.57:0/2910049286 >> 172.21.15.57:0/2910049286 conn(0x7f4c5c0775e0 msgr2=0x7f4c5c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.768+0000 7f4c60c00640 1 -- 172.21.15.57:0/2910049286 shutdown_connections 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.769+0000 7f4c60c00640 1 -- 172.21.15.57:0/2910049286 wait complete. 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 Processor -- start 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 -- start start 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c108620 0x7f4c5c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c5c109810 0x7f4c5c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c114150 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c113fd0 con 0x7f4c5c108620 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.770+0000 7f4c60c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1142d0 con 0x7f4c5c109810 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c1a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51908/0 (socket says 172.21.15.57:51908) 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 -- 172.21.15.57:0/2184799534 learned_addr learned my addr 172.21.15.57:0/2184799534 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c51400640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c5c109810 0x7f4c5c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c59e00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c108620 0x7f4c5c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 -- 172.21.15.57:0/2184799534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c5c109810 msgr2=0x7f4c5c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c5c109810 0x7f4c5c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 -- 172.21.15.57:0/2184799534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c108620 msgr2=0x7f4c5c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c108620 0x7f4c5c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c5a800640 1 -- 172.21.15.57:0/2184799534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c1a6440 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.771+0000 7f4c59e00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c108620 0x7f4c5c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:16:59.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.772+0000 7f4c5a800640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c1a5dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4c54032380 tx=0x7f4c54032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:59.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.772+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54041020 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.772+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c54003e20 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.772+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54039040 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.772+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c1a66d0 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.772+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1a6c10 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.773+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c10fa10 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.776+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c54044020 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.777+0000 7f4c53400640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c400a3c30 0x7f4c400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:16:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.777+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4c540f2ee0 con 0x7f4c5c10a1d0 2024-04-13T05:16:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.777+0000 7f4c59e00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c400a3c30 0x7f4c400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:16:59.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.778+0000 7f4c59e00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c400a3c30 0x7f4c400a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0058c0 tx=0x7f4c4c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:16:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:16:59.780+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c5404b020 con 0x7f4c5c10a1d0 2024-04-13T05:17:00.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.029+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c5c02d7a0 con 0x7f4c5c10a1d0 2024-04-13T05:17:00.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.030+0000 7f4c53400640 1 -- 172.21.15.57:0/2184799534 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f4c5c02d7a0 con 0x7f4c5c10a1d0 2024-04-13T05:17:00.032 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:00.032 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:15:22.311895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:17:00.033 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c400a3c30 msgr2=0x7f4c400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c400a3c30 0x7f4c400a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0058c0 tx=0x7f4c4c005850 comp rx=0 tx=0).stop 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 msgr2=0x7f4c5c1a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c1a5dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4c54032380 tx=0x7f4c54032820 comp rx=0 tx=0).stop 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 shutdown_connections 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4c400a3c30 0x7f4c400a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c5c10a1d0 0x7f4c5c1a5dd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c5c109810 0x7f4c5c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 --2- 172.21.15.57:0/2184799534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c5c108620 0x7f4c5c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 >> 172.21.15.57:0/2184799534 conn(0x7f4c5c0775e0 msgr2=0x7f4c5c10cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:00.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 shutdown_connections 2024-04-13T05:17:00.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.033+0000 7f4c60c00640 1 -- 172.21.15.57:0/2184799534 wait complete. 2024-04-13T05:17:00.053 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:00.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/688555406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:00.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:16:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1477093400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:00.128 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/688555406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:00.128 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:16:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1477093400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:00.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/688555406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:00.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:16:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1477093400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 -- 172.21.15.57:0/2796771780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f0108620 msgr2=0x7f30f0108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 --2- 172.21.15.57:0/2796771780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f0108620 0x7f30f0108a40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009900 tx=0x7f30e002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 -- 172.21.15.57:0/2796771780 shutdown_connections 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 --2- 172.21.15.57:0/2796771780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f010a1d0 0x7f30f0110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 --2- 172.21.15.57:0/2796771780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30f0109810 0x7f30f0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 --2- 172.21.15.57:0/2796771780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f0108620 0x7f30f0108a40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 -- 172.21.15.57:0/2796771780 >> 172.21.15.57:0/2796771780 conn(0x7f30f00775e0 msgr2=0x7f30f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 -- 172.21.15.57:0/2796771780 shutdown_connections 2024-04-13T05:17:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.261+0000 7f30f5200640 1 -- 172.21.15.57:0/2796771780 wait complete. 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.262+0000 7f30f5200640 1 Processor -- start 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.262+0000 7f30f5200640 1 -- start start 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f0108620 0x7f30f019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30f5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30f0109810 0x7f30f019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30f5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 0x7f30f01a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30f5200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f30f01141a0 con 0x7f30f010a1d0 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30f5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0114020 con 0x7f30f0108620 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30f5200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0114320 con 0x7f30f0109810 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 0x7f30f01a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 0x7f30f01a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51916/0 (socket says 172.21.15.57:51916) 2024-04-13T05:17:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 -- 172.21.15.57:0/1907038512 learned_addr learned my addr 172.21.15.57:0/1907038512 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30e7e00640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30f0109810 0x7f30f019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ede00640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f0108620 0x7f30f019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 -- 172.21.15.57:0/1907038512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30f0109810 msgr2=0x7f30f019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30f0109810 0x7f30f019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 -- 172.21.15.57:0/1907038512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f0108620 msgr2=0x7f30f019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f0108620 0x7f30f019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.263+0000 7f30ee800640 1 -- 172.21.15.57:0/1907038512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f01a6510 con 0x7f30f010a1d0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.264+0000 7f30ee800640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 0x7f30f01a5ea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30e800cad0 tx=0x7f30e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.264+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800ae20 con 0x7f30f010a1d0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.264+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e8005020 con 0x7f30f010a1d0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.264+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800f6b0 con 0x7f30f010a1d0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.264+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f01a6800 con 0x7f30f010a1d0 2024-04-13T05:17:00.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.264+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01a6d00 con 0x7f30f010a1d0 2024-04-13T05:17:00.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.265+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f010f9e0 con 0x7f30f010a1d0 2024-04-13T05:17:00.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.267+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f30e80051d0 con 0x7f30f010a1d0 2024-04-13T05:17:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.268+0000 7f30e6a00640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f30d40a3bb0 0x7f30d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.269+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f30e80d0930 con 0x7f30f010a1d0 2024-04-13T05:17:00.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.269+0000 7f30ede00640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f30d40a3bb0 0x7f30d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.269+0000 7f30ede00640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f30d40a3bb0 0x7f30d40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30e002c240 tx=0x7f30e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:00.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.273+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30e8014030 con 0x7f30f010a1d0 2024-04-13T05:17:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.479+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30f0062bc0 con 0x7f30f010a1d0 2024-04-13T05:17:00.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.483+0000 7f30e6a00640 1 -- 172.21.15.57:0/1907038512 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f30e809cf10 con 0x7f30f010a1d0 2024-04-13T05:17:00.486 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.486+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f30d40a3bb0 msgr2=0x7f30d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.486+0000 7f30f5200640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f30d40a3bb0 0x7f30d40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30e002c240 tx=0x7f30e0000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.486+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 msgr2=0x7f30f01a5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.486+0000 7f30f5200640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 0x7f30f01a5ea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30e800cad0 tx=0x7f30e800cf90 comp rx=0 tx=0).stop 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 shutdown_connections 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f30d40a3bb0 0x7f30d40a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30f010a1d0 0x7f30f01a5ea0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30f0109810 0x7f30f019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 --2- 172.21.15.57:0/1907038512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f0108620 0x7f30f019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 >> 172.21.15.57:0/1907038512 conn(0x7f30f00775e0 msgr2=0x7f30f010cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 shutdown_connections 2024-04-13T05:17:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.487+0000 7f30f5200640 1 -- 172.21.15.57:0/1907038512 wait complete. 2024-04-13T05:17:00.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:17:00.510 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.731+0000 7f9301a00640 1 -- 172.21.15.57:0/2785315677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc105370 msgr2=0x7f92fc107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.731+0000 7f9301a00640 1 --2- 172.21.15.57:0/2785315677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc105370 0x7f92fc107760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009900 tx=0x7f92e802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 -- 172.21.15.57:0/2785315677 shutdown_connections 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 --2- 172.21.15.57:0/2785315677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc107ca0 0x7f92fc10a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 --2- 172.21.15.57:0/2785315677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc105370 0x7f92fc107760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 --2- 172.21.15.57:0/2785315677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92fc102a40 0x7f92fc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 -- 172.21.15.57:0/2785315677 >> 172.21.15.57:0/2785315677 conn(0x7f92fc0fc390 msgr2=0x7f92fc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 -- 172.21.15.57:0/2785315677 shutdown_connections 2024-04-13T05:17:00.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.732+0000 7f9301a00640 1 -- 172.21.15.57:0/2785315677 wait complete. 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 Processor -- start 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 -- start start 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 0x7f92fc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc105370 0x7f92fc19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92fc107ca0 0x7f92fc1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10da30 con 0x7f92fc102a40 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10d8b0 con 0x7f92fc105370 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.733+0000 7f9301a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10dbb0 con 0x7f92fc107ca0 2024-04-13T05:17:00.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 0x7f92fc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 0x7f92fc19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51926/0 (socket says 172.21.15.57:51926) 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 -- 172.21.15.57:0/2052967724 learned_addr learned my addr 172.21.15.57:0/2052967724 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fb400640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc105370 0x7f92fc19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f9301000640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92fc107ca0 0x7f92fc1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 -- 172.21.15.57:0/2052967724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92fc107ca0 msgr2=0x7f92fc1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92fc107ca0 0x7f92fc1a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 -- 172.21.15.57:0/2052967724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc105370 msgr2=0x7f92fc19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc105370 0x7f92fc19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fbe00640 1 -- 172.21.15.57:0/2052967724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc1a1fa0 con 0x7f92fc102a40 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.734+0000 7f92fb400640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc105370 0x7f92fc19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.735+0000 7f92fbe00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 0x7f92fc19a380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d950 tx=0x7f92ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:00.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.735+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec017070 con 0x7f92fc102a40 2024-04-13T05:17:00.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.735+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec009c30 con 0x7f92fc102a40 2024-04-13T05:17:00.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.735+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc1a2290 con 0x7f92fc102a40 2024-04-13T05:17:00.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.735+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00f620 con 0x7f92fc102a40 2024-04-13T05:17:00.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.735+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a27e0 con 0x7f92fc102a40 2024-04-13T05:17:00.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.736+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc109790 con 0x7f92fc102a40 2024-04-13T05:17:00.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.738+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92ec0109a0 con 0x7f92fc102a40 2024-04-13T05:17:00.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.740+0000 7f92f8c00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92e00a3bb0 0x7f92e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:00.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.740+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f92ec0d0480 con 0x7f92fc102a40 2024-04-13T05:17:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.740+0000 7f92fb400640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92e00a3bb0 0x7f92e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.741+0000 7f92fb400640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92e00a3bb0 0x7f92e00a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92e8036040 tx=0x7f92e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:00.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.744+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92ec009de0 con 0x7f92fc102a40 2024-04-13T05:17:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.945+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92fc02d5e0 con 0x7f92fc102a40 2024-04-13T05:17:00.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.946+0000 7f92f8c00640 1 -- 172.21.15.57:0/2052967724 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f92fc02d5e0 con 0x7f92fc102a40 2024-04-13T05:17:00.948 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:17:00.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92e00a3bb0 msgr2=0x7f92e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92e00a3bb0 0x7f92e00a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92e8036040 tx=0x7f92e8000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 msgr2=0x7f92fc19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 0x7f92fc19a380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d950 tx=0x7f92ec00de10 comp rx=0 tx=0).stop 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 shutdown_connections 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92e00a3bb0 0x7f92e00a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92fc105370 0x7f92fc19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92fc107ca0 0x7f92fc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 --2- 172.21.15.57:0/2052967724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92fc102a40 0x7f92fc19a380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 >> 172.21.15.57:0/2052967724 conn(0x7f92fc0fc390 msgr2=0x7f92fc102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 shutdown_connections 2024-04-13T05:17:00.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:00.948+0000 7f9301a00640 1 -- 172.21.15.57:0/2052967724 wait complete. 2024-04-13T05:17:00.968 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 -- 172.21.15.57:0/4148295895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 msgr2=0x7fa0b0109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/4148295895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b0109b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009900 tx=0x7fa09c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 -- 172.21.15.57:0/4148295895 shutdown_connections 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/4148295895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0b010a0b0 0x7fa0b0110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/4148295895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b0109b70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/4148295895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b0108500 0x7fa0b0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 -- 172.21.15.57:0/4148295895 >> 172.21.15.57:0/4148295895 conn(0x7fa0b0077490 msgr2=0x7fa0b00798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 -- 172.21.15.57:0/4148295895 shutdown_connections 2024-04-13T05:17:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.192+0000 7fa0b4c00640 1 -- 172.21.15.57:0/4148295895 wait complete. 2024-04-13T05:17:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.193+0000 7fa0b4c00640 1 Processor -- start 2024-04-13T05:17:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.193+0000 7fa0b4c00640 1 -- start start 2024-04-13T05:17:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0b4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0b0108500 0x7fa0b019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0b4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0b4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b010a0b0 0x7fa0b01a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0b4c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0114040 con 0x7fa0b01096f0 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0b4c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0113ec0 con 0x7fa0b0108500 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0b4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01141c0 con 0x7fa0b010a0b0 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51948/0 (socket says 172.21.15.57:51948) 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0aea00640 1 -- 172.21.15.57:0/2126331330 learned_addr learned my addr 172.21.15.57:0/2126331330 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.194+0000 7fa0af400640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0b0108500 0x7fa0b019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0afe00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b010a0b0 0x7fa0b01a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0aea00640 1 -- 172.21.15.57:0/2126331330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b010a0b0 msgr2=0x7fa0b01a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0aea00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b010a0b0 0x7fa0b01a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0aea00640 1 -- 172.21.15.57:0/2126331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0b0108500 msgr2=0x7fa0b019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0aea00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0b0108500 0x7fa0b019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0aea00640 1 -- 172.21.15.57:0/2126331330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b01a6290 con 0x7fa0b01096f0 2024-04-13T05:17:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0afe00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b010a0b0 0x7fa0b01a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0aea00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b019ebb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa09c036040 tx=0x7fa09c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c043020 con 0x7fa0b01096f0 2024-04-13T05:17:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.195+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09c0054c0 con 0x7fa0b01096f0 2024-04-13T05:17:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.196+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c003d70 con 0x7fa0b01096f0 2024-04-13T05:17:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.196+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b01a6520 con 0x7fa0b01096f0 2024-04-13T05:17:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.196+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b01a6a70 con 0x7fa0b01096f0 2024-04-13T05:17:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.197+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b010f820 con 0x7fa0b01096f0 2024-04-13T05:17:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.199+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa09c046020 con 0x7fa0b01096f0 2024-04-13T05:17:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.200+0000 7fa0a7e00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0940a3bb0 0x7fa0940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.201+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa09c0f2d90 con 0x7fa0b01096f0 2024-04-13T05:17:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.201+0000 7fa0af400640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0940a3bb0 0x7fa0940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.202+0000 7fa0af400640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0940a3bb0 0x7fa0940a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005db0 tx=0x7fa0a0005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:01.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.205+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa09c04d020 con 0x7fa0b01096f0 2024-04-13T05:17:01.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:00 smithi083 ceph-mon[35133]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:01.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2184799534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:01.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1907038512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:01.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:00 smithi057 ceph-mon[32331]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:01.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2184799534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:01.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1907038512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:01.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:00 smithi189 ceph-mon[35531]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:01.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2184799534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:01.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1907038512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:01.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.412+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0b0062bc0 con 0x7fa0b01096f0 2024-04-13T05:17:01.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.413+0000 7fa0a7e00640 1 -- 172.21.15.57:0/2126331330 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fa09c0bf370 con 0x7fa0b01096f0 2024-04-13T05:17:01.415 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.415+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0940a3bb0 msgr2=0x7fa0940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.415+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0940a3bb0 0x7fa0940a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005db0 tx=0x7fa0a0005d40 comp rx=0 tx=0).stop 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.415+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 msgr2=0x7fa0b019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.415+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b019ebb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa09c036040 tx=0x7fa09c0042f0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 shutdown_connections 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa0940a3bb0 0x7fa0940a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0b010a0b0 0x7fa0b01a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0b01096f0 0x7fa0b019ebb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 --2- 172.21.15.57:0/2126331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0b0108500 0x7fa0b019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 >> 172.21.15.57:0/2126331330 conn(0x7fa0b0077490 msgr2=0x7fa0b010c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 shutdown_connections 2024-04-13T05:17:01.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.416+0000 7fa0b4c00640 1 -- 172.21.15.57:0/2126331330 wait complete. 2024-04-13T05:17:01.438 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:17:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 -- 172.21.15.57:0/1287103343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 msgr2=0x7ff85810c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 --2- 172.21.15.57:0/1287103343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff85810c540 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff850009ca0 tx=0x7ff85002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 -- 172.21.15.57:0/1287103343 shutdown_connections 2024-04-13T05:17:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 --2- 172.21.15.57:0/1287103343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff85810c540 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 --2- 172.21.15.57:0/1287103343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8581052f0 0x7ff858105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 --2- 172.21.15.57:0/1287103343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff858104100 0x7ff858104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 -- 172.21.15.57:0/1287103343 >> 172.21.15.57:0/1287103343 conn(0x7ff8580ff670 msgr2=0x7ff858101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 -- 172.21.15.57:0/1287103343 shutdown_connections 2024-04-13T05:17:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.669+0000 7ff85d800640 1 -- 172.21.15.57:0/1287103343 wait complete. 2024-04-13T05:17:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 Processor -- start 2024-04-13T05:17:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 -- start start 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858104100 0x7ff85819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8581052f0 0x7ff85819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff8581a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff85810fc10 con 0x7ff858105cb0 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff85810fa90 con 0x7ff8581052f0 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff85810fd90 con 0x7ff858104100 2024-04-13T05:17:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.671+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff8581a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff8581a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51968/0 (socket says 172.21.15.57:51968) 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 -- 172.21.15.57:0/2975062569 learned_addr learned my addr 172.21.15.57:0/2975062569 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff857e00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858104100 0x7ff85819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff857400640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8581052f0 0x7ff85819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 -- 172.21.15.57:0/2975062569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858104100 msgr2=0x7ff85819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858104100 0x7ff85819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 -- 172.21.15.57:0/2975062569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8581052f0 msgr2=0x7ff85819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8581052f0 0x7ff85819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 -- 172.21.15.57:0/2975062569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8581a1f30 con 0x7ff858105cb0 2024-04-13T05:17:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.672+0000 7ff85ce00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff8581a18c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff850037040 tx=0x7ff850032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.673+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850041020 con 0x7ff858105cb0 2024-04-13T05:17:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.673+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff850032940 con 0x7ff858105cb0 2024-04-13T05:17:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.673+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850039040 con 0x7ff858105cb0 2024-04-13T05:17:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.673+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8581a21c0 con 0x7ff858105cb0 2024-04-13T05:17:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.673+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8581a2660 con 0x7ff858105cb0 2024-04-13T05:17:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.674+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85810b540 con 0x7ff858105cb0 2024-04-13T05:17:01.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.676+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff850044020 con 0x7ff858105cb0 2024-04-13T05:17:01.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.678+0000 7ff854c00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff83c0a3c80 0x7ff83c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:01.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.678+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff8500f2a30 con 0x7ff858105cb0 2024-04-13T05:17:01.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.678+0000 7ff857e00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff83c0a3c80 0x7ff83c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:01.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.679+0000 7ff857e00640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff83c0a3c80 0x7ff83c0a6140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff848005fd0 tx=0x7ff848005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:01.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.682+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff85004b020 con 0x7ff858105cb0 2024-04-13T05:17:01.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.890+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff858062bc0 con 0x7ff858105cb0 2024-04-13T05:17:01.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.891+0000 7ff854c00640 1 -- 172.21.15.57:0/2975062569 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ff8500098b0 con 0x7ff858105cb0 2024-04-13T05:17:01.893 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff83c0a3c80 msgr2=0x7ff83c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff83c0a3c80 0x7ff83c0a6140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff848005fd0 tx=0x7ff848005850 comp rx=0 tx=0).stop 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 msgr2=0x7ff8581a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff8581a18c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff850037040 tx=0x7ff850032820 comp rx=0 tx=0).stop 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 shutdown_connections 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff83c0a3c80 0x7ff83c0a6140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8581052f0 0x7ff85819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff858105cb0 0x7ff8581a18c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 --2- 172.21.15.57:0/2975062569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858104100 0x7ff85819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 >> 172.21.15.57:0/2975062569 conn(0x7ff8580ff670 msgr2=0x7ff858108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.894+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 shutdown_connections 2024-04-13T05:17:01.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:01.895+0000 7ff85d800640 1 -- 172.21.15.57:0/2975062569 wait complete. 2024-04-13T05:17:01.916 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:17:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 -- 172.21.15.57:0/3125931154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c109810 msgr2=0x7f5a7c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3125931154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c109810 0x7f5a7c109c90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009900 tx=0x7f5a6802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 -- 172.21.15.57:0/3125931154 shutdown_connections 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3125931154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a7c10a1d0 0x7f5a7c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3125931154 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c109810 0x7f5a7c109c90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3125931154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c108620 0x7f5a7c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 -- 172.21.15.57:0/3125931154 >> 172.21.15.57:0/3125931154 conn(0x7f5a7c0775c0 msgr2=0x7f5a7c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 -- 172.21.15.57:0/3125931154 shutdown_connections 2024-04-13T05:17:02.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.125+0000 7f5a80e00640 1 -- 172.21.15.57:0/3125931154 wait complete. 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.126+0000 7f5a80e00640 1 Processor -- start 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 -- start start 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 0x7f5a7c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c109810 0x7f5a7c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a7c10a1d0 0x7f5a7c1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c1141b0 con 0x7f5a7c108620 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c114030 con 0x7f5a7c109810 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a80e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c114330 con 0x7f5a7c10a1d0 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 0x7f5a7c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 0x7f5a7c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51976/0 (socket says 172.21.15.57:51976) 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.127+0000 7f5a79e00640 1 -- 172.21.15.57:0/3100069169 learned_addr learned my addr 172.21.15.57:0/3100069169 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79400640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c109810 0x7f5a7c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a7a800640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a7c10a1d0 0x7f5a7c1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79e00640 1 -- 172.21.15.57:0/3100069169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a7c10a1d0 msgr2=0x7f5a7c1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a7c10a1d0 0x7f5a7c1a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79e00640 1 -- 172.21.15.57:0/3100069169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c109810 msgr2=0x7f5a7c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c109810 0x7f5a7c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79e00640 1 -- 172.21.15.57:0/3100069169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c1a6420 con 0x7f5a7c108620 2024-04-13T05:17:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.128+0000 7f5a79e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 0x7f5a7c19e800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c970 tx=0x7f5a6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.129+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c007c10 con 0x7f5a7c108620 2024-04-13T05:17:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.129+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a6c007db0 con 0x7f5a7c108620 2024-04-13T05:17:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.129+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00f870 con 0x7f5a7c108620 2024-04-13T05:17:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.129+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c1a6710 con 0x7f5a7c108620 2024-04-13T05:17:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.129+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c1a6c10 con 0x7f5a7c108620 2024-04-13T05:17:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.130+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7c10f940 con 0x7f5a7c108620 2024-04-13T05:17:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.132+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a6c00fa10 con 0x7f5a7c108620 2024-04-13T05:17:02.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.133+0000 7f5a72a00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a600a3bb0 0x7f5a600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.134+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5a6c0d0c20 con 0x7f5a7c108620 2024-04-13T05:17:02.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.134+0000 7f5a79400640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a600a3bb0 0x7f5a600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.135+0000 7f5a79400640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a600a3bb0 0x7f5a600a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c19fd10 tx=0x7f5a68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:02.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.138+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a6c09d290 con 0x7f5a7c108620 2024-04-13T05:17:02.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2052967724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:02.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2126331330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:02.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2052967724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:02.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2126331330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:02.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2052967724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:02.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2126331330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:02.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.352+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5a7c062c60 con 0x7f5a7c108620 2024-04-13T05:17:02.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.352+0000 7f5a72a00640 1 -- 172.21.15.57:0/3100069169 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f5a6c09cb90 con 0x7f5a7c108620 2024-04-13T05:17:02.354 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:02.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a600a3bb0 msgr2=0x7f5a600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a600a3bb0 0x7f5a600a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c19fd10 tx=0x7f5a68000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 msgr2=0x7f5a7c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 0x7f5a7c19e800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c970 tx=0x7f5a6c00ce30 comp rx=0 tx=0).stop 2024-04-13T05:17:02.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 shutdown_connections 2024-04-13T05:17:02.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a600a3bb0 0x7f5a600a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c109810 0x7f5a7c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a7c10a1d0 0x7f5a7c1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 --2- 172.21.15.57:0/3100069169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a7c108620 0x7f5a7c19e800 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.355+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 >> 172.21.15.57:0/3100069169 conn(0x7f5a7c0775c0 msgr2=0x7f5a7c10cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:02.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.356+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 shutdown_connections 2024-04-13T05:17:02.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.356+0000 7f5a80e00640 1 -- 172.21.15.57:0/3100069169 wait complete. 2024-04-13T05:17:02.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.593+0000 7f2992c00640 1 -- 172.21.15.57:0/3769014137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c076980 msgr2=0x7f298c076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.593+0000 7f2992c00640 1 --2- 172.21.15.57:0/3769014137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c076980 0x7f298c076da0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f297c009900 tx=0x7f297c02bc40 comp rx=0 tx=0).stop 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 -- 172.21.15.57:0/3769014137 shutdown_connections 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 --2- 172.21.15.57:0/3769014137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c078560 0x7f298c078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 --2- 172.21.15.57:0/3769014137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c077bc0 0x7f298c078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 --2- 172.21.15.57:0/3769014137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c076980 0x7f298c076da0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 -- 172.21.15.57:0/3769014137 >> 172.21.15.57:0/3769014137 conn(0x7f298c0ff7b0 msgr2=0x7f298c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 -- 172.21.15.57:0/3769014137 shutdown_connections 2024-04-13T05:17:02.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.594+0000 7f2992c00640 1 -- 172.21.15.57:0/3769014137 wait complete. 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 Processor -- start 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 -- start start 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c076980 0x7f298c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 0x7f298c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c078560 0x7f298c1a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1141c0 con 0x7f298c077bc0 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f298c114040 con 0x7f298c076980 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.595+0000 7f2992c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f298c114340 con 0x7f298c078560 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 0x7f298c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 0x7f298c19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51992/0 (socket says 172.21.15.57:51992) 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 -- 172.21.15.57:0/3794701524 learned_addr learned my addr 172.21.15.57:0/3794701524 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298be00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c076980 0x7f298c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f2990c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c078560 0x7f298c1a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 -- 172.21.15.57:0/3794701524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c078560 msgr2=0x7f298c1a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c078560 0x7f298c1a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 -- 172.21.15.57:0/3794701524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c076980 msgr2=0x7f298c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c076980 0x7f298c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.596+0000 7f298b400640 1 -- 172.21.15.57:0/3794701524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c1a64c0 con 0x7f298c077bc0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.597+0000 7f298b400640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 0x7f298c19ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f297800d770 tx=0x7f297800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.597+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978017070 con 0x7f298c077bc0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.597+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29780054d0 con 0x7f298c077bc0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.597+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978004cf0 con 0x7f298c077bc0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.597+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c1a67b0 con 0x7f298c077bc0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.597+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c1a6cb0 con 0x7f298c077bc0 2024-04-13T05:17:02.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.598+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298c110e40 con 0x7f298c077bc0 2024-04-13T05:17:02.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.600+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2978004e90 con 0x7f298c077bc0 2024-04-13T05:17:02.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.602+0000 7f2988c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f29700a3c30 0x7f29700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:02.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.602+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f29780d01f0 con 0x7f298c077bc0 2024-04-13T05:17:02.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.602+0000 7f298be00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f29700a3c30 0x7f29700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:02.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.603+0000 7f298be00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f29700a3c30 0x7f29700a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f297c000c00 tx=0x7f297c02c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:02.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.606+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f297809c7d0 con 0x7f298c077bc0 2024-04-13T05:17:02.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.859+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f298c062bc0 con 0x7f298c077bc0 2024-04-13T05:17:02.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.864+0000 7f2988c00640 1 -- 172.21.15.57:0/3794701524 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f297809c0d0 con 0x7f298c077bc0 2024-04-13T05:17:02.866 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:02.867 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:15:22.311895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:17:02.867 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:17:02.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f29700a3c30 msgr2=0x7f29700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f29700a3c30 0x7f29700a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f297c000c00 tx=0x7f297c02c7f0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 msgr2=0x7f298c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 0x7f298c19ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f297800d770 tx=0x7f297800dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 shutdown_connections 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f29700a3c30 0x7f29700a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c078560 0x7f298c1a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c077bc0 0x7f298c19ede0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 --2- 172.21.15.57:0/3794701524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c076980 0x7f298c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.867+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 >> 172.21.15.57:0/3794701524 conn(0x7f298c0ff7b0 msgr2=0x7f298c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.868+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 shutdown_connections 2024-04-13T05:17:02.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:02.868+0000 7f2992c00640 1 -- 172.21.15.57:0/3794701524 wait complete. 2024-04-13T05:17:02.891 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.095+0000 7f58e3200640 1 -- 172.21.15.57:0/1681002471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e41051f0 msgr2=0x7f58e41075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.095+0000 7f58e3200640 1 --2- 172.21.15.57:0/1681002471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e41051f0 0x7f58e41075e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009900 tx=0x7f58cc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 -- 172.21.15.57:0/1681002471 shutdown_connections 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 --2- 172.21.15.57:0/1681002471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4107b20 0x7f58e4109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 --2- 172.21.15.57:0/1681002471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e41051f0 0x7f58e41075e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 --2- 172.21.15.57:0/1681002471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41028c0 0x7f58e4104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 -- 172.21.15.57:0/1681002471 >> 172.21.15.57:0/1681002471 conn(0x7f58e40fc290 msgr2=0x7f58e40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 -- 172.21.15.57:0/1681002471 shutdown_connections 2024-04-13T05:17:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.096+0000 7f58e3200640 1 -- 172.21.15.57:0/1681002471 wait complete. 2024-04-13T05:17:03.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.097+0000 7f58e3200640 1 Processor -- start 2024-04-13T05:17:03.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.097+0000 7f58e3200640 1 -- start start 2024-04-13T05:17:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41028c0 0x7f58e419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41051f0 0x7f58e419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e3200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 0x7f58e41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e3200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410d9b0 con 0x7f58e4107b20 2024-04-13T05:17:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410d830 con 0x7f58e41028c0 2024-04-13T05:17:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410db30 con 0x7f58e41051f0 2024-04-13T05:17:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 0x7f58e41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 0x7f58e41a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52014/0 (socket says 172.21.15.57:52014) 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e2800640 1 -- 172.21.15.57:0/2359996026 learned_addr learned my addr 172.21.15.57:0/2359996026 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58dbe00640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41051f0 0x7f58e419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.098+0000 7f58e1e00640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41028c0 0x7f58e419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e2800640 1 -- 172.21.15.57:0/2359996026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41051f0 msgr2=0x7f58e419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e2800640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41051f0 0x7f58e419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e2800640 1 -- 172.21.15.57:0/2359996026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41028c0 msgr2=0x7f58e419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e2800640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41028c0 0x7f58e419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e2800640 1 -- 172.21.15.57:0/2359996026 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41a1e40 con 0x7f58e4107b20 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e2800640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 0x7f58e41a17d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00eaa0 tx=0x7f58dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00cdb0 con 0x7f58e4107b20 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e41a2130 con 0x7f58e4107b20 2024-04-13T05:17:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58dc005020 con 0x7f58e4107b20 2024-04-13T05:17:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a2650 con 0x7f58e4107b20 2024-04-13T05:17:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.099+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc01f060 con 0x7f58e4107b20 2024-04-13T05:17:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.101+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0005290 con 0x7f58e4107b20 2024-04-13T05:17:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.103+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f58dc004a40 con 0x7f58e4107b20 2024-04-13T05:17:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.104+0000 7f58daa00640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f58c80a3bb0 0x7f58c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.104+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f58dc0d0c90 con 0x7f58e4107b20 2024-04-13T05:17:03.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.104+0000 7f58e1e00640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f58c80a3bb0 0x7f58c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.105+0000 7f58e1e00640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f58c80a3bb0 0x7f58c80a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58d4005e10 tx=0x7f58d4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.108+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58dc016030 con 0x7f58e4107b20 2024-04-13T05:17:03.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:02 smithi083 ceph-mon[35133]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:03.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:02 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2975062569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:03.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:02 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3100069169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:02 smithi057 ceph-mon[32331]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2975062569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3100069169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:03.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.316+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58c0005850 con 0x7f58e4107b20 2024-04-13T05:17:03.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.321+0000 7f58daa00640 1 -- 172.21.15.57:0/2359996026 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f58dc09d2d0 con 0x7f58e4107b20 2024-04-13T05:17:03.323 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:03.323 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:03.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.323+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f58c80a3bb0 msgr2=0x7f58c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.323+0000 7f58e3200640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f58c80a3bb0 0x7f58c80a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58d4005e10 tx=0x7f58d4005da0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.323+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 msgr2=0x7f58e41a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.323+0000 7f58e3200640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 0x7f58e41a17d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00eaa0 tx=0x7f58dc00ef60 comp rx=0 tx=0).stop 2024-04-13T05:17:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 shutdown_connections 2024-04-13T05:17:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f58c80a3bb0 0x7f58c80a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f58e4107b20 0x7f58e41a17d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41051f0 0x7f58e419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 --2- 172.21.15.57:0/2359996026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41028c0 0x7f58e419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 >> 172.21.15.57:0/2359996026 conn(0x7f58e40fc290 msgr2=0x7f58e4102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 shutdown_connections 2024-04-13T05:17:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.324+0000 7f58e3200640 1 -- 172.21.15.57:0/2359996026 wait complete. 2024-04-13T05:17:03.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:02 smithi189 ceph-mon[35531]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:03.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:02 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2975062569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:03.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:02 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3100069169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:03.346 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:17:03.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 -- 172.21.15.57:0/3848226795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 msgr2=0x7f7490104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 --2- 172.21.15.57:0/3848226795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f7490104d20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7480009900 tx=0x7f748002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 -- 172.21.15.57:0/3848226795 shutdown_connections 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 --2- 172.21.15.57:0/3848226795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 0x7f7490107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 --2- 172.21.15.57:0/3848226795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490107b90 0x7f7490109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 --2- 172.21.15.57:0/3848226795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f7490104d20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 -- 172.21.15.57:0/3848226795 >> 172.21.15.57:0/3848226795 conn(0x7f74900fc360 msgr2=0x7f74900fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 -- 172.21.15.57:0/3848226795 shutdown_connections 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.567+0000 7f7494e00640 1 -- 172.21.15.57:0/3848226795 wait complete. 2024-04-13T05:17:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.568+0000 7f7494e00640 1 Processor -- start 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.568+0000 7f7494e00640 1 -- start start 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7494e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f749019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7494e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 0x7f749019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7494e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490107b90 0x7f74901a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7494e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f749010d990 con 0x7f7490102930 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7494e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f749010d810 con 0x7f7490105260 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7494e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f749010db10 con 0x7f7490107b90 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f749019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f749019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52046/0 (socket says 172.21.15.57:52046) 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 -- 172.21.15.57:0/56474159 learned_addr learned my addr 172.21.15.57:0/56474159 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748fe00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490107b90 0x7f74901a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 -- 172.21.15.57:0/56474159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490107b90 msgr2=0x7f74901a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7486a00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 0x7f749019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490107b90 0x7f74901a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 -- 172.21.15.57:0/56474159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 msgr2=0x7f749019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 0x7f749019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f748f400640 1 -- 172.21.15.57:0/56474159 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74901a1f40 con 0x7f7490102930 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.569+0000 7f7486a00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 0x7f749019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.570+0000 7f748f400640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f749019a320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7480006010 tx=0x7f74800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.570+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480043020 con 0x7f7490102930 2024-04-13T05:17:03.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.570+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7480004aa0 con 0x7f7490102930 2024-04-13T05:17:03.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.570+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480003c60 con 0x7f7490102930 2024-04-13T05:17:03.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.570+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74901a21d0 con 0x7f7490102930 2024-04-13T05:17:03.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.570+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a2770 con 0x7f7490102930 2024-04-13T05:17:03.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.571+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74901096e0 con 0x7f7490102930 2024-04-13T05:17:03.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.573+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7480046020 con 0x7f7490102930 2024-04-13T05:17:03.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.574+0000 7f748cc00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74740a3c30 0x7f74740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:03.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.575+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f74800f2bf0 con 0x7f7490102930 2024-04-13T05:17:03.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.575+0000 7f7486a00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74740a3c30 0x7f74740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:03.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.575+0000 7f7486a00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74740a3c30 0x7f74740a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f749019b830 tx=0x7f74780093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:03.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.579+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f748004d020 con 0x7f7490102930 2024-04-13T05:17:03.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.824+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f749002d5e0 con 0x7f7490102930 2024-04-13T05:17:03.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.825+0000 7f748cc00640 1 -- 172.21.15.57:0/56474159 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f74800bf1d0 con 0x7f7490102930 2024-04-13T05:17:03.828 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:03.828 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:15:22.311895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:17:03.828 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.828+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74740a3c30 msgr2=0x7f74740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.828+0000 7f7494e00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74740a3c30 0x7f74740a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f749019b830 tx=0x7f74780093e0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.828+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 msgr2=0x7f749019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.828+0000 7f7494e00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f749019a320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7480006010 tx=0x7f74800042f0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 shutdown_connections 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74740a3c30 0x7f74740a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7490105260 0x7f749019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490107b90 0x7f74901a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 --2- 172.21.15.57:0/56474159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7490102930 0x7f749019a320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:03.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 >> 172.21.15.57:0/56474159 conn(0x7f74900fc360 msgr2=0x7f7490102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:03.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 shutdown_connections 2024-04-13T05:17:03.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:03.829+0000 7f7494e00640 1 -- 172.21.15.57:0/56474159 wait complete. 2024-04-13T05:17:03.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-13T05:14:53.072497+0000', 'modified': '2024-04-13T05:15:22.311895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34499, 'mds_1': 34457, 'mds_2': 34427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6837/563533422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 563533422}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 563533422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6839/1915044018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 1915044018}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 1915044018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34397': {'gid': 34397, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6833/617814721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6832', 'nonce': 617814721}, {'type': 'v1', 'addr': '172.21.15.83:6833', 'nonce': 617814721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.83:6835/3245344075', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3245344075}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3245344075}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34457': {'gid': 34457, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.83:6837/741118790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 741118790}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 741118790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34499': {'gid': 34499, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/3095811120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3095811120}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3095811120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34499, 'qdb_cluster': [34427, 34499, 34457]} 2024-04-13T05:17:03.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-13T05:17:03.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-13T05:17:04.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 -- 172.21.15.57:0/2314409574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c109810 msgr2=0x7f954c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 --2- 172.21.15.57:0/2314409574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c109810 0x7f954c109c90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9538009900 tx=0x7f953802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:04.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 -- 172.21.15.57:0/2314409574 shutdown_connections 2024-04-13T05:17:04.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 --2- 172.21.15.57:0/2314409574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c10a1d0 0x7f954c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 --2- 172.21.15.57:0/2314409574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c109810 0x7f954c109c90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 --2- 172.21.15.57:0/2314409574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f954c108620 0x7f954c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 -- 172.21.15.57:0/2314409574 >> 172.21.15.57:0/2314409574 conn(0x7f954c0775d0 msgr2=0x7f954c0799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 -- 172.21.15.57:0/2314409574 shutdown_connections 2024-04-13T05:17:04.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.062+0000 7f9553e00640 1 -- 172.21.15.57:0/2314409574 wait complete. 2024-04-13T05:17:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.063+0000 7f9553e00640 1 Processor -- start 2024-04-13T05:17:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.063+0000 7f9553e00640 1 -- start start 2024-04-13T05:17:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9553e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f954c108620 0x7f954c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9553e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c109810 0x7f954c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9553e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 0x7f954c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9553e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f954c114190 con 0x7f954c10a1d0 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9553e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f954c114010 con 0x7f954c108620 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9553e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f954c114310 con 0x7f954c109810 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9551e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 0x7f954c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9551e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 0x7f954c1a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52068/0 (socket says 172.21.15.57:52068) 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9551e00640 1 -- 172.21.15.57:0/2882629475 learned_addr learned my addr 172.21.15.57:0/2882629475 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9551400640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f954c108620 0x7f954c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.064+0000 7f9550a00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c109810 0x7f954c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9551e00640 1 -- 172.21.15.57:0/2882629475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c109810 msgr2=0x7f954c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9551e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c109810 0x7f954c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9551e00640 1 -- 172.21.15.57:0/2882629475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f954c108620 msgr2=0x7f954c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9551e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f954c108620 0x7f954c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9551e00640 1 -- 172.21.15.57:0/2882629475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c1a6450 con 0x7f954c10a1d0 2024-04-13T05:17:04.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9550a00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c109810 0x7f954c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9551e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 0x7f954c1a5de0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f954400cce0 tx=0x7f954400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f954a000640 1 -- 172.21.15.57:0/2882629475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954400f450 con 0x7f954c10a1d0 2024-04-13T05:17:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c1a6740 con 0x7f954c10a1d0 2024-04-13T05:17:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f954a000640 1 -- 172.21.15.57:0/2882629475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f954400faa0 con 0x7f954c10a1d0 2024-04-13T05:17:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f954a000640 1 -- 172.21.15.57:0/2882629475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544018b70 con 0x7f954c10a1d0 2024-04-13T05:17:04.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.065+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c1a6c60 con 0x7f954c10a1d0 2024-04-13T05:17:04.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.070+0000 7f9556c62740 1 Processor -- start 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.070+0000 7f954a000640 1 -- 172.21.15.57:0/2882629475 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9544018d10 con 0x7f954c10a1d0 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.070+0000 7f9556c62740 1 -- start start 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9556c62740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68925990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9556c62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9556c62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6892d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9556c62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ba6888b990 con 0x55ba6892b0d0 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9556c62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba68832530 con 0x55ba68925f60 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9556c62740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba68887950 con 0x55ba68925570 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9550a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6892d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9550a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6892d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52072/0 (socket says 172.21.15.57:52072) 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9550a00640 1 -- 172.21.15.57:0/3700923853 learned_addr learned my addr 172.21.15.57:0/3700923853 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9551400640 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.071+0000 7f9550a00640 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68925990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 -- 172.21.15.57:0/3700923853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 msgr2=0x55ba68925990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68925990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 -- 172.21.15.57:0/3700923853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 msgr2=0x55ba6892aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 -- 172.21.15.57:0/3700923853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6859e2d0 con 0x55ba6892b0d0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f954a000640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95340a3c30 0x7f95340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f954a000640 1 -- 172.21.15.57:0/2882629475 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f95440d0bb0 con 0x7f954c10a1d0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95340a3c30 0x7f95340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.072+0000 7f9550a00640 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6892d560 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95380096c0 tx=0x7f9538005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bf478568e54e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.073+0000 7f953e600640 1 -- 172.21.15.57:0/3700923853 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95380043a0 con 0x55ba6892b0d0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.073+0000 7f953e600640 1 -- 172.21.15.57:0/3700923853 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95380055e0 con 0x55ba6892b0d0 2024-04-13T05:17:04.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.073+0000 7f953e600640 1 -- 172.21.15.57:0/3700923853 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538003970 con 0x55ba6892b0d0 2024-04-13T05:17:04.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 -- 172.21.15.57:0/3700923853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 msgr2=0x55ba6892d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6892d560 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95380096c0 tx=0x7f9538005b50 comp rx=0 tx=0).stop 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9550a00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95340a3c30 0x7f95340a60f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ba689267c0 tx=0x7f9538048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 -- 172.21.15.57:0/3700923853 shutdown_connections 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6892d560 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 --2- 172.21.15.57:0/3700923853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68925990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 -- 172.21.15.57:0/3700923853 >> 172.21.15.57:0/3700923853 conn(0x55ba68920ae0 msgr2=0x55ba68920ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 -- 172.21.15.57:0/3700923853 shutdown_connections 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 -- 172.21.15.57:0/3700923853 wait complete. 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 -- start start 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.074+0000 7f9556c62740 1 Processor -- start 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 0x55ba6892faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6892b0d0 0x55ba68937020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ba6892e080 con 0x55ba68925570 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba688458f0 con 0x55ba68925f60 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba6892e200 con 0x55ba6892b0d0 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9556c62740 10 client.0 _ll_register_callbacks cb 0x55ba688452e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9551400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 0x55ba6892faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9551400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 0x55ba6892faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52084/0 (socket says 172.21.15.57:52084) 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.075+0000 7f9551400640 1 -- 172.21.15.57:0/2214633221 learned_addr learned my addr 172.21.15.57:0/2214633221 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9550a00640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9551400640 1 -- 172.21.15.57:0/2214633221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6892b0d0 msgr2=0x55ba68937020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9551400640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6892b0d0 0x55ba68937020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9551400640 1 -- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 msgr2=0x55ba6892ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9551400640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9551400640 1 -- 172.21.15.57:0/2214633221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba686686b0 con 0x55ba68925570 2024-04-13T05:17:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9550a00640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9551400640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 0x55ba6892faa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f954000bad0 tx=0x7f954000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e76ae2e8972d4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9532a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9532a00640 1 -- 172.21.15.57:0/2214633221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540003dc0 con 0x55ba68925570 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9532a00640 1 -- 172.21.15.57:0/2214633221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9540011440 con 0x55ba68925570 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.076+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95380055e0 con 0x55ba68925570 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9532a00640 1 -- 172.21.15.57:0/2214633221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9540011640 con 0x55ba68925570 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ba6893a460 con 0x55ba68925570 2024-04-13T05:17:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9532a00640 1 -- 172.21.15.57:0/2214633221 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95380055e0 con 0x55ba68925570 2024-04-13T05:17:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9532a00640 1 -- 172.21.15.57:0/2214633221 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f954001b920 con 0x55ba68925570 2024-04-13T05:17:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 10 client.15222 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-13T05:17:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 20 client.15222 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba6893cc10 0x55ba6893e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 4 client.15222 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:17:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.077+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba68831b40 con 0x55ba6893cc10 2024-04-13T05:17:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.078+0000 7f9550a00640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba6893cc10 0x55ba6893e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.079+0000 7f9550a00640 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba6893cc10 0x55ba6893e080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=970705313fb23b45 server_cookie=2b58a32ae31edb31 in_seq=0 out_seq=0 2024-04-13T05:17:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.080+0000 7f9532a00640 10 client.15222 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:04.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9532a00640 1 -- 172.21.15.57:0/2214633221 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba68831b40 con 0x55ba6893cc10 2024-04-13T05:17:04.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9532a00640 10 client.15222 handle_command_reply: tid=0 2024-04-13T05:17:04.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 client.15222 shutdown 2024-04-13T05:17:04.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 msgr2=0x55ba6892faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 0x55ba6892faa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f954000bad0 tx=0x7f954000bf90 comp rx=0 tx=0).stop 2024-04-13T05:17:04.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 shutdown_connections 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba6893cc10 0x55ba6893e080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba6892ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6892b0d0 0x55ba68937020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 --2- 172.21.15.57:0/2214633221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba68925570 0x55ba6892faa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9532a00640 0 client.15222 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 >> 172.21.15.57:0/2214633221 conn(0x55ba68920ae0 msgr2=0x55ba6891fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 shutdown_connections 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 1 -- 172.21.15.57:0/2214633221 wait complete. 2024-04-13T05:17:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.082+0000 7f9556c62740 20 client.15222 trim_cache size 0 max 16384 2024-04-13T05:17:04.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 Processor -- start 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 -- start start 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba6892d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6891bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ba6893c3c0 con 0x55ba6892b0d0 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba6893ca90 con 0x55ba68925f60 2024-04-13T05:17:04.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9556c62740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba6893d160 con 0x55ba68925570 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9551e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6891bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9551e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6891bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52090/0 (socket says 172.21.15.57:52090) 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.103+0000 7f9551e00640 1 -- 172.21.15.57:0/495452953 learned_addr learned my addr 172.21.15.57:0/495452953 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9550a00640 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551400640 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba6892d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551e00640 1 -- 172.21.15.57:0/495452953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 msgr2=0x55ba6892d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551e00640 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba6892d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551e00640 1 -- 172.21.15.57:0/495452953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 msgr2=0x55ba68936fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551e00640 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551e00640 1 -- 172.21.15.57:0/495452953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba68926340 con 0x55ba6892b0d0 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551400640 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba6892d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f9551e00640 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6891bce0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f95440a51d0 tx=0x7f954409c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc074c52b62da91e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f953dc00640 1 -- 172.21.15.57:0/495452953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954409ca90 con 0x55ba6892b0d0 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f953dc00640 1 -- 172.21.15.57:0/495452953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95440a0e40 con 0x55ba6892b0d0 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.104+0000 7f953dc00640 1 -- 172.21.15.57:0/495452953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95440a54d0 con 0x55ba6892b0d0 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 -- 172.21.15.57:0/495452953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 msgr2=0x55ba6891bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6891bce0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f95440a51d0 tx=0x7f954409c3f0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 -- 172.21.15.57:0/495452953 shutdown_connections 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba6891bce0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 --2- 172.21.15.57:0/495452953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba6892d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 -- 172.21.15.57:0/495452953 >> 172.21.15.57:0/495452953 conn(0x55ba68920ae0 msgr2=0x55ba68920ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 -- 172.21.15.57:0/495452953 shutdown_connections 2024-04-13T05:17:04.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.105+0000 7f9556c62740 1 -- 172.21.15.57:0/495452953 wait complete. 2024-04-13T05:17:04.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 -- start start 2024-04-13T05:17:04.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 Processor -- start 2024-04-13T05:17:04.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68929ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba689a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ba6893df00 con 0x55ba6892b0d0 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba6893d830 con 0x55ba68925f60 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba6893e5d0 con 0x55ba68925570 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.106+0000 7f9556c62740 10 client.0 _ll_register_callbacks cb 0x7f95400043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba689a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba689a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52096/0 (socket says 172.21.15.57:52096) 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 -- 172.21.15.57:0/3067552270 learned_addr learned my addr 172.21.15.57:0/3067552270 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551400640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68929ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9550a00640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 -- 172.21.15.57:0/3067552270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 msgr2=0x55ba68929ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68929ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 -- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 msgr2=0x55ba68936fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 -- 172.21.15.57:0/3067552270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba689a1490 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9550a00640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9551e00640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba689a0c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9544018660 tx=0x7f9544018500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56fad1cf7e09845 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.107+0000 7f9532a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9532a00640 1 -- 172.21.15.57:0/3067552270 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954400f7f0 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9532a00640 1 -- 172.21.15.57:0/3067552270 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f954409bee0 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95440a0e40 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9532a00640 1 -- 172.21.15.57:0/3067552270 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f95440a5700 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9556c62740 10 client.15228 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ba689a2700 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.108+0000 7f9532a00640 1 -- 172.21.15.57:0/3067552270 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95440a0e40 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9532a00640 1 -- 172.21.15.57:0/3067552270 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f95440a1fd0 con 0x55ba6892b0d0 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9556c62740 10 client.15228 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9556c62740 10 client.15228 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-13T05:17:04.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9556c62740 20 client.15228 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:04.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9556c62740 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba689a6db0 0x55ba689a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9556c62740 4 client.15228 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:17:04.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "status"}) -- 0x55ba688993b0 con 0x55ba689a6db0 2024-04-13T05:17:04.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.109+0000 7f9551400640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba689a6db0 0x55ba689a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.110+0000 7f9551400640 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba689a6db0 0x55ba689a38c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd797d16f024c3c9 server_cookie=50f88dec987a6584 in_seq=0 out_seq=0 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.110+0000 7f9532a00640 10 client.15228 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.111+0000 7f9532a00640 1 -- 172.21.15.57:0/3067552270 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba688993b0 con 0x55ba689a6db0 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.111+0000 7f9532a00640 10 client.15228 handle_command_reply: tid=0 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.111+0000 7f9556c62740 1 client.15228 shutdown 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.111+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 msgr2=0x55ba689a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba689a0c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9544018660 tx=0x7f9544018500 comp rx=0 tx=0).stop 2024-04-13T05:17:04.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 shutdown_connections 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55ba689a6db0 0x55ba689a38c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba68925f60 0x55ba68936fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ba6892b0d0 0x55ba689a0c60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 --2- 172.21.15.57:0/3067552270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba68925570 0x55ba68929ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 >> 172.21.15.57:0/3067552270 conn(0x55ba68920ae0 msgr2=0x55ba6891bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9532a00640 0 client.15228 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 shutdown_connections 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 1 -- 172.21.15.57:0/3067552270 wait complete. 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.112+0000 7f9556c62740 20 client.15228 trim_cache size 0 max 16384 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34427, 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 102.81870378000001, 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 94, 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 165.78175643099999 2024-04-13T05:17:04.117 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95340a3c30 msgr2=0x7f95340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95340a3c30 0x7f95340a60f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ba689267c0 tx=0x7f9538048000 comp rx=0 tx=0).stop 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 msgr2=0x7f954c1a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 0x7f954c1a5de0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f954400cce0 tx=0x7f954400a620 comp rx=0 tx=0).stop 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 shutdown_connections 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95340a3c30 0x7f95340a60f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f954c10a1d0 0x7f954c1a5de0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f954c109810 0x7f954c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 --2- 172.21.15.57:0/2882629475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f954c108620 0x7f954c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 >> 172.21.15.57:0/2882629475 conn(0x7f954c0775d0 msgr2=0x7f954c10cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 shutdown_connections 2024-04-13T05:17:04.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.113+0000 7f9553e00640 1 -- 172.21.15.57:0/2882629475 wait complete. 2024-04-13T05:17:04.130 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:03 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3794701524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.130 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:03 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2359996026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.130 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:03 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/56474159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-13T05:17:04.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:03 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3794701524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:03 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2359996026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:03 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/56474159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:03 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3794701524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:03 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2359996026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:03 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/56474159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 -- 172.21.15.57:0/793731889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 msgr2=0x7f2964108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 --2- 172.21.15.57:0/793731889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f2964108a40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2958009900 tx=0x7f295802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 -- 172.21.15.57:0/793731889 shutdown_connections 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 --2- 172.21.15.57:0/793731889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 0x7f2964110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 --2- 172.21.15.57:0/793731889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2964109810 0x7f2964109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 --2- 172.21.15.57:0/793731889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f2964108a40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 -- 172.21.15.57:0/793731889 >> 172.21.15.57:0/793731889 conn(0x7f29640775c0 msgr2=0x7f2964079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 -- 172.21.15.57:0/793731889 shutdown_connections 2024-04-13T05:17:04.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.354+0000 7f296be00640 1 -- 172.21.15.57:0/793731889 wait complete. 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.355+0000 7f296be00640 1 Processor -- start 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.355+0000 7f296be00640 1 -- start start 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.355+0000 7f296be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f296419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.355+0000 7f296be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2964109810 0x7f296419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.355+0000 7f296be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 0x7f29641a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f296be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2964114150 con 0x7f2964108620 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f296be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2964113fd0 con 0x7f296410a1d0 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f296be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29641142d0 con 0x7f2964109810 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f296419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f296419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52120/0 (socket says 172.21.15.57:52120) 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 -- 172.21.15.57:0/1935845191 learned_addr learned my addr 172.21.15.57:0/1935845191 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2968a00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2964109810 0x7f296419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969e00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 0x7f29641a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 -- 172.21.15.57:0/1935845191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2964109810 msgr2=0x7f296419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2964109810 0x7f296419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 -- 172.21.15.57:0/1935845191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 msgr2=0x7f29641a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 0x7f29641a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969400640 1 -- 172.21.15.57:0/1935845191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29641a64a0 con 0x7f2964108620 2024-04-13T05:17:04.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.356+0000 7f2969e00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 0x7f29641a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:04.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.357+0000 7f2969400640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f296419e880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2958000c00 tx=0x7f2958004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.357+0000 7f2962000640 1 -- 172.21.15.57:0/1935845191 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958043020 con 0x7f2964108620 2024-04-13T05:17:04.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.357+0000 7f2962000640 1 -- 172.21.15.57:0/1935845191 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2958004aa0 con 0x7f2964108620 2024-04-13T05:17:04.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.357+0000 7f2962000640 1 -- 172.21.15.57:0/1935845191 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958003d00 con 0x7f2964108620 2024-04-13T05:17:04.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.357+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29641a6730 con 0x7f2964108620 2024-04-13T05:17:04.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.357+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29641a6c30 con 0x7f2964108620 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.360+0000 7f296eded740 1 Processor -- start 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 -- start start 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f2962000640 1 -- 172.21.15.57:0/1935845191 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2958046020 con 0x7f2964108620 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec08990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08f60 0x5626aec0da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 0x5626aec10560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5626aeb6e990 con 0x5626aec0e0d0 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5626aeb15530 con 0x5626aec08570 2024-04-13T05:17:04.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f296eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626aeb6a950 con 0x5626aec08f60 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f2968a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 0x5626aec10560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f2968a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 0x5626aec10560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52126/0 (socket says 172.21.15.57:52126) 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.361+0000 7f2968a00640 1 -- 172.21.15.57:0/2059208410 learned_addr learned my addr 172.21.15.57:0/2059208410 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2969e00640 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec08990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2969400640 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08f60 0x5626aec0da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2968a00640 1 -- 172.21.15.57:0/2059208410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08f60 msgr2=0x5626aec0da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2968a00640 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08f60 0x5626aec0da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2968a00640 1 -- 172.21.15.57:0/2059208410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 msgr2=0x5626aec08990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2968a00640 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec08990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2968a00640 1 -- 172.21.15.57:0/2059208410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ae8812d0 con 0x5626aec0e0d0 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2969e00640 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec08990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:04.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2968a00640 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 0x5626aec10560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f295000bc90 tx=0x7f295000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edba8a99acdc62db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2962000640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f294c0a3c30 0x7f294c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.362+0000 7f2956600640 1 -- 172.21.15.57:0/2059208410 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950015070 con 0x5626aec0e0d0 2024-04-13T05:17:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f2962000640 1 -- 172.21.15.57:0/1935845191 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f29580f2d60 con 0x7f2964108620 2024-04-13T05:17:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f2969e00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f294c0a3c30 0x7f294c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f2956600640 1 -- 172.21.15.57:0/2059208410 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f295001a910 con 0x5626aec0e0d0 2024-04-13T05:17:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f2956600640 1 -- 172.21.15.57:0/2059208410 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950011840 con 0x5626aec0e0d0 2024-04-13T05:17:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f296eded740 1 -- 172.21.15.57:0/2059208410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 msgr2=0x5626aec10560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f296eded740 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 0x5626aec10560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f295000bc90 tx=0x7f295000f730 comp rx=0 tx=0).stop 2024-04-13T05:17:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.363+0000 7f2969e00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f294c0a3c30 0x7f294c0a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5626aec097c0 tx=0x7f295c008570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 -- 172.21.15.57:0/2059208410 shutdown_connections 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec0e0d0 0x5626aec10560 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08f60 0x5626aec0da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 --2- 172.21.15.57:0/2059208410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec08990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 -- 172.21.15.57:0/2059208410 >> 172.21.15.57:0/2059208410 conn(0x5626aec03ae0 msgr2=0x5626aec03ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 -- 172.21.15.57:0/2059208410 shutdown_connections 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 -- 172.21.15.57:0/2059208410 wait complete. 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.364+0000 7f296eded740 1 -- start start 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 Processor -- start 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec12a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec12fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5626aec11080 con 0x5626aec08f60 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5626aeb288f0 con 0x5626aec08570 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626aec11200 con 0x5626aec0e0d0 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f296eded740 10 client.0 _ll_register_callbacks cb 0x5626aeb282e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f2968a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec12fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f2968a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec12fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52142/0 (socket says 172.21.15.57:52142) 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.365+0000 7f2968a00640 1 -- 172.21.15.57:0/2801014854 learned_addr learned my addr 172.21.15.57:0/2801014854 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2969400640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec12a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2969e00640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2968a00640 1 -- 172.21.15.57:0/2801014854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 msgr2=0x5626aec1a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2968a00640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec1a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2968a00640 1 -- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 msgr2=0x5626aec12a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2968a00640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec12a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2968a00640 1 -- 172.21.15.57:0/2801014854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ae94b6b0 con 0x5626aec08f60 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f2968a00640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec12fd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f295001a6c0 tx=0x7f295001a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8f3b9b62e98406 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f294aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f294aa00640 1 -- 172.21.15.57:0/2801014854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950028060 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.366+0000 7f294aa00640 1 -- 172.21.15.57:0/2801014854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f295001acd0 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.367+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f295001a910 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.367+0000 7f294aa00640 1 -- 172.21.15.57:0/2801014854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f295002f050 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.367+0000 7f296eded740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.367+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5626aec1d380 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.367+0000 7f294aa00640 1 -- 172.21.15.57:0/2801014854 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f295001a910 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.367+0000 7f294aa00640 1 -- 172.21.15.57:0/2801014854 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2950004b80 con 0x5626aec08f60 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f296eded740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f296eded740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f296eded740 20 client.15240 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f296eded740 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec1f610 0x5626aec20a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f296eded740 4 client.15240 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:17:04.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626aeb14b40 con 0x5626aec1f610 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.368+0000 7f2969400640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec1f610 0x5626aec20a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.370+0000 7f2969400640 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec1f610 0x5626aec20a80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97ffb487c94e2ec5 server_cookie=223082dc34fd0df6 in_seq=0 out_seq=0 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.370+0000 7f294aa00640 10 client.15240 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.371+0000 7f294aa00640 1 -- 172.21.15.57:0/2801014854 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626aeb14b40 con 0x5626aec1f610 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.371+0000 7f294aa00640 10 client.15240 handle_command_reply: tid=0 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 client.15240 shutdown 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 msgr2=0x5626aec12fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec12fd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f295001a6c0 tx=0x7f295001a6f0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 shutdown_connections 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec1f610 0x5626aec20a80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec1a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec12fd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 --2- 172.21.15.57:0/2801014854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08570 0x5626aec12a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 >> 172.21.15.57:0/2801014854 conn(0x5626aec03ae0 msgr2=0x5626aec02d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f294aa00640 0 client.15240 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 shutdown_connections 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 1 -- 172.21.15.57:0/2801014854 wait complete. 2024-04-13T05:17:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.372+0000 7f296eded740 20 client.15240 trim_cache size 0 max 16384 2024-04-13T05:17:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.392+0000 7f296eded740 1 Processor -- start 2024-04-13T05:17:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.392+0000 7f296eded740 1 -- start start 2024-04-13T05:17:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.392+0000 7f296eded740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 0x5626aec10f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f296eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08f60 0x5626aebfece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f296eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec02470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f296eded740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5626aec1edc0 con 0x5626aec08570 2024-04-13T05:17:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f296eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5626aec1f490 con 0x5626aec08f60 2024-04-13T05:17:04.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f296eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626aec1fb60 con 0x5626aec0e0d0 2024-04-13T05:17:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f2969400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 0x5626aec10f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f2969400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 0x5626aec10f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52150/0 (socket says 172.21.15.57:52150) 2024-04-13T05:17:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f2969400640 1 -- 172.21.15.57:0/2067872117 learned_addr learned my addr 172.21.15.57:0/2067872117 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f2968a00640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08f60 0x5626aebfece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f2969e00640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec02470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.393+0000 7f2969400640 1 -- 172.21.15.57:0/2067872117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 msgr2=0x5626aec02470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2969400640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec02470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2969400640 1 -- 172.21.15.57:0/2067872117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08f60 msgr2=0x5626aebfece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2969400640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08f60 0x5626aebfece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2969400640 1 -- 172.21.15.57:0/2067872117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626aec09340 con 0x5626aec08570 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2968a00640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08f60 0x5626aebfece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2969e00640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec02470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2969400640 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 0x5626aec10f60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2958005820 tx=0x7f29580df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86abfa866eeb9410 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2955c00640 1 -- 172.21.15.57:0/2067872117 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580dde60 con 0x5626aec08570 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2955c00640 1 -- 172.21.15.57:0/2067872117 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29580bf3a0 con 0x5626aec08570 2024-04-13T05:17:04.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.394+0000 7f2955c00640 1 -- 172.21.15.57:0/2067872117 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958003680 con 0x5626aec08570 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 -- 172.21.15.57:0/2067872117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 msgr2=0x5626aec10f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 0x5626aec10f60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2958005820 tx=0x7f29580df040 comp rx=0 tx=0).stop 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 -- 172.21.15.57:0/2067872117 shutdown_connections 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec08f60 0x5626aebfece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec0e0d0 0x5626aec02470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 --2- 172.21.15.57:0/2067872117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08570 0x5626aec10f60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 -- 172.21.15.57:0/2067872117 >> 172.21.15.57:0/2067872117 conn(0x5626aec03ae0 msgr2=0x5626aec03ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 -- 172.21.15.57:0/2067872117 shutdown_connections 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.395+0000 7f296eded740 1 -- 172.21.15.57:0/2067872117 wait complete. 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 -- start start 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 Processor -- start 2024-04-13T05:17:04.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08570 0x5626aec0bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec0c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec0e0d0 0x5626aec18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5626aec20900 con 0x5626aec08f60 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5626aec20230 con 0x5626aec0e0d0 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626aec20fd0 con 0x5626aec08570 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.396+0000 7f296eded740 10 client.0 _ll_register_callbacks cb 0x7f295000bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec0c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec0c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52152/0 (socket says 172.21.15.57:52152) 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 -- 172.21.15.57:0/3581696126 learned_addr learned my addr 172.21.15.57:0/3581696126 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2969e00640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec0e0d0 0x5626aec18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2969400640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08570 0x5626aec0bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 -- 172.21.15.57:0/3581696126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08570 msgr2=0x5626aec0bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08570 0x5626aec0bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 -- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec0e0d0 msgr2=0x5626aec18050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec0e0d0 0x5626aec18050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 -- 172.21.15.57:0/3581696126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626aec18690 con 0x5626aec08f60 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f2968a00640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec0c4c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f295000f770 tx=0x7f2950005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7290677582b0c639 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.397+0000 7f294aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f294aa00640 1 -- 172.21.15.57:0/3581696126 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950028070 con 0x5626aec08f60 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f294aa00640 1 -- 172.21.15.57:0/3581696126 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2950005000 con 0x5626aec08f60 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29580bf3a0 con 0x5626aec08f60 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f294aa00640 1 -- 172.21.15.57:0/3581696126 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2950015070 con 0x5626aec08f60 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f296eded740 10 client.15246 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:04.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5626aec855c0 con 0x5626aec08f60 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f294aa00640 1 -- 172.21.15.57:0/3581696126 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29580bf3a0 con 0x5626aec08f60 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f294aa00640 1 -- 172.21.15.57:0/3581696126 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2950004ae0 con 0x5626aec08f60 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f296eded740 10 client.15246 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f296eded740 10 client.15246 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.398+0000 7f296eded740 20 client.15246 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.399+0000 7f296eded740 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec8a090 0x5626aec88da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.399+0000 7f296eded740 4 client.15246 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.399+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "status"}) -- 0x5626aeb7c3b0 con 0x5626aec8a090 2024-04-13T05:17:04.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.399+0000 7f2969400640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec8a090 0x5626aec88da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.400+0000 7f2969400640 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec8a090 0x5626aec88da0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b28452985086ade3 server_cookie=d8dbdafa309eead8 in_seq=0 out_seq=0 2024-04-13T05:17:04.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.400+0000 7f294aa00640 10 client.15246 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:04.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f294aa00640 1 -- 172.21.15.57:0/3581696126 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5626aeb7c3b0 con 0x5626aec8a090 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f294aa00640 10 client.15246 handle_command_reply: tid=0 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 client.15246 shutdown 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 msgr2=0x5626aec0c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec0c4c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f295000f770 tx=0x7f2950005b90 comp rx=0 tx=0).stop 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 shutdown_connections 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5626aec8a090 0x5626aec88da0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5626aec0e0d0 0x5626aec18050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626aec08f60 0x5626aec0c4c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 --2- 172.21.15.57:0/3581696126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626aec08570 0x5626aec0bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f294aa00640 0 client.15246 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 >> 172.21.15.57:0/3581696126 conn(0x5626aec03ae0 msgr2=0x5626aebfece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 shutdown_connections 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 1 -- 172.21.15.57:0/3581696126 wait complete. 2024-04-13T05:17:04.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.401+0000 7f296eded740 20 client.15246 trim_cache size 0 max 16384 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34457, 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 104.127989996, 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 94, 2024-04-13T05:17:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T05:17:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 153.78230509100001 2024-04-13T05:17:04.408 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:04.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f294c0a3c30 msgr2=0x7f294c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f294c0a3c30 0x7f294c0a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5626aec097c0 tx=0x7f295c008570 comp rx=0 tx=0).stop 2024-04-13T05:17:04.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 msgr2=0x7f296419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f296419e880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2958000c00 tx=0x7f2958004270 comp rx=0 tx=0).stop 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 shutdown_connections 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f294c0a3c30 0x7f294c0a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f296410a1d0 0x7f29641a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2964109810 0x7f296419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 --2- 172.21.15.57:0/1935845191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2964108620 0x7f296419e880 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 >> 172.21.15.57:0/1935845191 conn(0x7f29640775c0 msgr2=0x7f296410cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 shutdown_connections 2024-04-13T05:17:04.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.402+0000 7f296be00640 1 -- 172.21.15.57:0/1935845191 wait complete. 2024-04-13T05:17:04.424 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-13T05:17:04.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 -- 172.21.15.57:0/1224367214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 msgr2=0x7f0748104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 --2- 172.21.15.57:0/1224367214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f0748104d50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0738009900 tx=0x7f073802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:04.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 -- 172.21.15.57:0/1224367214 shutdown_connections 2024-04-13T05:17:04.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 --2- 172.21.15.57:0/1224367214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748105290 0x7f0748107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 --2- 172.21.15.57:0/1224367214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0748107bc0 0x7f074810a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 --2- 172.21.15.57:0/1224367214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f0748104d50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 -- 172.21.15.57:0/1224367214 >> 172.21.15.57:0/1224367214 conn(0x7f07480fc370 msgr2=0x7f07480fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 -- 172.21.15.57:0/1224367214 shutdown_connections 2024-04-13T05:17:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.643+0000 7f074d600640 1 -- 172.21.15.57:0/1224367214 wait complete. 2024-04-13T05:17:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.644+0000 7f074d600640 1 Processor -- start 2024-04-13T05:17:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.644+0000 7f074d600640 1 -- start start 2024-04-13T05:17:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f07481980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748105290 0x7f0748198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0748107bc0 0x7f074819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074d600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f074810da20 con 0x7f0748102960 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f074810d8a0 con 0x7f0748105290 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074d600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f074810dba0 con 0x7f0748107bc0 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f0747e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f07481980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f0747e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f07481980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52178/0 (socket says 172.21.15.57:52178) 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f0747e00640 1 -- 172.21.15.57:0/4048920145 learned_addr learned my addr 172.21.15.57:0/4048920145 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.645+0000 7f074cc00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0748107bc0 0x7f074819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747400640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748105290 0x7f0748198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747e00640 1 -- 172.21.15.57:0/4048920145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0748107bc0 msgr2=0x7f074819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747e00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0748107bc0 0x7f074819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747e00640 1 -- 172.21.15.57:0/4048920145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748105290 msgr2=0x7f0748198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747e00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748105290 0x7f0748198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747e00640 1 -- 172.21.15.57:0/4048920145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074819fd10 con 0x7f0748102960 2024-04-13T05:17:04.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0747e00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f07481980f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0738006010 tx=0x7f0738005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0744c00640 1 -- 172.21.15.57:0/4048920145 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738043020 con 0x7f0748102960 2024-04-13T05:17:04.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0744c00640 1 -- 172.21.15.57:0/4048920145 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07380043d0 con 0x7f0748102960 2024-04-13T05:17:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f0744c00640 1 -- 172.21.15.57:0/4048920145 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738003b80 con 0x7f0748102960 2024-04-13T05:17:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.646+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074819ffa0 con 0x7f0748102960 2024-04-13T05:17:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.647+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07481a03b0 con 0x7f0748102960 2024-04-13T05:17:04.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 Processor -- start 2024-04-13T05:17:04.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0744c00640 1 -- 172.21.15.57:0/4048920145 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0738046020 con 0x7f0748102960 2024-04-13T05:17:04.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 -- start start 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b634fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b63a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b63cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56325b596990 con 0x56325b6355b0 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56325b53d530 con 0x56325b63a720 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.650+0000 7f0751bd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325b592950 con 0x56325b634bc0 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b63a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b63a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52188/0 (socket says 172.21.15.57:52188) 2024-04-13T05:17:04.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 -- 172.21.15.57:0/2280078480 learned_addr learned my addr 172.21.15.57:0/2280078480 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f074cc00640 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b634fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747400640 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b63cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 -- 172.21.15.57:0/2280078480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 msgr2=0x56325b634fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b634fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 -- 172.21.15.57:0/2280078480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 msgr2=0x56325b63cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b63cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747e00640 1 -- 172.21.15.57:0/2280078480 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325b2a92d0 con 0x56325b6355b0 2024-04-13T05:17:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.651+0000 7f0747400640 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b63cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:04.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f0747e00640 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b63a0b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f07380f6ac0 tx=0x7f07380f6af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c4f4f6764ab0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f073d200640 1 -- 172.21.15.57:0/2280078480 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07380fb8e0 con 0x56325b6355b0 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f0744c00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f072c0a3bb0 0x7f072c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f073d200640 1 -- 172.21.15.57:0/2280078480 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07380f9070 con 0x56325b6355b0 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f073d200640 1 -- 172.21.15.57:0/2280078480 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07380baca0 con 0x56325b6355b0 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f0744c00640 1 -- 172.21.15.57:0/4048920145 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f07380f2b20 con 0x7f0748102960 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.652+0000 7f074cc00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f072c0a3bb0 0x7f072c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 -- 172.21.15.57:0/2280078480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 msgr2=0x56325b63a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b63a0b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f07380f6ac0 tx=0x7f07380f6af0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 -- 172.21.15.57:0/2280078480 shutdown_connections 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b63cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b63a0b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2280078480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b634fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 -- 172.21.15.57:0/2280078480 >> 172.21.15.57:0/2280078480 conn(0x56325b62bd20 msgr2=0x56325b62c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 -- 172.21.15.57:0/2280078480 shutdown_connections 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f074cc00640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f072c0a3bb0 0x7f072c0a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56325b635e10 tx=0x7f0740009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.653+0000 7f0751bd0740 1 -- 172.21.15.57:0/2280078480 wait complete. 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 -- start start 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 Processor -- start 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 0x56325b63ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56325b63d490 con 0x56325b634bc0 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56325b5508f0 con 0x56325b63a720 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.654+0000 7f0751bd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325b63d610 con 0x56325b6355b0 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0751bd0740 10 client.0 _ll_register_callbacks cb 0x56325b5502e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:04.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 0x56325b63ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 0x56325b63ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52198/0 (socket says 172.21.15.57:52198) 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 -- 172.21.15.57:0/2622344615 learned_addr learned my addr 172.21.15.57:0/2622344615 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747400640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f074cc00640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 -- 172.21.15.57:0/2622344615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 msgr2=0x56325b63f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 -- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 msgr2=0x56325b646380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.655+0000 7f0747e00640 1 -- 172.21.15.57:0/2622344615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325b3736b0 con 0x56325b634bc0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0747e00640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 0x56325b63ee00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07380be990 tx=0x7f0738005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7572a7819edd70a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0731600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0731600640 1 -- 172.21.15.57:0/2622344615 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07380be180 con 0x56325b634bc0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0731600640 1 -- 172.21.15.57:0/2622344615 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f07380bb070 con 0x56325b634bc0 2024-04-13T05:17:04.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07380f9070 con 0x56325b634bc0 2024-04-13T05:17:04.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0731600640 1 -- 172.21.15.57:0/2622344615 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f07380be360 con 0x56325b634bc0 2024-04-13T05:17:04.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0751bd0740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:04.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56325b63d490 con 0x56325b634bc0 2024-04-13T05:17:04.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0731600640 1 -- 172.21.15.57:0/2622344615 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07380d9080 con 0x56325b634bc0 2024-04-13T05:17:04.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.656+0000 7f0731600640 1 -- 172.21.15.57:0/2622344615 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0738100420 con 0x56325b634bc0 2024-04-13T05:17:04.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0751bd0740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:04.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0751bd0740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T05:17:04.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0751bd0740 20 client.15258 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:04.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b64b8f0 0x56325b64cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0751bd0740 4 client.15258 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56325b53cb40 con 0x56325b64b8f0 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.657+0000 7f0747400640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b64b8f0 0x56325b64cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.658+0000 7f0747400640 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b64b8f0 0x56325b64cd60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb32bf3e148b10c7 server_cookie=e480fcae8e9a1277 in_seq=0 out_seq=0 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.658+0000 7f0731600640 10 client.15258 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.660+0000 7f0731600640 1 -- 172.21.15.57:0/2622344615 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56325b53cb40 con 0x56325b64b8f0 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.660+0000 7f0731600640 10 client.15258 handle_command_reply: tid=0 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 client.15258 shutdown 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 msgr2=0x56325b63ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 0x56325b63ee00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07380be990 tx=0x7f0738005b50 comp rx=0 tx=0).stop 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 shutdown_connections 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b64b8f0 0x56325b64cd60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2622344615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b634bc0 0x56325b63ee00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 >> 172.21.15.57:0/2622344615 conn(0x56325b62bd20 msgr2=0x56325b62af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0731600640 0 client.15258 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:04.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 shutdown_connections 2024-04-13T05:17:04.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 1 -- 172.21.15.57:0/2622344615 wait complete. 2024-04-13T05:17:04.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.661+0000 7f0751bd0740 20 client.15258 trim_cache size 0 max 16384 2024-04-13T05:17:04.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.681+0000 7f0751bd0740 1 Processor -- start 2024-04-13T05:17:04.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 -- start start 2024-04-13T05:17:04.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b634bc0 0x56325b634390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 0x56325b636ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56325b64b0a0 con 0x56325b63a720 2024-04-13T05:17:04.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56325b64b770 con 0x56325b634bc0 2024-04-13T05:17:04.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0751bd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325b64be40 con 0x56325b6355b0 2024-04-13T05:17:04.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f074cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 0x56325b636ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f074cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 0x56325b636ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52200/0 (socket says 172.21.15.57:52200) 2024-04-13T05:17:04.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f074cc00640 1 -- 172.21.15.57:0/2603340141 learned_addr learned my addr 172.21.15.57:0/2603340141 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.682+0000 7f0747e00640 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b634bc0 0x56325b634390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f0747400640 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f074cc00640 1 -- 172.21.15.57:0/2603340141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 msgr2=0x56325b63cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f074cc00640 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f074cc00640 1 -- 172.21.15.57:0/2603340141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b634bc0 msgr2=0x56325b634390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f074cc00640 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b634bc0 0x56325b634390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f074cc00640 1 -- 172.21.15.57:0/2603340141 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325b635990 con 0x56325b63a720 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f0747e00640 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b634bc0 0x56325b634390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.683+0000 7f074cc00640 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 0x56325b636ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f07400049c0 tx=0x7f0740015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4f363f81a88c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0733e00640 1 -- 172.21.15.57:0/2603340141 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740026460 con 0x56325b63a720 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0733e00640 1 -- 172.21.15.57:0/2603340141 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0740026ab0 con 0x56325b63a720 2024-04-13T05:17:04.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0733e00640 1 -- 172.21.15.57:0/2603340141 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074001daa0 con 0x56325b63a720 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 -- 172.21.15.57:0/2603340141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 msgr2=0x56325b636ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 0x56325b636ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f07400049c0 tx=0x7f0740015000 comp rx=0 tx=0).stop 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 -- 172.21.15.57:0/2603340141 shutdown_connections 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b63a720 0x56325b636ab0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b6355b0 0x56325b63cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 --2- 172.21.15.57:0/2603340141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b634bc0 0x56325b634390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 -- 172.21.15.57:0/2603340141 >> 172.21.15.57:0/2603340141 conn(0x56325b62bd20 msgr2=0x56325b626da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 -- 172.21.15.57:0/2603340141 shutdown_connections 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.684+0000 7f0751bd0740 1 -- 172.21.15.57:0/2603340141 wait complete. 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 -- start start 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 Processor -- start 2024-04-13T05:17:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b632fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b633500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56325b64cbe0 con 0x56325b6355b0 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56325b64c510 con 0x56325b63a720 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325b64d2b0 con 0x56325b634bc0 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.685+0000 7f0751bd0740 10 client.0 _ll_register_callbacks cb 0x7f0738004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b633500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b633500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52206/0 (socket says 172.21.15.57:52206) 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 -- 172.21.15.57:0/1576963773 learned_addr learned my addr 172.21.15.57:0/1576963773 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747e00640 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b632fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 -- 172.21.15.57:0/1576963773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 msgr2=0x56325b632fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b632fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 -- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 msgr2=0x56325b646310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 -- 172.21.15.57:0/1576963773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325b633d50 con 0x56325b6355b0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0747400640 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b633500 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x56325b62cc90 tx=0x7f073400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a4aa5c9cca5e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.686+0000 7f0731600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0731600640 1 -- 172.21.15.57:0/1576963773 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734009e00 con 0x56325b6355b0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0731600640 1 -- 172.21.15.57:0/1576963773 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f073401c630 con 0x56325b6355b0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0740026ab0 con 0x56325b6355b0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0731600640 1 -- 172.21.15.57:0/1576963773 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f073402b050 con 0x56325b6355b0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0751bd0740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56325b6b51e0 con 0x56325b6355b0 2024-04-13T05:17:04.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0731600640 1 -- 172.21.15.57:0/1576963773 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0740026ab0 con 0x56325b6355b0 2024-04-13T05:17:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.687+0000 7f0731600640 1 -- 172.21.15.57:0/1576963773 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0734003aa0 con 0x56325b6355b0 2024-04-13T05:17:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0751bd0740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0751bd0740 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T05:17:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0751bd0740 20 client.15264 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0751bd0740 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b6b9e90 0x56325b6b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0751bd0740 4 client.15264 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "status"}) -- 0x56325b5a43b0 con 0x56325b6b9e90 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.688+0000 7f0747e00640 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b6b9e90 0x56325b6b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.689+0000 7f0747e00640 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b6b9e90 0x56325b6b8ba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed735072a5aa98e server_cookie=a656469cbb15368c in_seq=0 out_seq=0 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.689+0000 7f0731600640 10 client.15264 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0731600640 1 -- 172.21.15.57:0/1576963773 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56325b5a43b0 con 0x56325b6b9e90 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0731600640 10 client.15264 handle_command_reply: tid=0 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 client.15264 shutdown 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 msgr2=0x56325b633500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b633500 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x56325b62cc90 tx=0x7f073400f7f0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 shutdown_connections 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56325b6b9e90 0x56325b6b8ba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56325b63a720 0x56325b646310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56325b6355b0 0x56325b633500 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 --2- 172.21.15.57:0/1576963773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325b634bc0 0x56325b632fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 >> 172.21.15.57:0/1576963773 conn(0x56325b62bd20 msgr2=0x56325b636ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0731600640 0 client.15264 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 shutdown_connections 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 1 -- 172.21.15.57:0/1576963773 wait complete. 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.690+0000 7f0751bd0740 20 client.15264 trim_cache size 0 max 16384 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34499, 2024-04-13T05:17:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 131.09211487300001, 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 94, 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 137.77333794899999 2024-04-13T05:17:04.696 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f072c0a3bb0 msgr2=0x7f072c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f072c0a3bb0 0x7f072c0a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56325b635e10 tx=0x7f0740009500 comp rx=0 tx=0).stop 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 msgr2=0x7f07481980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f07481980f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0738006010 tx=0x7f0738005e40 comp rx=0 tx=0).stop 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 shutdown_connections 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f072c0a3bb0 0x7f072c0a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0748105290 0x7f0748198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0748107bc0 0x7f074819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 --2- 172.21.15.57:0/4048920145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0748102960 0x7f07481980f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:04.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 >> 172.21.15.57:0/4048920145 conn(0x7f07480fc370 msgr2=0x7f0748103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:04.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 shutdown_connections 2024-04-13T05:17:04.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:04.691+0000 7f074d600640 1 -- 172.21.15.57:0/4048920145 wait complete. 2024-04-13T05:17:04.711 INFO:tasks.cephadm:Setup complete, yielding 2024-04-13T05:17:04.712 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-13T05:17:04.721 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T05:17:04.722 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch status' 2024-04-13T05:17:05.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:04 smithi057 ceph-mon[32331]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:05.207 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:05.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:04 smithi083 ceph-mon[35133]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:05.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:04 smithi189 ceph-mon[35531]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:07.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:06 smithi057 ceph-mon[32331]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:07.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:06 smithi083 ceph-mon[35133]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:07.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:06 smithi189 ceph-mon[35531]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:07.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 -- 172.21.15.57:0/1623571117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95741035a0 msgr2=0x7f9574105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 --2- 172.21.15.57:0/1623571117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95741035a0 0x7f9574105990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9564009990 tx=0x7f9564009ca0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 -- 172.21.15.57:0/1623571117 shutdown_connections 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 --2- 172.21.15.57:0/1623571117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9574108800 0x7f957410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 --2- 172.21.15.57:0/1623571117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95741035a0 0x7f9574105990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 --2- 172.21.15.57:0/1623571117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9574105ed0 0x7f95741082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 -- 172.21.15.57:0/1623571117 >> 172.21.15.57:0/1623571117 conn(0x7f95740fcfb0 msgr2=0x7f95740ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 -- 172.21.15.57:0/1623571117 shutdown_connections 2024-04-13T05:17:07.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.634+0000 7f9579600700 1 -- 172.21.15.57:0/1623571117 wait complete. 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.635+0000 7f9579600700 1 Processor -- start 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.635+0000 7f9579600700 1 -- start start 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9579600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95741035a0 0x7f957419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9579600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9574105ed0 0x7f957419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9579600700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 0x7f95741a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9579600700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f957410e5e0 con 0x7f9574108800 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9579600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f957410e460 con 0x7f95741035a0 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9579600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f957410e760 con 0x7f9574105ed0 2024-04-13T05:17:07.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9573400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 0x7f95741a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9573400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 0x7f95741a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54678/0 (socket says 172.21.15.57:54678) 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9573400700 1 -- 172.21.15.57:0/1408054444 learned_addr learned my addr 172.21.15.57:0/1408054444 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f9572a00700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95741035a0 0x7f957419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.636+0000 7f956be00700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9574105ed0 0x7f957419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9573400700 1 -- 172.21.15.57:0/1408054444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9574105ed0 msgr2=0x7f957419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9573400700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9574105ed0 0x7f957419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9573400700 1 -- 172.21.15.57:0/1408054444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95741035a0 msgr2=0x7f957419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9573400700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95741035a0 0x7f957419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9573400700 1 -- 172.21.15.57:0/1408054444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741a2b90 con 0x7f9574108800 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9573400700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 0x7f95741a24f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f956c007ee0 tx=0x7f956c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c004db0 con 0x7f9574108800 2024-04-13T05:17:07.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956c00dbd0 con 0x7f9574108800 2024-04-13T05:17:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741a2e80 con 0x7f9574108800 2024-04-13T05:17:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.637+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c015400 con 0x7f9574108800 2024-04-13T05:17:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.638+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95741a3330 con 0x7f9574108800 2024-04-13T05:17:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.639+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95740697d0 con 0x7f9574108800 2024-04-13T05:17:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.641+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f956c005020 con 0x7f9574108800 2024-04-13T05:17:07.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.642+0000 7f956b400700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95580a3cc0 0x7f95580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:07.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.643+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f956c0d0260 con 0x7f9574108800 2024-04-13T05:17:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.643+0000 7f9572a00700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95580a3cc0 0x7f95580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:07.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.644+0000 7f9572a00700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95580a3cc0 0x7f95580a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f956400b5c0 tx=0x7f9564003850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.647+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95740697d0 con 0x7f9574108800 2024-04-13T05:17:07.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.899+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9574101ce0 con 0x7f95580a3cc0 2024-04-13T05:17:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.901+0000 7f956b400700 1 -- 172.21.15.57:0/1408054444 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9574101ce0 con 0x7f95580a3cc0 2024-04-13T05:17:07.903 INFO:teuthology.orchestra.run.smithi057.stdout:Backend: cephadm 2024-04-13T05:17:07.903 INFO:teuthology.orchestra.run.smithi057.stdout:Available: Yes 2024-04-13T05:17:07.903 INFO:teuthology.orchestra.run.smithi057.stdout:Paused: No 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.903+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95580a3cc0 msgr2=0x7f95580a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95580a3cc0 0x7f95580a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f956400b5c0 tx=0x7f9564003850 comp rx=0 tx=0).stop 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 msgr2=0x7f95741a24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 0x7f95741a24f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f956c007ee0 tx=0x7f956c00d3b0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 shutdown_connections 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f95580a3cc0 0x7f95580a6180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9574108800 0x7f95741a24f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95741035a0 0x7f957419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 --2- 172.21.15.57:0/1408054444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9574105ed0 0x7f957419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.904+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 >> 172.21.15.57:0/1408054444 conn(0x7f95740fcfb0 msgr2=0x7f95741015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.905+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 shutdown_connections 2024-04-13T05:17:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:07.905+0000 7f9579600700 1 -- 172.21.15.57:0/1408054444 wait complete. 2024-04-13T05:17:08.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-13T05:17:09.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:08 smithi057 ceph-mon[32331]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:09.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:08 smithi057 ceph-mon[32331]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:08 smithi083 ceph-mon[35133]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:08 smithi083 ceph-mon[35133]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:09.270 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:09.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:08 smithi189 ceph-mon[35531]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:08 smithi189 ceph-mon[35531]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:11.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:10 smithi057 ceph-mon[32331]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:11.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:10 smithi083 ceph-mon[35133]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:11.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:10 smithi189 ceph-mon[35531]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.613+0000 7f0fefe00700 1 -- 172.21.15.57:0/2852385471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe810acf0 msgr2=0x7f0fe81115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.613+0000 7f0fefe00700 1 --2- 172.21.15.57:0/2852385471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe810acf0 0x7f0fe81115a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000b0d0 tx=0x7f0fe000b3e0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 -- 172.21.15.57:0/2852385471 shutdown_connections 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 --2- 172.21.15.57:0/2852385471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe810acf0 0x7f0fe81115a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 --2- 172.21.15.57:0/2852385471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe8109150 0x7f0fe8109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 --2- 172.21.15.57:0/2852385471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fe810a350 0x7f0fe810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 -- 172.21.15.57:0/2852385471 >> 172.21.15.57:0/2852385471 conn(0x7f0fe8078120 msgr2=0x7f0fe807a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:11.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 -- 172.21.15.57:0/2852385471 shutdown_connections 2024-04-13T05:17:11.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.614+0000 7f0fefe00700 1 -- 172.21.15.57:0/2852385471 wait complete. 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.615+0000 7f0fefe00700 1 Processor -- start 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.615+0000 7f0fefe00700 1 -- start start 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.615+0000 7f0fefe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 0x7f0fe819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fefe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fe810a350 0x7f0fe819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fefe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810acf0 0x7f0fe81a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fefe00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe8114c40 con 0x7f0fe8109150 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fefe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe8114ac0 con 0x7f0fe810acf0 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fefe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe8114dc0 con 0x7f0fe810a350 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 0x7f0fe819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 0x7f0fe819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54702/0 (socket says 172.21.15.57:54702) 2024-04-13T05:17:11.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 -- 172.21.15.57:0/3757455263 learned_addr learned my addr 172.21.15.57:0/3757455263 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:11.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0feca00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fe810a350 0x7f0fe819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fede00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810acf0 0x7f0fe81a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 -- 172.21.15.57:0/3757455263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fe810a350 msgr2=0x7f0fe819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fe810a350 0x7f0fe819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 -- 172.21.15.57:0/3757455263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810acf0 msgr2=0x7f0fe81a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810acf0 0x7f0fe81a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fed400700 1 -- 172.21.15.57:0/3757455263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe81a6e20 con 0x7f0fe8109150 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.616+0000 7f0fede00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810acf0 0x7f0fe81a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.617+0000 7f0fed400700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 0x7f0fe819f1c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00d9b0 tx=0x7f0fdc00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.617+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc017030 con 0x7f0fe8109150 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.617+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fdc0044d0 con 0x7f0fe8109150 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.617+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe81a7110 con 0x7f0fe8109150 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.617+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc0039e0 con 0x7f0fe8109150 2024-04-13T05:17:11.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.617+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81a7650 con 0x7f0fe8109150 2024-04-13T05:17:11.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.618+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe80697a0 con 0x7f0fe8109150 2024-04-13T05:17:11.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.621+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0fdc01a070 con 0x7f0fe8109150 2024-04-13T05:17:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.622+0000 7f0fe6000700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0fd00a3c40 0x7f0fd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.622+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0fdc0d01c0 con 0x7f0fe8109150 2024-04-13T05:17:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.622+0000 7f0feca00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0fd00a3c40 0x7f0fd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:11.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.623+0000 7f0feca00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0fd00a3c40 0x7f0fd00a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4005950 tx=0x7f0fd40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.627+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fdc09c7a0 con 0x7f0fe8109150 2024-04-13T05:17:11.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.870+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0fe8062ef0 con 0x7f0fd00a3c40 2024-04-13T05:17:11.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.912+0000 7f0fe6000700 1 -- 172.21.15.57:0/3757455263 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f0fe8062ef0 con 0x7f0fd00a3c40 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.a smithi057 running (2m) 27s ago 2m 12.4M - 19.0.0-2884-g218a6914 a4e4811815e4 3c7383db5f10 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.b smithi083 running (2m) 72s ago 2m 24.5M - 19.0.0-2884-g218a6914 a4e4811815e4 77b13aa5330f 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.c smithi189 running (2m) 73s ago 2m 14.5M - 19.0.0-2884-g218a6914 a4e4811815e4 adfde2f770c8 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.d smithi057 running (2m) 27s ago 2m 12.5M - 19.0.0-2884-g218a6914 a4e4811815e4 5339d0a98846 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.e smithi083 running (2m) 72s ago 2m 15.6M - 19.0.0-2884-g218a6914 a4e4811815e4 b1c3859e1ae6 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.f smithi189 running (2m) 73s ago 2m 12.4M - 19.0.0-2884-g218a6914 a4e4811815e4 3c258f412145 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.g smithi057 running (2m) 27s ago 2m 12.4M - 19.0.0-2884-g218a6914 a4e4811815e4 b39c9f192169 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.h smithi083 running (2m) 72s ago 2m 15.6M - 19.0.0-2884-g218a6914 a4e4811815e4 62c3fc99265c 2024-04-13T05:17:11.915 INFO:teuthology.orchestra.run.smithi057.stdout:mds.i smithi189 running (2m) 73s ago 2m 14.5M - 19.0.0-2884-g218a6914 a4e4811815e4 30c89f050b75 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mds.j smithi057 running (3m) 27s ago 3m 12.7M - 19.0.0-2884-g218a6914 a4e4811815e4 a7ae329c7c74 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mds.k smithi083 running (3m) 72s ago 3m 16.1M - 19.0.0-2884-g218a6914 a4e4811815e4 313a3904677d 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mds.l smithi189 running (3m) 73s ago 3m 12.5M - 19.0.0-2884-g218a6914 a4e4811815e4 1775acccbbd0 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi057 *:9283,8765 running (13m) 27s ago 13m 516M - 19.0.0-2884-g218a6914 a4e4811815e4 86589998f3f4 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi083 *:8443,8765 running (10m) 72s ago 10m 452M - 19.0.0-2884-g218a6914 a4e4811815e4 7e61f2ee308f 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.z smithi189 *:8443,8765 running (10m) 73s ago 10m 451M - 19.0.0-2884-g218a6914 a4e4811815e4 093273afa422 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (13m) 27s ago 13m 50.9M 2048M 19.0.0-2884-g218a6914 a4e4811815e4 2e6ff8f4edab 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi083 running (10m) 72s ago 10m 40.0M 2048M 19.0.0-2884-g218a6914 a4e4811815e4 6ac9349280a8 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi189 running (10m) 73s ago 10m 42.4M 2048M 19.0.0-2884-g218a6914 a4e4811815e4 c9c076e507de 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (9m) 27s ago 9m 52.5M 4273M 19.0.0-2884-g218a6914 a4e4811815e4 6823cb58d3a2 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi083 running (9m) 72s ago 9m 52.4M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 802ae1273d88 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi189 running (8m) 73s ago 8m 53.3M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 574b77fe01ba 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (8m) 27s ago 8m 43.8M 4273M 19.0.0-2884-g218a6914 a4e4811815e4 1e7e9c8ffd6d 2024-04-13T05:17:11.916 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi083 running (7m) 72s ago 7m 41.8M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 8e2b7b150434 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi189 running (7m) 73s ago 7m 53.6M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 04ba64ebf40e 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi057 running (6m) 27s ago 6m 41.2M 4273M 19.0.0-2884-g218a6914 a4e4811815e4 c6b1c2225113 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi083 running (5m) 72s ago 5m 43.1M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 1a5b1b0587bb 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 smithi189 running (5m) 73s ago 5m 42.6M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 82332f64c71e 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 smithi057 running (4m) 27s ago 4m 42.3M 4273M 19.0.0-2884-g218a6914 a4e4811815e4 7e1bc1d72448 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 smithi083 running (4m) 72s ago 4m 49.3M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 2ed01318733b 2024-04-13T05:17:11.917 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 smithi189 running (3m) 73s ago 3m 48.9M 4272M 19.0.0-2884-g218a6914 a4e4811815e4 bab63ebb3039 2024-04-13T05:17:11.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0fd00a3c40 msgr2=0x7f0fd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0fd00a3c40 0x7f0fd00a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4005950 tx=0x7f0fd40058e0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 msgr2=0x7f0fe819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 0x7f0fe819f1c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00d9b0 tx=0x7f0fdc00dd70 comp rx=0 tx=0).stop 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 shutdown_connections 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0fd00a3c40 0x7f0fd00a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810acf0 0x7f0fe81a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0fe8109150 0x7f0fe819f1c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 --2- 172.21.15.57:0/3757455263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fe810a350 0x7f0fe819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.916+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 >> 172.21.15.57:0/3757455263 conn(0x7f0fe8078120 msgr2=0x7f0fe810d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.917+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 shutdown_connections 2024-04-13T05:17:11.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:11.917+0000 7f0fefe00700 1 -- 172.21.15.57:0/3757455263 wait complete. 2024-04-13T05:17:12.902 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-13T05:17:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:12 smithi083 ceph-mon[35133]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:12 smithi083 ceph-mon[35133]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:13.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:12 smithi057 ceph-mon[32331]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:13.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:12 smithi057 ceph-mon[32331]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:12 smithi189 ceph-mon[35531]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:12 smithi189 ceph-mon[35531]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:13.402 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:15.208 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:14 smithi057 ceph-mon[32331]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:14 smithi083 ceph-mon[35133]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:15.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:14 smithi189 ceph-mon[35531]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:15.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 -- 172.21.15.57:0/56029635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba48104d20 msgr2=0x7fba48105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 --2- 172.21.15.57:0/56029635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba48104d20 0x7fba48105140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fba380099c0 tx=0x7fba38009cd0 comp rx=0 tx=0).stop 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 -- 172.21.15.57:0/56029635 shutdown_connections 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 --2- 172.21.15.57:0/56029635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba481068c0 0x7fba4810d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 --2- 172.21.15.57:0/56029635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba48104d20 0x7fba48105140 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 --2- 172.21.15.57:0/56029635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba48105f20 0x7fba48106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 -- 172.21.15.57:0/56029635 >> 172.21.15.57:0/56029635 conn(0x7fba481002c0 msgr2=0x7fba48102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 -- 172.21.15.57:0/56029635 shutdown_connections 2024-04-13T05:17:15.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.771+0000 7fba4e200700 1 -- 172.21.15.57:0/56029635 wait complete. 2024-04-13T05:17:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.772+0000 7fba4e200700 1 Processor -- start 2024-04-13T05:17:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 -- start start 2024-04-13T05:17:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba48104d20 0x7fba4819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:15.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba48105f20 0x7fba4819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:15.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 0x7fba481a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:15.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fba481107e0 con 0x7fba481068c0 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba48110660 con 0x7fba48104d20 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba4e200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba48110960 con 0x7fba48105f20 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.773+0000 7fba47e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 0x7fba481a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 0x7fba481a2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36970/0 (socket says 172.21.15.57:36970) 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 -- 172.21.15.57:0/3974115830 learned_addr learned my addr 172.21.15.57:0/3974115830 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47400700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba48104d20 0x7fba4819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba3ea00700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba48105f20 0x7fba4819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 -- 172.21.15.57:0/3974115830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba48105f20 msgr2=0x7fba4819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba48105f20 0x7fba4819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 -- 172.21.15.57:0/3974115830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba48104d20 msgr2=0x7fba4819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba48104d20 0x7fba4819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba47e00700 1 -- 172.21.15.57:0/3974115830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba481a2b10 con 0x7fba481068c0 2024-04-13T05:17:15.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.774+0000 7fba3ea00700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba48105f20 0x7fba4819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.775+0000 7fba47e00700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 0x7fba481a2470 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fba4000add0 tx=0x7fba4000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.775+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40017020 con 0x7fba481068c0 2024-04-13T05:17:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.775+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba4000ae00 con 0x7fba481068c0 2024-04-13T05:17:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.775+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba481a2e00 con 0x7fba481068c0 2024-04-13T05:17:15.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.775+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40005350 con 0x7fba481068c0 2024-04-13T05:17:15.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.775+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba481a3260 con 0x7fba481068c0 2024-04-13T05:17:15.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.776+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba480697a0 con 0x7fba481068c0 2024-04-13T05:17:15.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.778+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fba4000c940 con 0x7fba481068c0 2024-04-13T05:17:15.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.780+0000 7fba44c00700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fba2c0a3c40 0x7fba2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:15.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.780+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fba400cffa0 con 0x7fba481068c0 2024-04-13T05:17:15.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.780+0000 7fba47400700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fba2c0a3c40 0x7fba2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:15.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.781+0000 7fba47400700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fba2c0a3c40 0x7fba2c0a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba38000c00 tx=0x7fba38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:15.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:15.785+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba40021020 con 0x7fba481068c0 2024-04-13T05:17:16.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.027+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fba48062ef0 con 0x7fba2c0a3c40 2024-04-13T05:17:16.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.045+0000 7fba44c00700 1 -- 172.21.15.57:0/3974115830 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fba48062ef0 con 0x7fba2c0a3c40 2024-04-13T05:17:16.046 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-13T05:17:16.046 INFO:teuthology.orchestra.run.smithi057.stdout:mds.all 12/12 77s ago 3m smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi083=b;smithi083=e;smithi083=h;smithi083=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-13T05:17:16.046 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 3/3 77s ago 10m smithi057=x;smithi083=y;smithi189=z;count:3 2024-04-13T05:17:16.047 INFO:teuthology.orchestra.run.smithi057.stdout:mon 3/3 77s ago 11m smithi057:172.21.15.57=a;smithi083:172.21.15.83=b;smithi189:172.21.15.189=c;count:3 2024-04-13T05:17:16.047 INFO:teuthology.orchestra.run.smithi057.stdout:osd 12 77s ago - 2024-04-13T05:17:16.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fba2c0a3c40 msgr2=0x7fba2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:16.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fba2c0a3c40 0x7fba2c0a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba38000c00 tx=0x7fba38005fb0 comp rx=0 tx=0).stop 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 msgr2=0x7fba481a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 0x7fba481a2470 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fba4000add0 tx=0x7fba4000c330 comp rx=0 tx=0).stop 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 shutdown_connections 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fba2c0a3c40 0x7fba2c0a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba481068c0 0x7fba481a2470 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba48104d20 0x7fba4819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 --2- 172.21.15.57:0/3974115830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba48105f20 0x7fba4819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 >> 172.21.15.57:0/3974115830 conn(0x7fba481002c0 msgr2=0x7fba481092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 shutdown_connections 2024-04-13T05:17:16.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:16.048+0000 7fba4e200700 1 -- 172.21.15.57:0/3974115830 wait complete. 2024-04-13T05:17:17.002 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-13T05:17:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:16 smithi083 ceph-mon[35133]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:16 smithi083 ceph-mon[35133]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:17.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:16 smithi057 ceph-mon[32331]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:17.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:16 smithi057 ceph-mon[32331]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:17.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:16 smithi189 ceph-mon[35531]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:17.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:16 smithi189 ceph-mon[35531]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:17.483 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:19.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:18 smithi057 ceph-mon[32331]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:19.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:18 smithi083 ceph-mon[35133]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:19.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:18 smithi189 ceph-mon[35531]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 -- 172.21.15.57:0/29776607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638104d50 msgr2=0x7f8638105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 --2- 172.21.15.57:0/29776607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638104d50 0x7f8638105170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f862c0099c0 tx=0x7f862c009cd0 comp rx=0 tx=0).stop 2024-04-13T05:17:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 -- 172.21.15.57:0/29776607 shutdown_connections 2024-04-13T05:17:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 --2- 172.21.15.57:0/29776607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86381068f0 0x7f863810d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 --2- 172.21.15.57:0/29776607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638104d50 0x7f8638105170 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 --2- 172.21.15.57:0/29776607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8638105f50 0x7f86381063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 -- 172.21.15.57:0/29776607 >> 172.21.15.57:0/29776607 conn(0x7f8638100310 msgr2=0x7f8638102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.837+0000 7f863f800700 1 -- 172.21.15.57:0/29776607 shutdown_connections 2024-04-13T05:17:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.838+0000 7f863f800700 1 -- 172.21.15.57:0/29776607 wait complete. 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 Processor -- start 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 -- start start 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638104d50 0x7f863819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 0x7f863819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86381068f0 0x7f86381a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f86381107e0 con 0x7f8638105f50 2024-04-13T05:17:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8638110660 con 0x7f86381068f0 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.839+0000 7f863f800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8638110960 con 0x7f8638104d50 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 0x7f863819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 0x7f863819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36982/0 (socket says 172.21.15.57:36982) 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 -- 172.21.15.57:0/1987843972 learned_addr learned my addr 172.21.15.57:0/1987843972 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f863ce00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638104d50 0x7f863819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f863d800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86381068f0 0x7f86381a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 -- 172.21.15.57:0/1987843972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638104d50 msgr2=0x7f863819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638104d50 0x7f863819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 -- 172.21.15.57:0/1987843972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86381068f0 msgr2=0x7f86381a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86381068f0 0x7f86381a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f8637e00700 1 -- 172.21.15.57:0/1987843972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381a2a60 con 0x7f8638105f50 2024-04-13T05:17:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.840+0000 7f863ce00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638104d50 0x7f863819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.841+0000 7f8637e00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 0x7f863819b340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f862400d980 tx=0x7f862400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.841+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624016070 con 0x7f8638105f50 2024-04-13T05:17:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.841+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8624010460 con 0x7f8638105f50 2024-04-13T05:17:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.841+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381a2d50 con 0x7f8638105f50 2024-04-13T05:17:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.841+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400f620 con 0x7f8638105f50 2024-04-13T05:17:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.841+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381a3250 con 0x7f8638105f50 2024-04-13T05:17:19.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.843+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863802dfa0 con 0x7f8638105f50 2024-04-13T05:17:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.845+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8624019070 con 0x7f8638105f50 2024-04-13T05:17:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.846+0000 7f8635600700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86200a3cc0 0x7f86200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.846+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f86240d06e0 con 0x7f8638105f50 2024-04-13T05:17:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.846+0000 7f863ce00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86200a3cc0 0x7f86200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:19.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.847+0000 7f863ce00700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86200a3cc0 0x7f86200a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f862c000c00 tx=0x7f862c003890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:19.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:19.851+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f862409ccc0 con 0x7f8638105f50 2024-04-13T05:17:20.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.095+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8638062ef0 con 0x7f86200a3cc0 2024-04-13T05:17:20.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.098+0000 7f8635600700 1 -- 172.21.15.57:0/1987843972 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8638062ef0 con 0x7f86200a3cc0 2024-04-13T05:17:20.099 INFO:teuthology.orchestra.run.smithi057.stdout:HOST ADDR LABELS STATUS 2024-04-13T05:17:20.099 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 172.21.15.57 2024-04-13T05:17:20.099 INFO:teuthology.orchestra.run.smithi057.stdout:smithi083 172.21.15.83 2024-04-13T05:17:20.099 INFO:teuthology.orchestra.run.smithi057.stdout:smithi189 172.21.15.189 2024-04-13T05:17:20.099 INFO:teuthology.orchestra.run.smithi057.stdout:3 hosts in cluster 2024-04-13T05:17:20.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86200a3cc0 msgr2=0x7f86200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86200a3cc0 0x7f86200a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f862c000c00 tx=0x7f862c003890 comp rx=0 tx=0).stop 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 msgr2=0x7f863819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 0x7f863819b340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f862400d980 tx=0x7f862400dd40 comp rx=0 tx=0).stop 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 shutdown_connections 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638104d50 0x7f863819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f86200a3cc0 0x7f86200a6180 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86381068f0 0x7f86381a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 --2- 172.21.15.57:0/1987843972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8638105f50 0x7f863819b340 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.101+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 >> 172.21.15.57:0/1987843972 conn(0x7f8638100310 msgr2=0x7f86381092d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.102+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 shutdown_connections 2024-04-13T05:17:20.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:20.102+0000 7f863f800700 1 -- 172.21.15.57:0/1987843972 wait complete. 2024-04-13T05:17:21.068 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-13T05:17:21.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:20 smithi083 ceph-mon[35133]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:21.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:20 smithi083 ceph-mon[35133]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:21.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:20 smithi057 ceph-mon[32331]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:21.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:20 smithi057 ceph-mon[32331]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:20 smithi189 ceph-mon[35531]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:20 smithi189 ceph-mon[35531]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:21.545 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:23.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:22 smithi083 ceph-mon[35133]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:23.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:22 smithi057 ceph-mon[32331]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:23.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:22 smithi189 ceph-mon[35531]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 -- 172.21.15.57:0/1005210514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084103d10 msgr2=0x7f1084104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 --2- 172.21.15.57:0/1005210514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084103d10 0x7f1084104170 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10700099c0 tx=0x7f1070009cd0 comp rx=0 tx=0).stop 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 -- 172.21.15.57:0/1005210514 shutdown_connections 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 --2- 172.21.15.57:0/1005210514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10841046b0 0x7f108410af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 --2- 172.21.15.57:0/1005210514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1084102b10 0x7f1084102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 --2- 172.21.15.57:0/1005210514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084103d10 0x7f1084104170 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 -- 172.21.15.57:0/1005210514 >> 172.21.15.57:0/1005210514 conn(0x7f10840fe0b0 msgr2=0x7f10841004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.950+0000 7f108a800700 1 -- 172.21.15.57:0/1005210514 shutdown_connections 2024-04-13T05:17:23.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.951+0000 7f108a800700 1 -- 172.21.15.57:0/1005210514 wait complete. 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 Processor -- start 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 -- start start 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 0x7f1084076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1084103d10 0x7f1084074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10841046b0 0x7f10840753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f108410e600 con 0x7f1084102b10 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f108410e480 con 0x7f10841046b0 2024-04-13T05:17:23.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.952+0000 7f108a800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f108410e780 con 0x7f1084103d10 2024-04-13T05:17:23.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.953+0000 7f1083400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 0x7f1084076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:23.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.953+0000 7f1083400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 0x7f1084076850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37004/0 (socket says 172.21.15.57:37004) 2024-04-13T05:17:23.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.953+0000 7f1083400700 1 -- 172.21.15.57:0/1064963293 learned_addr learned my addr 172.21.15.57:0/1064963293 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:23.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.953+0000 7f1083e00700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10841046b0 0x7f10840753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:23.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.953+0000 7f1082a00700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1084103d10 0x7f1084074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.953+0000 7f1083400700 1 -- 172.21.15.57:0/1064963293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1084103d10 msgr2=0x7f1084074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f1083400700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1084103d10 0x7f1084074ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f1083400700 1 -- 172.21.15.57:0/1064963293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10841046b0 msgr2=0x7f10840753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f1083400700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10841046b0 0x7f10840753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f1083400700 1 -- 172.21.15.57:0/1064963293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1084075c90 con 0x7f1084102b10 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f1083e00700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10841046b0 0x7f10840753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:23.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f1083400700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 0x7f1084076850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f107400d980 tx=0x7f107400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074017030 con 0x7f1084102b10 2024-04-13T05:17:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.954+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10740044d0 con 0x7f1084102b10 2024-04-13T05:17:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.955+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10841a2e30 con 0x7f1084102b10 2024-04-13T05:17:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.955+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10740039e0 con 0x7f1084102b10 2024-04-13T05:17:23.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.955+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a3230 con 0x7f1084102b10 2024-04-13T05:17:23.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.956+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108402dfa0 con 0x7f1084102b10 2024-04-13T05:17:23.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.960+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f107401a070 con 0x7f1084102b10 2024-04-13T05:17:23.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.961+0000 7f107be00700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f10680a3c40 0x7f10680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:23.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.962+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f10740d04c0 con 0x7f1084102b10 2024-04-13T05:17:23.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.962+0000 7f1082a00700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f10680a3c40 0x7f10680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:23.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.962+0000 7f1082a00700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f10680a3c40 0x7f10680a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f10700038f0 tx=0x7f1070005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:23.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:23.964+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f107409caa0 con 0x7f1084102b10 2024-04-13T05:17:24.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.204+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1084062ef0 con 0x7f10680a3c40 2024-04-13T05:17:24.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.208+0000 7f107be00700 1 -- 172.21.15.57:0/1064963293 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1084062ef0 con 0x7f10680a3c40 2024-04-13T05:17:24.209 INFO:teuthology.orchestra.run.smithi057.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-13T05:17:24.209 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE7361049H4P0KGN 3726G No 4m ago Has a FileSystem, LVM detected 2024-04-13T05:17:24.209 INFO:teuthology.orchestra.run.smithi057.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-13T05:17:24.209 INFO:teuthology.orchestra.run.smithi057.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-13T05:17:24.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.211+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f10680a3c40 msgr2=0x7f10680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:24.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.211+0000 7f108a800700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f10680a3c40 0x7f10680a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f10700038f0 tx=0x7f1070005fb0 comp rx=0 tx=0).stop 2024-04-13T05:17:24.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.211+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 msgr2=0x7f1084076850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:24.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.211+0000 7f108a800700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 0x7f1084076850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f107400d980 tx=0x7f107400dc90 comp rx=0 tx=0).stop 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 shutdown_connections 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f10680a3c40 0x7f10680a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10841046b0 0x7f10840753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1084102b10 0x7f1084076850 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 --2- 172.21.15.57:0/1064963293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1084103d10 0x7f1084074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 >> 172.21.15.57:0/1064963293 conn(0x7f10840fe0b0 msgr2=0x7f1084107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 shutdown_connections 2024-04-13T05:17:24.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:24.212+0000 7f108a800700 1 -- 172.21.15.57:0/1064963293 wait complete. 2024-04-13T05:17:25.011 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-13T05:17:25.020 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T05:17:25.020 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-13T05:17:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:24 smithi083 ceph-mon[35133]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:24 smithi083 ceph-mon[35133]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:25.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:24 smithi057 ceph-mon[32331]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:25.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:24 smithi057 ceph-mon[32331]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:25.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:24 smithi189 ceph-mon[35531]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:24 smithi189 ceph-mon[35531]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-13T05:17:25.491 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:26 smithi083 ceph-mon[35133]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:27.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:26 smithi057 ceph-mon[32331]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:27.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:26 smithi189 ceph-mon[35531]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.959+0000 7f2346800700 1 -- 172.21.15.57:0/3692350738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2340102b10 msgr2=0x7f2340102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 --2- 172.21.15.57:0/3692350738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2340102b10 0x7f2340102f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f23300099c0 tx=0x7f2330009cd0 comp rx=0 tx=0).stop 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 -- 172.21.15.57:0/3692350738 shutdown_connections 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 --2- 172.21.15.57:0/3692350738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23401046b0 0x7f234010af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 --2- 172.21.15.57:0/3692350738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2340102b10 0x7f2340102f30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 --2- 172.21.15.57:0/3692350738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2340103d10 0x7f2340104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 -- 172.21.15.57:0/3692350738 >> 172.21.15.57:0/3692350738 conn(0x7f23400fe0b0 msgr2=0x7f23401004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:27.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 -- 172.21.15.57:0/3692350738 shutdown_connections 2024-04-13T05:17:27.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.960+0000 7f2346800700 1 -- 172.21.15.57:0/3692350738 wait complete. 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.961+0000 7f2346800700 1 Processor -- start 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.961+0000 7f2346800700 1 -- start start 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.961+0000 7f2346800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2340102b10 0x7f234019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f2346800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2340103d10 0x7f234019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f2346800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 0x7f23401a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f2346800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f234010e5d0 con 0x7f23401046b0 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f2346800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f234010e450 con 0x7f2340103d10 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f2346800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f234010e750 con 0x7f2340102b10 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f233fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 0x7f23401a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f233fe00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 0x7f23401a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42126/0 (socket says 172.21.15.57:42126) 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f233fe00700 1 -- 172.21.15.57:0/45392282 learned_addr learned my addr 172.21.15.57:0/45392282 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:27.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f233f400700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2340102b10 0x7f234019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:27.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f233ea00700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2340103d10 0x7f234019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:27.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.962+0000 7f233fe00700 1 -- 172.21.15.57:0/45392282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2340102b10 msgr2=0x7f234019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:27.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f233fe00700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2340102b10 0x7f234019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:27.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f233fe00700 1 -- 172.21.15.57:0/45392282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2340103d10 msgr2=0x7f234019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f233fe00700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2340103d10 0x7f234019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f233fe00700 1 -- 172.21.15.57:0/45392282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23401a2a50 con 0x7f23401046b0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f233fe00700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 0x7f23401a23b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f233800c930 tx=0x7f233800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338007ac0 con 0x7f23401046b0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2338007c60 con 0x7f23401046b0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23401a2d40 con 0x7f23401046b0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338018660 con 0x7f23401046b0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.963+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401a31d0 con 0x7f23401046b0 2024-04-13T05:17:27.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.965+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23400697d0 con 0x7f23401046b0 2024-04-13T05:17:27.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.967+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f233800f450 con 0x7f23401046b0 2024-04-13T05:17:27.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.968+0000 7f2337e00700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f23240a3c40 0x7f23240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:27.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.968+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f23380d06d0 con 0x7f23401046b0 2024-04-13T05:17:27.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.968+0000 7f233f400700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f23240a3c40 0x7f23240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:27.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.969+0000 7f233f400700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f23240a3c40 0x7f23240a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2330000c00 tx=0x7f2330005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:27.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:27.971+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23400697d0 con 0x7f23401046b0 2024-04-13T05:17:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.253+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f23401a34b0 con 0x7f23401046b0 2024-04-13T05:17:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.254+0000 7f2337e00700 1 -- 172.21.15.57:0/45392282 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3040 (secure 0 0 0) 0x7f2338098030 con 0x7f23401046b0 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:e24 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 24 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-04-13T05:14:53.072497+0000 2024-04-13T05:17:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-04-13T05:15:22.311895+0000 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 300 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 0 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 3 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1,2 2024-04-13T05:17:28.258 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=34499,1=34457,2=34427} 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3,4] 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 0 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:qdb_cluster leader: 34499 members: 34427,34499,34457 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b{0:34499} state up:active seq 3 addr [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.k{0:34397} state up:standby-replay seq 1 addr [v2:172.21.15.83:6832/617814721,v1:172.21.15.83:6833/617814721] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.h{1:34457} state up:active seq 14 addr [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.i{1:24490} state up:standby-replay seq 1 addr [v2:172.21.15.189:6838/1915044018,v1:172.21.15.189:6839/1915044018] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.e{2:34427} state up:active seq 17 addr [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.259 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c{2:24466} state up:standby-replay seq 1 addr [v2:172.21.15.189:6836/563533422,v1:172.21.15.189:6837/563533422] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.57:6834/378251169,v1:172.21.15.57:6835/378251169] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.57:6836/3282486019,v1:172.21.15.57:6837/3282486019] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.g{-1:14619} state up:standby seq 1 addr [v2:172.21.15.57:6838/59311703,v1:172.21.15.57:6839/59311703] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a{-1:14631} state up:standby seq 1 addr [v2:172.21.15.57:6840/2975868204,v1:172.21.15.57:6841/2975868204] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.189:6832/550433387,v1:172.21.15.189:6833/550433387] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.260 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.f{-1:24451} state up:standby seq 1 addr [v2:172.21.15.189:6834/3836752416,v1:172.21.15.189:6835/3836752416] compat {c=[1],r=[1],i=[1fff]}] 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f23240a3c40 msgr2=0x7f23240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f23240a3c40 0x7f23240a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2330000c00 tx=0x7f2330005fb0 comp rx=0 tx=0).stop 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 msgr2=0x7f23401a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 0x7f23401a23b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f233800c930 tx=0x7f233800cc40 comp rx=0 tx=0).stop 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 shutdown_connections 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2340102b10 0x7f234019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f23240a3c40 0x7f23240a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23401046b0 0x7f23401a23b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 --2- 172.21.15.57:0/45392282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2340103d10 0x7f234019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:28.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 >> 172.21.15.57:0/45392282 conn(0x7f23400fe0b0 msgr2=0x7f2340074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 shutdown_connections 2024-04-13T05:17:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:28.258+0000 7f2346800700 1 -- 172.21.15.57:0/45392282 wait complete. 2024-04-13T05:17:28.262 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:17:29.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:28 smithi057 ceph-mon[32331]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:29.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:28 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/45392282' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-13T05:17:29.244 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-13T05:17:29.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:28 smithi083 ceph-mon[35133]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:29.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:28 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/45392282' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-13T05:17:29.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:28 smithi189 ceph-mon[35531]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:29.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:28 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/45392282' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-13T05:17:29.724 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:30 smithi057 ceph-mon[32331]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:30 smithi083 ceph-mon[35133]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:30 smithi189 ceph-mon[35531]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:32.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.089+0000 7f8ffa000700 1 -- 172.21.15.57:0/3380410689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4103d10 msgr2=0x7f8ff4104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:32.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.089+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3380410689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4103d10 0x7f8ff4104170 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00099c0 tx=0x7f8fe0009cd0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 -- 172.21.15.57:0/3380410689 shutdown_connections 2024-04-13T05:17:32.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3380410689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ff4102b10 0x7f8ff4102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3380410689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff41046b0 0x7f8ff410af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3380410689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4103d10 0x7f8ff4104170 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 -- 172.21.15.57:0/3380410689 >> 172.21.15.57:0/3380410689 conn(0x7f8ff40fe0b0 msgr2=0x7f8ff41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:32.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 -- 172.21.15.57:0/3380410689 shutdown_connections 2024-04-13T05:17:32.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.090+0000 7f8ffa000700 1 -- 172.21.15.57:0/3380410689 wait complete. 2024-04-13T05:17:32.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 Processor -- start 2024-04-13T05:17:32.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 -- start start 2024-04-13T05:17:32.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 0x7f8ff419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:32.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ff4103d10 0x7f8ff419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff41046b0 0x7f8ff41a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410e5d0 con 0x7f8ff4102b10 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410e450 con 0x7f8ff41046b0 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.091+0000 7f8ffa000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff410e750 con 0x7f8ff4103d10 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 0x7f8ff419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 0x7f8ff419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42144/0 (socket says 172.21.15.57:42144) 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 -- 172.21.15.57:0/3701604923 learned_addr learned my addr 172.21.15.57:0/3701604923 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff2a00700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ff4103d10 0x7f8ff419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3e00700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff41046b0 0x7f8ff41a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 -- 172.21.15.57:0/3701604923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ff4103d10 msgr2=0x7f8ff419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ff4103d10 0x7f8ff419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 -- 172.21.15.57:0/3701604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff41046b0 msgr2=0x7f8ff41a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:32.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff41046b0 0x7f8ff41a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.092+0000 7f8ff3400700 1 -- 172.21.15.57:0/3701604923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff41a2b50 con 0x7f8ff4102b10 2024-04-13T05:17:32.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.093+0000 7f8ff3400700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 0x7f8ff419af40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d9b0 tx=0x7f8fe400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:32.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.093+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4017030 con 0x7f8ff4102b10 2024-04-13T05:17:32.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.093+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe40044d0 con 0x7f8ff4102b10 2024-04-13T05:17:32.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.093+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41a2e40 con 0x7f8ff4102b10 2024-04-13T05:17:32.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.093+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40039e0 con 0x7f8ff4102b10 2024-04-13T05:17:32.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.093+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a32a0 con 0x7f8ff4102b10 2024-04-13T05:17:32.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.094+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff40697d0 con 0x7f8ff4102b10 2024-04-13T05:17:32.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.096+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8fe401a070 con 0x7f8ff4102b10 2024-04-13T05:17:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.098+0000 7f8febe00700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.098+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8fe40d04d0 con 0x7f8ff4102b10 2024-04-13T05:17:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.098+0000 7f8ff2a00700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:32.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.099+0000 7f8ff2a00700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00038f0 tx=0x7f8fe0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:32.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.101+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ff40697d0 con 0x7f8ff4102b10 2024-04-13T05:17:32.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.330+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f8ff41a35b0 con 0x7f8ff4102b10 2024-04-13T05:17:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.333+0000 7f8febe00700 1 -- 172.21.15.57:0/3701604923 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5634 (secure 0 0 0) 0x7f8fe409cb30 con 0x7f8ff4102b10 2024-04-13T05:17:32.335 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 94 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:fsid 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-04-13T05:04:01.249890+0000 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-04-13T05:16:14.199430+0000 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:crush_version 27 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:full_ratio 0.95 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:backfillfull_ratio 0.9 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:nearfull_ratio 0.85 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:require_min_compat_client luminous 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:min_compat_client luminous 2024-04-13T05:17:32.336 INFO:teuthology.orchestra.run.smithi057.stdout:require_osd_release squid 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:stretch_mode_enabled false 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:max_osd 12 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6802/3403486452,v1:172.21.15.57:6803/3403486452] [v2:172.21.15.57:6804/3403486452,v1:172.21.15.57:6805/3403486452] exists,up 8f852b23-d1b5-4c86-8946-0a9213323ecd 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/3349778520,v1:172.21.15.83:6801/3349778520] [v2:172.21.15.83:6802/3349778520,v1:172.21.15.83:6803/3349778520] exists,up 3eb9f311-c136-431f-a658-5084aa4c2fc6 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/2908933106,v1:172.21.15.189:6801/2908933106] [v2:172.21.15.189:6802/2908933106,v1:172.21.15.189:6803/2908933106] exists,up 83f5d37c-64d6-4a58-81c2-c0f1b7a492ac 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6810/744892058,v1:172.21.15.57:6811/744892058] [v2:172.21.15.57:6812/744892058,v1:172.21.15.57:6813/744892058] exists,up 1e4840c9-26d6-4827-bca1-2fe9391d51ef 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/4161937818,v1:172.21.15.83:6809/4161937818] [v2:172.21.15.83:6810/4161937818,v1:172.21.15.83:6811/4161937818] exists,up 03b6258a-179f-48c4-acce-0e2929065e44 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 up in weight 1 up_from 38 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/3029406488,v1:172.21.15.189:6809/3029406488] [v2:172.21.15.189:6810/3029406488,v1:172.21.15.189:6811/3029406488] exists,up ad5b3787-2395-43e1-9595-98db29fc5e47 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 up in weight 1 up_from 46 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6818/830219588,v1:172.21.15.57:6819/830219588] [v2:172.21.15.57:6820/830219588,v1:172.21.15.57:6821/830219588] exists,up b7e957fe-81a0-4abe-828b-8c1cb7b14983 2024-04-13T05:17:32.337 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 up in weight 1 up_from 51 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/977203651,v1:172.21.15.83:6817/977203651] [v2:172.21.15.83:6818/977203651,v1:172.21.15.83:6819/977203651] exists,up 14b5cd3c-1c3d-4896-949e-5f77372aac7b 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/2146992789,v1:172.21.15.189:6817/2146992789] [v2:172.21.15.189:6818/2146992789,v1:172.21.15.189:6819/2146992789] exists,up 041dc02b-fe99-4ac2-836c-33c8ac7ccc1e 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 up in weight 1 up_from 67 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6826/772937506,v1:172.21.15.57:6827/772937506] [v2:172.21.15.57:6828/772937506,v1:172.21.15.57:6829/772937506] exists,up f43fea2e-9521-419d-85de-fc3068b68ffd 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6824/3322982382,v1:172.21.15.83:6825/3322982382] [v2:172.21.15.83:6826/3322982382,v1:172.21.15.83:6827/3322982382] exists,up bdef097c-4d96-492e-9d28-1eba9cc759fe 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 up in weight 1 up_from 81 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/368428802,v1:172.21.15.189:6825/368428802] [v2:172.21.15.189:6826/368428802,v1:172.21.15.189:6827/368428802] exists,up 476f1e68-924c-49f6-87b8-bcb1267ff3e2 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.a [2,11] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.10 [2,1] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.1d [8,11] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.1f [8,4] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.e [4,6] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.25 [5,9] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.27 [4,8,3,9] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.36 [4,9] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.6 [9,11] 2024-04-13T05:17:32.338 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.19 [9,8] 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.1b [5,10] 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.1c [9,8] 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/2270359290 expires 2024-04-14T05:05:11.407697+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1023620189 expires 2024-04-14T05:04:34.865891+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/570139496 expires 2024-04-14T05:04:34.865891+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/863867962 expires 2024-04-14T05:05:11.407697+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1502699862 expires 2024-04-14T05:04:34.865891+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/3670387557 expires 2024-04-14T05:05:11.407697+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/570139496 expires 2024-04-14T05:04:34.865891+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1172825336 expires 2024-04-14T05:04:34.865891+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/3670387557 expires 2024-04-14T05:05:11.407697+0000 2024-04-13T05:17:32.339 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/980869677 expires 2024-04-14T05:05:11.407697+0000 2024-04-13T05:17:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8fd80a3cc0 msgr2=0x7f8fd80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00038f0 tx=0x7f8fe0005fb0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 msgr2=0x7f8ff419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 0x7f8ff419af40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d9b0 tx=0x7f8fe400dd70 comp rx=0 tx=0).stop 2024-04-13T05:17:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 shutdown_connections 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff41046b0 0x7f8ff41a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ff4102b10 0x7f8ff419af40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 --2- 172.21.15.57:0/3701604923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ff4103d10 0x7f8ff419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.336+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 >> 172.21.15.57:0/3701604923 conn(0x7f8ff40fe0b0 msgr2=0x7f8ff4074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.337+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 shutdown_connections 2024-04-13T05:17:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:32.337+0000 7f8ffa000700 1 -- 172.21.15.57:0/3701604923 wait complete. 2024-04-13T05:17:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:32 smithi083 ceph-mon[35133]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:32 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3701604923' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-13T05:17:33.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:32 smithi057 ceph-mon[32331]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:33.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:32 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3701604923' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-13T05:17:33.296 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-13T05:17:33.307 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:32 smithi189 ceph-mon[35531]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:32 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3701604923' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-13T05:17:33.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 -- 172.21.15.57:0/3137554873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 msgr2=0x7f1f88101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 --2- 172.21.15.57:0/3137554873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f88101640 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f7002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 -- 172.21.15.57:0/3137554873 shutdown_connections 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 --2- 172.21.15.57:0/3137554873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f88101b80 0x7f1f88102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 --2- 172.21.15.57:0/3137554873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f88101640 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 --2- 172.21.15.57:0/3137554873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 0x7f1f88100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 -- 172.21.15.57:0/3137554873 >> 172.21.15.57:0/3137554873 conn(0x7f1f880fc4a0 msgr2=0x7f1f880fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.545+0000 7f1f8e200640 1 -- 172.21.15.57:0/3137554873 shutdown_connections 2024-04-13T05:17:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.546+0000 7f1f8e200640 1 -- 172.21.15.57:0/3137554873 wait complete. 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 Processor -- start 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 -- start start 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 0x7f1f8819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f8819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f88101b80 0x7f1f881a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8810fd70 con 0x7f1f881011e0 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8810fbf0 con 0x7f1f88100880 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f8e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8810fef0 con 0x7f1f88101b80 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f7fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f8819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f7fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f8819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42158/0 (socket says 172.21.15.57:42158) 2024-04-13T05:17:33.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.547+0000 7f1f7fe00640 1 -- 172.21.15.57:0/2446600310 learned_addr learned my addr 172.21.15.57:0/2446600310 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f87400640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 0x7f1f8819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f7fe00640 1 -- 172.21.15.57:0/2446600310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f88101b80 msgr2=0x7f1f881a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f7fe00640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f88101b80 0x7f1f881a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f7fe00640 1 -- 172.21.15.57:0/2446600310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 msgr2=0x7f1f8819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f7fe00640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 0x7f1f8819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f7fe00640 1 -- 172.21.15.57:0/2446600310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f881a2040 con 0x7f1f881011e0 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f87400640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 0x7f1f8819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:33.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.548+0000 7f1f7fe00640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f8819a960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:33.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.549+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70043020 con 0x7f1f881011e0 2024-04-13T05:17:33.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.549+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f700054c0 con 0x7f1f881011e0 2024-04-13T05:17:33.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.549+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f881a22d0 con 0x7f1f881011e0 2024-04-13T05:17:33.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.549+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70003d40 con 0x7f1f881011e0 2024-04-13T05:17:33.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.549+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f881a2820 con 0x7f1f881011e0 2024-04-13T05:17:33.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.550+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8810b7d0 con 0x7f1f881011e0 2024-04-13T05:17:33.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.552+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1f70046020 con 0x7f1f881011e0 2024-04-13T05:17:33.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.554+0000 7f1f84c00640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:33.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.554+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1f700f2c50 con 0x7f1f881011e0 2024-04-13T05:17:33.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.554+0000 7f1f87400640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:33.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.555+0000 7f1f87400640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009b40 tx=0x7f1f78005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:33.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.558+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f7004d020 con 0x7f1f881011e0 2024-04-13T05:17:33.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.773+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f88062c60 con 0x7f1f881011e0 2024-04-13T05:17:33.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.778+0000 7f1f84c00640 1 -- 172.21.15.57:0/2446600310 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f1f700bf230 con 0x7f1f881011e0 2024-04-13T05:17:33.780 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:33.780 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:33.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f6c0a3bb0 msgr2=0x7f1f6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:33.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009b40 tx=0x7f1f78005e50 comp rx=0 tx=0).stop 2024-04-13T05:17:33.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 msgr2=0x7f1f8819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:33.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f8819a960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f700042f0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 shutdown_connections 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f88101b80 0x7f1f881a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f881011e0 0x7f1f8819a960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 --2- 172.21.15.57:0/2446600310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f88100880 0x7f1f8819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 >> 172.21.15.57:0/2446600310 conn(0x7f1f880fc4a0 msgr2=0x7f1f880fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.781+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 shutdown_connections 2024-04-13T05:17:33.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:33.782+0000 7f1f8e200640 1 -- 172.21.15.57:0/2446600310 wait complete. 2024-04-13T05:17:33.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:17:33.804 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:17:34.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 -- 172.21.15.57:0/2630732893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e541029a0 msgr2=0x7f7e54104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 --2- 172.21.15.57:0/2630732893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e541029a0 0x7f7e54104d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009900 tx=0x7f7e4402bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 -- 172.21.15.57:0/2630732893 shutdown_connections 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 --2- 172.21.15.57:0/2630732893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e54107c00 0x7f7e5410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 --2- 172.21.15.57:0/2630732893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541052d0 0x7f7e541076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 --2- 172.21.15.57:0/2630732893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e541029a0 0x7f7e54104d90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 -- 172.21.15.57:0/2630732893 >> 172.21.15.57:0/2630732893 conn(0x7f7e540fc390 msgr2=0x7f7e540fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 -- 172.21.15.57:0/2630732893 shutdown_connections 2024-04-13T05:17:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.036+0000 7f7e59a00640 1 -- 172.21.15.57:0/2630732893 wait complete. 2024-04-13T05:17:34.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.037+0000 7f7e59a00640 1 Processor -- start 2024-04-13T05:17:34.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.037+0000 7f7e59a00640 1 -- start start 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541029a0 0x7f7e54198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e541052d0 0x7f7e54198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 0x7f7e5419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410da40 con 0x7f7e54107c00 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410d8c0 con 0x7f7e541052d0 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410dbc0 con 0x7f7e541029a0 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 0x7f7e5419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 0x7f7e5419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42174/0 (socket says 172.21.15.57:42174) 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e59000640 1 -- 172.21.15.57:0/4203744149 learned_addr learned my addr 172.21.15.57:0/4203744149 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e53e00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541029a0 0x7f7e54198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.038+0000 7f7e53400640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e541052d0 0x7f7e54198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59000640 1 -- 172.21.15.57:0/4203744149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541029a0 msgr2=0x7f7e54198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59000640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541029a0 0x7f7e54198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59000640 1 -- 172.21.15.57:0/4203744149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e541052d0 msgr2=0x7f7e54198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59000640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e541052d0 0x7f7e54198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59000640 1 -- 172.21.15.57:0/4203744149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5419fd20 con 0x7f7e54107c00 2024-04-13T05:17:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e53e00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541029a0 0x7f7e54198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59000640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 0x7f7e5419f6b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00cce0 tx=0x7f7e4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00f450 con 0x7f7e54107c00 2024-04-13T05:17:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e4c00faa0 con 0x7f7e54107c00 2024-04-13T05:17:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c018b70 con 0x7f7e54107c00 2024-04-13T05:17:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.039+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5419ffb0 con 0x7f7e54107c00 2024-04-13T05:17:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.040+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e541a04d0 con 0x7f7e54107c00 2024-04-13T05:17:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.041+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e540fde80 con 0x7f7e54107c00 2024-04-13T05:17:34.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.043+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e4c018d10 con 0x7f7e54107c00 2024-04-13T05:17:34.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.044+0000 7f7e50c00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7e380a3c30 0x7f7e380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.045+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7e4c0d0b90 con 0x7f7e54107c00 2024-04-13T05:17:34.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.045+0000 7f7e53e00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7e380a3c30 0x7f7e380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.046+0000 7f7e53e00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7e380a3c30 0x7f7e380a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000c00 tx=0x7f7e440095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:34.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.049+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e4c014030 con 0x7f7e54107c00 2024-04-13T05:17:34.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.253+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e54062bc0 con 0x7f7e54107c00 2024-04-13T05:17:34.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.254+0000 7f7e50c00640 1 -- 172.21.15.57:0/4203744149 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f7e4c09d170 con 0x7f7e54107c00 2024-04-13T05:17:34.256 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7e380a3c30 msgr2=0x7f7e380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7e380a3c30 0x7f7e380a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000c00 tx=0x7f7e440095f0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 msgr2=0x7f7e5419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 0x7f7e5419f6b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00cce0 tx=0x7f7e4c00a620 comp rx=0 tx=0).stop 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 shutdown_connections 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7e380a3c30 0x7f7e380a60f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e541052d0 0x7f7e54198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e54107c00 0x7f7e5419f6b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 --2- 172.21.15.57:0/4203744149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e541029a0 0x7f7e54198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 >> 172.21.15.57:0/4203744149 conn(0x7f7e540fc390 msgr2=0x7f7e541032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 shutdown_connections 2024-04-13T05:17:34.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.257+0000 7f7e59a00640 1 -- 172.21.15.57:0/4203744149 wait complete. 2024-04-13T05:17:34.276 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:34 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2446600310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:34.280 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:17:34.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:34 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2446600310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:34.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:34 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2446600310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:34.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.512+0000 7f9c9c000640 1 -- 172.21.15.57:0/3442084443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94105dd0 msgr2=0x7f9c9410c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.512+0000 7f9c9c000640 1 --2- 172.21.15.57:0/3442084443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94105dd0 0x7f9c9410c660 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009ca0 tx=0x7f9c8c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 -- 172.21.15.57:0/3442084443 shutdown_connections 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 --2- 172.21.15.57:0/3442084443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94105dd0 0x7f9c9410c660 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 --2- 172.21.15.57:0/3442084443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94105410 0x7f9c94105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 --2- 172.21.15.57:0/3442084443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c94104220 0x7f9c94104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 -- 172.21.15.57:0/3442084443 >> 172.21.15.57:0/3442084443 conn(0x7f9c940ff7b0 msgr2=0x7f9c94101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 -- 172.21.15.57:0/3442084443 shutdown_connections 2024-04-13T05:17:34.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.513+0000 7f9c9c000640 1 -- 172.21.15.57:0/3442084443 wait complete. 2024-04-13T05:17:34.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 Processor -- start 2024-04-13T05:17:34.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 -- start start 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 0x7f9c9419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c94105410 0x7f9c9419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94105dd0 0x7f9c941a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9410fd60 con 0x7f9c94104220 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9410fbe0 con 0x7f9c94105410 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.514+0000 7f9c9c000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9410fee0 con 0x7f9c94105dd0 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 0x7f9c9419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 0x7f9c9419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42196/0 (socket says 172.21.15.57:42196) 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 -- 172.21.15.57:0/2762285085 learned_addr learned my addr 172.21.15.57:0/2762285085 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c98c00640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c94105410 0x7f9c9419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c9a000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94105dd0 0x7f9c941a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 -- 172.21.15.57:0/2762285085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94105dd0 msgr2=0x7f9c941a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94105dd0 0x7f9c941a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 -- 172.21.15.57:0/2762285085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c94105410 msgr2=0x7f9c9419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c94105410 0x7f9c9419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.515+0000 7f9c99600640 1 -- 172.21.15.57:0/2762285085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a2030 con 0x7f9c94104220 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.516+0000 7f9c99600640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 0x7f9c9419a410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d770 tx=0x7f9c8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.516+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017070 con 0x7f9c94104220 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.516+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c88009c30 con 0x7f9c94104220 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.516+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88003d00 con 0x7f9c94104220 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.516+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a2320 con 0x7f9c94104220 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.516+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a28c0 con 0x7f9c94104220 2024-04-13T05:17:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.517+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9410b560 con 0x7f9c94104220 2024-04-13T05:17:34.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.521+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9c88003ea0 con 0x7f9c94104220 2024-04-13T05:17:34.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.522+0000 7f9c92000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9c7c0a3c30 0x7f9c7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.522+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9c880d0560 con 0x7f9c94104220 2024-04-13T05:17:34.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.523+0000 7f9c98c00640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9c7c0a3c30 0x7f9c7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.523+0000 7f9c98c00640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9c7c0a3c30 0x7f9c7c0a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c80005fd0 tx=0x7f9c80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.525+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c8809cbc0 con 0x7f9c94104220 2024-04-13T05:17:34.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.736+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c94062bf0 con 0x7f9c94104220 2024-04-13T05:17:34.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.737+0000 7f9c92000640 1 -- 172.21.15.57:0/2762285085 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f9c88009de0 con 0x7f9c94104220 2024-04-13T05:17:34.739 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9c7c0a3c30 msgr2=0x7f9c7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9c7c0a3c30 0x7f9c7c0a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c80005fd0 tx=0x7f9c80005850 comp rx=0 tx=0).stop 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 msgr2=0x7f9c9419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 0x7f9c9419a410 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d770 tx=0x7f9c8800dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 shutdown_connections 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9c7c0a3c30 0x7f9c7c0a60f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c94105410 0x7f9c9419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94105dd0 0x7f9c941a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 --2- 172.21.15.57:0/2762285085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c94104220 0x7f9c9419a410 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 >> 172.21.15.57:0/2762285085 conn(0x7f9c940ff7b0 msgr2=0x7f9c94108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 shutdown_connections 2024-04-13T05:17:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.740+0000 7f9c9c000640 1 -- 172.21.15.57:0/2762285085 wait complete. 2024-04-13T05:17:34.761 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 -- 172.21.15.57:0/403925120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d810a0b0 msgr2=0x7f46d8110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 --2- 172.21.15.57:0/403925120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d810a0b0 0x7f46d8110940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 -- 172.21.15.57:0/403925120 shutdown_connections 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 --2- 172.21.15.57:0/403925120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d81096f0 0x7f46d8109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 --2- 172.21.15.57:0/403925120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d810a0b0 0x7f46d8110940 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 --2- 172.21.15.57:0/403925120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d8108500 0x7f46d8108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 -- 172.21.15.57:0/403925120 >> 172.21.15.57:0/403925120 conn(0x7f46d8077490 msgr2=0x7f46d80798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 -- 172.21.15.57:0/403925120 shutdown_connections 2024-04-13T05:17:34.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.985+0000 7f46dcc00640 1 -- 172.21.15.57:0/403925120 wait complete. 2024-04-13T05:17:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 Processor -- start 2024-04-13T05:17:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 -- start start 2024-04-13T05:17:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d8108500 0x7f46d819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 0x7f46d819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d810a0b0 0x7f46d81a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8114040 con 0x7f46d81096f0 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8113ec0 con 0x7f46d810a0b0 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.986+0000 7f46dcc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81141c0 con 0x7f46d8108500 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 0x7f46d819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 0x7f46d819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44652/0 (socket says 172.21.15.57:44652) 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 -- 172.21.15.57:0/1850686272 learned_addr learned my addr 172.21.15.57:0/1850686272 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46d7400640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d8108500 0x7f46d819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 -- 172.21.15.57:0/1850686272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d8108500 msgr2=0x7f46d819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d8108500 0x7f46d819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 -- 172.21.15.57:0/1850686272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d810a0b0 msgr2=0x7f46d81a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d810a0b0 0x7f46d81a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.987+0000 7f46cea00640 1 -- 172.21.15.57:0/1850686272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d81a62b0 con 0x7f46d81096f0 2024-04-13T05:17:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.988+0000 7f46cea00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 0x7f46d819ebd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f46c000d950 tx=0x7f46c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.988+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0017070 con 0x7f46d81096f0 2024-04-13T05:17:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.988+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c0009c30 con 0x7f46d81096f0 2024-04-13T05:17:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.988+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c000f620 con 0x7f46d81096f0 2024-04-13T05:17:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.988+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d81a65a0 con 0x7f46d81096f0 2024-04-13T05:17:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.988+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d81a6aa0 con 0x7f46d81096f0 2024-04-13T05:17:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.989+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d810f820 con 0x7f46d81096f0 2024-04-13T05:17:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.992+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f46c00109a0 con 0x7f46d81096f0 2024-04-13T05:17:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.993+0000 7f46d4c00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46bc0a3c30 0x7f46bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.994+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f46c00d0400 con 0x7f46d81096f0 2024-04-13T05:17:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.994+0000 7f46d7400640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46bc0a3c30 0x7f46bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.995+0000 7f46d7400640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46bc0a3c30 0x7f46bc0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f46c8005fd0 tx=0x7f46c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:34.998+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46c0009de0 con 0x7f46d81096f0 2024-04-13T05:17:35.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.210+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46d8062c60 con 0x7f46d81096f0 2024-04-13T05:17:35.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.211+0000 7f46d4c00640 1 -- 172.21.15.57:0/1850686272 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f46c009ca40 con 0x7f46d81096f0 2024-04-13T05:17:35.213 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:17:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46bc0a3c30 msgr2=0x7f46bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46bc0a3c30 0x7f46bc0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f46c8005fd0 tx=0x7f46c8005850 comp rx=0 tx=0).stop 2024-04-13T05:17:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 msgr2=0x7f46d819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 0x7f46d819ebd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f46c000d950 tx=0x7f46c000de10 comp rx=0 tx=0).stop 2024-04-13T05:17:35.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 shutdown_connections 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46bc0a3c30 0x7f46bc0a60f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46d810a0b0 0x7f46d81a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46d81096f0 0x7f46d819ebd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 --2- 172.21.15.57:0/1850686272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d8108500 0x7f46d819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.214+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 >> 172.21.15.57:0/1850686272 conn(0x7f46d8077490 msgr2=0x7f46d810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.215+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 shutdown_connections 2024-04-13T05:17:35.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.215+0000 7f46dcc00640 1 -- 172.21.15.57:0/1850686272 wait complete. 2024-04-13T05:17:35.237 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:17:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:35 smithi057 ceph-mon[32331]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:35 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4203744149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:35 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2762285085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:35.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:35 smithi189 ceph-mon[35531]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:35.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:35 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4203744149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:35.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:35 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2762285085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 -- 172.21.15.57:0/3864803959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4107bb0 msgr2=0x7fc7d410a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/3864803959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4107bb0 0x7fc7d410a000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009900 tx=0x7fc7cc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 -- 172.21.15.57:0/3864803959 shutdown_connections 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/3864803959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d4105280 0x7fc7d4107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/3864803959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4107bb0 0x7fc7d410a000 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/3864803959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7d4102950 0x7fc7d4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 -- 172.21.15.57:0/3864803959 >> 172.21.15.57:0/3864803959 conn(0x7fc7d40fc360 msgr2=0x7fc7d40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.460+0000 7fc7d8c00640 1 -- 172.21.15.57:0/3864803959 shutdown_connections 2024-04-13T05:17:35.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.461+0000 7fc7d8c00640 1 -- 172.21.15.57:0/3864803959 wait complete. 2024-04-13T05:17:35.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.461+0000 7fc7d8c00640 1 Processor -- start 2024-04-13T05:17:35.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 -- start start 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7d4102950 0x7fc7d419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 0x7fc7d419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d4107bb0 0x7fc7d41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d410da60 con 0x7fc7d4105280 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d410d8e0 con 0x7fc7d4107bb0 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d410dbe0 con 0x7fc7d4102950 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 0x7fc7d419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 0x7fc7d419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44670/0 (socket says 172.21.15.57:44670) 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d2a00640 1 -- 172.21.15.57:0/2110627838 learned_addr learned my addr 172.21.15.57:0/2110627838 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.462+0000 7fc7d3400640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7d4102950 0x7fc7d419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d3e00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d4107bb0 0x7fc7d41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d2a00640 1 -- 172.21.15.57:0/2110627838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7d4102950 msgr2=0x7fc7d419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d2a00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7d4102950 0x7fc7d419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d2a00640 1 -- 172.21.15.57:0/2110627838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d4107bb0 msgr2=0x7fc7d41a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d2a00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d4107bb0 0x7fc7d41a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d2a00640 1 -- 172.21.15.57:0/2110627838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d41a1f30 con 0x7fc7d4105280 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d2a00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 0x7fc7d419a850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d770 tx=0x7fc7c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0017070 con 0x7fc7d4105280 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7c0009c30 con 0x7fc7d4105280 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d41a2220 con 0x7fc7d4105280 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.463+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000f620 con 0x7fc7d4105280 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.464+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d41a2770 con 0x7fc7d4105280 2024-04-13T05:17:35.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.465+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d41096a0 con 0x7fc7d4105280 2024-04-13T05:17:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.469+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc7c001a070 con 0x7fc7d4105280 2024-04-13T05:17:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.471+0000 7fc7cbe00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7b80a3bb0 0x7fc7b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.471+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc7c00d0380 con 0x7fc7d4105280 2024-04-13T05:17:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.471+0000 7fc7d3400640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7b80a3bb0 0x7fc7b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.472+0000 7fc7d3400640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7b80a3bb0 0x7fc7b80a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4005fd0 tx=0x7fc7c4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.473+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7c0009de0 con 0x7fc7d4105280 2024-04-13T05:17:35.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:35 smithi083 ceph-mon[35133]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:35.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:35 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4203744149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:35.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:35 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2762285085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:35.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.679+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc7d402d770 con 0x7fc7d4105280 2024-04-13T05:17:35.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.680+0000 7fc7cbe00640 1 -- 172.21.15.57:0/2110627838 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fc7c009c960 con 0x7fc7d4105280 2024-04-13T05:17:35.681 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:35.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7b80a3bb0 msgr2=0x7fc7b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7b80a3bb0 0x7fc7b80a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4005fd0 tx=0x7fc7c4009870 comp rx=0 tx=0).stop 2024-04-13T05:17:35.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 msgr2=0x7fc7d419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 0x7fc7d419a850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d770 tx=0x7fc7c000dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:35.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 shutdown_connections 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7b80a3bb0 0x7fc7b80a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7d4107bb0 0x7fc7d41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7d4105280 0x7fc7d419a850 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 --2- 172.21.15.57:0/2110627838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7d4102950 0x7fc7d419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 >> 172.21.15.57:0/2110627838 conn(0x7fc7d40fc360 msgr2=0x7fc7d4102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 shutdown_connections 2024-04-13T05:17:35.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.683+0000 7fc7d8c00640 1 -- 172.21.15.57:0/2110627838 wait complete. 2024-04-13T05:17:35.706 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 -- 172.21.15.57:0/1227423043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38105410 msgr2=0x7fcd38105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/1227423043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38105410 0x7fcd38105890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009900 tx=0x7fcd2402bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 -- 172.21.15.57:0/1227423043 shutdown_connections 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/1227423043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 0x7fcd3810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/1227423043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38105410 0x7fcd38105890 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/1227423043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd38104220 0x7fcd38104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 -- 172.21.15.57:0/1227423043 >> 172.21.15.57:0/1227423043 conn(0x7fcd380ff7b0 msgr2=0x7fcd38101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 -- 172.21.15.57:0/1227423043 shutdown_connections 2024-04-13T05:17:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.974+0000 7fcd3ca00640 1 -- 172.21.15.57:0/1227423043 wait complete. 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.975+0000 7fcd3ca00640 1 Processor -- start 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.975+0000 7fcd3ca00640 1 -- start start 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd3ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 0x7fcd3819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd3ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd38105410 0x7fcd3819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd3ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 0x7fcd381a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd3ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3810fdb0 con 0x7fcd38104220 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd3ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3810fc30 con 0x7fcd38105410 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd3ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3810ff30 con 0x7fcd38105dd0 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd35e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 0x7fcd3819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd35e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 0x7fcd3819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44678/0 (socket says 172.21.15.57:44678) 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd35e00640 1 -- 172.21.15.57:0/638121800 learned_addr learned my addr 172.21.15.57:0/638121800 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd35400640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd38105410 0x7fcd3819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.976+0000 7fcd36800640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 0x7fcd381a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd35e00640 1 -- 172.21.15.57:0/638121800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 msgr2=0x7fcd381a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd35e00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 0x7fcd381a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd35e00640 1 -- 172.21.15.57:0/638121800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd38105410 msgr2=0x7fcd3819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd35e00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd38105410 0x7fcd3819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd35e00640 1 -- 172.21.15.57:0/638121800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd381a2050 con 0x7fcd38104220 2024-04-13T05:17:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd36800640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 0x7fcd381a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.977+0000 7fcd35e00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 0x7fcd3819a430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d950 tx=0x7fcd2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.978+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28017020 con 0x7fcd38104220 2024-04-13T05:17:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.978+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd381a2340 con 0x7fcd38104220 2024-04-13T05:17:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.978+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd28009c30 con 0x7fcd38104220 2024-04-13T05:17:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.978+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28005220 con 0x7fcd38104220 2024-04-13T05:17:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.978+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd381a2860 con 0x7fcd38104220 2024-04-13T05:17:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.980+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3810b540 con 0x7fcd38104220 2024-04-13T05:17:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.981+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcd2801a020 con 0x7fcd38104220 2024-04-13T05:17:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.983+0000 7fcd2ea00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd1c0a3bb0 0x7fcd1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.983+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcd280d0290 con 0x7fcd38104220 2024-04-13T05:17:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.983+0000 7fcd35400640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd1c0a3bb0 0x7fcd1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.984+0000 7fcd35400640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd1c0a3bb0 0x7fcd1c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd24000c00 tx=0x7fcd2402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:35.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:35.987+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd28021070 con 0x7fcd38104220 2024-04-13T05:17:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.239+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3802d5e0 con 0x7fcd38104220 2024-04-13T05:17:36.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.241+0000 7fcd2ea00640 1 -- 172.21.15.57:0/638121800 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7fcd28009de0 con 0x7fcd38104220 2024-04-13T05:17:36.244 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:36.244 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:15:22.311895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:17:36.244 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:17:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.244+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd1c0a3bb0 msgr2=0x7fcd1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.244+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd1c0a3bb0 0x7fcd1c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd24000c00 tx=0x7fcd2402c7e0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.244+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 msgr2=0x7fcd3819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.244+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 0x7fcd3819a430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d950 tx=0x7fcd2800de10 comp rx=0 tx=0).stop 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 shutdown_connections 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd1c0a3bb0 0x7fcd1c0a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd38105410 0x7fcd3819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd38105dd0 0x7fcd381a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 --2- 172.21.15.57:0/638121800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd38104220 0x7fcd3819a430 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 >> 172.21.15.57:0/638121800 conn(0x7fcd380ff7b0 msgr2=0x7fcd38108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 shutdown_connections 2024-04-13T05:17:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.245+0000 7fcd3ca00640 1 -- 172.21.15.57:0/638121800 wait complete. 2024-04-13T05:17:36.269 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:36.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:36 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1850686272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:36.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:36 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2110627838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:36.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:36 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1850686272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:36.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:36 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2110627838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 -- 172.21.15.57:0/3871123092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441032a0 msgr2=0x7f0344103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 --2- 172.21.15.57:0/3871123092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441032a0 0x7f0344103720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0330009930 tx=0x7f033002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 -- 172.21.15.57:0/3871123092 shutdown_connections 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 --2- 172.21.15.57:0/3871123092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 0x7f034410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 --2- 172.21.15.57:0/3871123092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441032a0 0x7f0344103720 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 --2- 172.21.15.57:0/3871123092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03441002d0 0x7f03441006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 -- 172.21.15.57:0/3871123092 >> 172.21.15.57:0/3871123092 conn(0x7f03440fbc00 msgr2=0x7f03440fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 -- 172.21.15.57:0/3871123092 shutdown_connections 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.482+0000 7f0348a00640 1 -- 172.21.15.57:0/3871123092 wait complete. 2024-04-13T05:17:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 Processor -- start 2024-04-13T05:17:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 -- start start 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 0x7f03441982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03441032a0 0x7f03441987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 0x7f034419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f034410dbc0 con 0x7f03441002d0 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f034410da40 con 0x7f03441032a0 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.484+0000 7f0348a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f034410dd40 con 0x7f0344103c60 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 0x7f03441982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 0x7f03441982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44696/0 (socket says 172.21.15.57:44696) 2024-04-13T05:17:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 -- 172.21.15.57:0/2177736854 learned_addr learned my addr 172.21.15.57:0/2177736854 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341400640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03441032a0 0x7f03441987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0342800640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 0x7f034419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 -- 172.21.15.57:0/2177736854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 msgr2=0x7f034419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 0x7f034419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 -- 172.21.15.57:0/2177736854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03441032a0 msgr2=0x7f03441987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03441032a0 0x7f03441987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 -- 172.21.15.57:0/2177736854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034419fe80 con 0x7f03441002d0 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0342800640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 0x7f034419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.485+0000 7f0341e00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 0x7f03441982b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f033400d770 tx=0x7f033400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.486+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334017070 con 0x7f03441002d0 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.486+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0334009c30 con 0x7f03441002d0 2024-04-13T05:17:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.486+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03441a0170 con 0x7f03441002d0 2024-04-13T05:17:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.486+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334003d00 con 0x7f03441002d0 2024-04-13T05:17:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.486+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03441a0690 con 0x7f03441002d0 2024-04-13T05:17:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.487+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03441094c0 con 0x7f03441002d0 2024-04-13T05:17:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.489+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0334003ea0 con 0x7f03441002d0 2024-04-13T05:17:36.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.491+0000 7f033aa00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03280a3bb0 0x7f03280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:36.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.491+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f03340d05c0 con 0x7f03441002d0 2024-04-13T05:17:36.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.491+0000 7f0341400640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03280a3bb0 0x7f03280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:36.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.492+0000 7f0341400640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03280a3bb0 0x7f03280a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0330009900 tx=0x7f0330005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:36.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.496+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f033409cba0 con 0x7f03441002d0 2024-04-13T05:17:36.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:36 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1850686272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:36.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:36 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2110627838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:36.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.705+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0344062bf0 con 0x7f03441002d0 2024-04-13T05:17:36.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.710+0000 7f033aa00640 1 -- 172.21.15.57:0/2177736854 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f0334009de0 con 0x7f03441002d0 2024-04-13T05:17:36.712 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:36.712 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03280a3bb0 msgr2=0x7f03280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03280a3bb0 0x7f03280a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0330009900 tx=0x7f0330005fd0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 msgr2=0x7f03441982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 0x7f03441982b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f033400d770 tx=0x7f033400dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 shutdown_connections 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03280a3bb0 0x7f03280a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03441032a0 0x7f03441987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0344103c60 0x7f034419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 --2- 172.21.15.57:0/2177736854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03441002d0 0x7f03441982b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 >> 172.21.15.57:0/2177736854 conn(0x7f03440fbc00 msgr2=0x7f0344106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:36.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 shutdown_connections 2024-04-13T05:17:36.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:36.713+0000 7f0348a00640 1 -- 172.21.15.57:0/2177736854 wait complete. 2024-04-13T05:17:36.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:17:36.734 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:17:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 -- 172.21.15.57:0/731712029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 msgr2=0x7f24b01057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 --2- 172.21.15.57:0/731712029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b01057b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f249c009900 tx=0x7f249c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:37.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 -- 172.21.15.57:0/731712029 shutdown_connections 2024-04-13T05:17:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 --2- 172.21.15.57:0/731712029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b0105cf0 0x7f24b010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 --2- 172.21.15.57:0/731712029 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b01057b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 --2- 172.21.15.57:0/731712029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24b0104140 0x7f24b0104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 -- 172.21.15.57:0/731712029 >> 172.21.15.57:0/731712029 conn(0x7f24b00ff6d0 msgr2=0x7f24b0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 -- 172.21.15.57:0/731712029 shutdown_connections 2024-04-13T05:17:37.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.006+0000 7f24b6000640 1 -- 172.21.15.57:0/731712029 wait complete. 2024-04-13T05:17:37.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.007+0000 7f24b6000640 1 Processor -- start 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.007+0000 7f24b6000640 1 -- start start 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24b0104140 0x7f24b019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b0105cf0 0x7f24b01a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b6000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f24b010fcd0 con 0x7f24b0105330 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b6000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24b010fb50 con 0x7f24b0105cf0 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b6000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24b010fe50 con 0x7f24b0104140 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24afe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24afe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44726/0 (socket says 172.21.15.57:44726) 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24afe00640 1 -- 172.21.15.57:0/2442310061 learned_addr learned my addr 172.21.15.57:0/2442310061 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.008+0000 7f24b4c00640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24b0104140 0x7f24b019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24b5600640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b0105cf0 0x7f24b01a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24afe00640 1 -- 172.21.15.57:0/2442310061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24b0104140 msgr2=0x7f24b019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24afe00640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24b0104140 0x7f24b019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24afe00640 1 -- 172.21.15.57:0/2442310061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b0105cf0 msgr2=0x7f24b01a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24afe00640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b0105cf0 0x7f24b01a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24afe00640 1 -- 172.21.15.57:0/2442310061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b01a1f80 con 0x7f24b0105330 2024-04-13T05:17:37.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24afe00640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b019a8a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f249c000c00 tx=0x7f249c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c043020 con 0x7f24b0105330 2024-04-13T05:17:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f249c0054c0 con 0x7f24b0105330 2024-04-13T05:17:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b01a2210 con 0x7f24b0105330 2024-04-13T05:17:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.009+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c003d70 con 0x7f24b0105330 2024-04-13T05:17:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.010+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b01a2760 con 0x7f24b0105330 2024-04-13T05:17:37.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.011+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b010b460 con 0x7f24b0105330 2024-04-13T05:17:37.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.013+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f249c046020 con 0x7f24b0105330 2024-04-13T05:17:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.014+0000 7f24ad600640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f24980a3bb0 0x7f24980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.014+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f249c0f30f0 con 0x7f24b0105330 2024-04-13T05:17:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.015+0000 7f24b4c00640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f24980a3bb0 0x7f24980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.015+0000 7f24b4c00640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f24980a3bb0 0x7f24980a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f24a4005fd0 tx=0x7f24a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.017+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f249c04d020 con 0x7f24b0105330 2024-04-13T05:17:37.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.225+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24b002d770 con 0x7f24b0105330 2024-04-13T05:17:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.226+0000 7f24ad600640 1 -- 172.21.15.57:0/2442310061 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f249c0bf6d0 con 0x7f24b0105330 2024-04-13T05:17:37.228 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f24980a3bb0 msgr2=0x7f24980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f24980a3bb0 0x7f24980a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f24a4005fd0 tx=0x7f24a4005850 comp rx=0 tx=0).stop 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 msgr2=0x7f24b019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b019a8a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f249c000c00 tx=0x7f249c0042f0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 shutdown_connections 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f24980a3bb0 0x7f24980a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b0105cf0 0x7f24b01a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24b0105330 0x7f24b019a8a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 --2- 172.21.15.57:0/2442310061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24b0104140 0x7f24b019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 >> 172.21.15.57:0/2442310061 conn(0x7f24b00ff6d0 msgr2=0x7f24b0108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 shutdown_connections 2024-04-13T05:17:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.229+0000 7f24b6000640 1 -- 172.21.15.57:0/2442310061 wait complete. 2024-04-13T05:17:37.251 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:17:37.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:37 smithi057 ceph-mon[32331]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:37.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:37 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/638121800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:37.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:37 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2177736854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:37 smithi189 ceph-mon[35531]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:37 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/638121800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:37 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2177736854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.474+0000 7f1a92800640 1 -- 172.21.15.57:0/2266041051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c1052f0 msgr2=0x7f1a8c105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.474+0000 7f1a92800640 1 --2- 172.21.15.57:0/2266041051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c1052f0 0x7f1a8c105770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009900 tx=0x7f1a7802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 -- 172.21.15.57:0/2266041051 shutdown_connections 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 --2- 172.21.15.57:0/2266041051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 --2- 172.21.15.57:0/2266041051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c1052f0 0x7f1a8c105770 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 --2- 172.21.15.57:0/2266041051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a8c104100 0x7f1a8c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 -- 172.21.15.57:0/2266041051 >> 172.21.15.57:0/2266041051 conn(0x7f1a8c0ff670 msgr2=0x7f1a8c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 -- 172.21.15.57:0/2266041051 shutdown_connections 2024-04-13T05:17:37.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.475+0000 7f1a92800640 1 -- 172.21.15.57:0/2266041051 wait complete. 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 Processor -- start 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 -- start start 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c104100 0x7f1a8c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a8c1052f0 0x7f1a8c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c10fc90 con 0x7f1a8c105cb0 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c10fb10 con 0x7f1a8c1052f0 2024-04-13T05:17:37.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.476+0000 7f1a92800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c10fe10 con 0x7f1a8c104100 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44748/0 (socket says 172.21.15.57:44748) 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 -- 172.21.15.57:0/11890090 learned_addr learned my addr 172.21.15.57:0/11890090 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a90a00640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a8c1052f0 0x7f1a8c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91400640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c104100 0x7f1a8c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 -- 172.21.15.57:0/11890090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c104100 msgr2=0x7f1a8c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c104100 0x7f1a8c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 -- 172.21.15.57:0/11890090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a8c1052f0 msgr2=0x7f1a8c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a8c1052f0 0x7f1a8c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91e00640 1 -- 172.21.15.57:0/11890090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c1a1f30 con 0x7f1a8c105cb0 2024-04-13T05:17:37.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.477+0000 7f1a91400640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c104100 0x7f1a8c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.478+0000 7f1a91e00640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c1a18c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400eaa0 tx=0x7f1a8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.478+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400cdb0 con 0x7f1a8c105cb0 2024-04-13T05:17:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.478+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a84005020 con 0x7f1a8c105cb0 2024-04-13T05:17:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.478+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c1a2220 con 0x7f1a8c105cb0 2024-04-13T05:17:37.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.478+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840116d0 con 0x7f1a8c105cb0 2024-04-13T05:17:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.478+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c1a2740 con 0x7f1a8c105cb0 2024-04-13T05:17:37.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.480+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c10b4f0 con 0x7f1a8c105cb0 2024-04-13T05:17:37.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.481+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1a840051d0 con 0x7f1a8c105cb0 2024-04-13T05:17:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.483+0000 7f1a8a000640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1a740a3bb0 0x7f1a740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.483+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1a840d0930 con 0x7f1a8c105cb0 2024-04-13T05:17:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.483+0000 7f1a91400640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1a740a3bb0 0x7f1a740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:37.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.484+0000 7f1a91400640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1a740a3bb0 0x7f1a740a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005d00 tx=0x7f1a80005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:37.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.488+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a84016030 con 0x7f1a8c105cb0 2024-04-13T05:17:37.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:37 smithi083 ceph-mon[35133]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:37.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:37 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/638121800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:37.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:37 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2177736854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:37.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.696+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a8c062bc0 con 0x7f1a8c105cb0 2024-04-13T05:17:37.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.697+0000 7f1a8a000640 1 -- 172.21.15.57:0/11890090 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f1a8409cf10 con 0x7f1a8c105cb0 2024-04-13T05:17:37.699 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1a740a3bb0 msgr2=0x7f1a740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1a740a3bb0 0x7f1a740a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005d00 tx=0x7f1a80005c90 comp rx=0 tx=0).stop 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 msgr2=0x7f1a8c1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c1a18c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400eaa0 tx=0x7f1a8400ef60 comp rx=0 tx=0).stop 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 shutdown_connections 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1a740a3bb0 0x7f1a740a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a8c1052f0 0x7f1a8c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a8c105cb0 0x7f1a8c1a18c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 --2- 172.21.15.57:0/11890090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c104100 0x7f1a8c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 >> 172.21.15.57:0/11890090 conn(0x7f1a8c0ff670 msgr2=0x7f1a8c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 shutdown_connections 2024-04-13T05:17:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.700+0000 7f1a92800640 1 -- 172.21.15.57:0/11890090 wait complete. 2024-04-13T05:17:37.724 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.996+0000 7f6c2a000640 1 -- 172.21.15.57:0/1776047183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24107bb0 msgr2=0x7f6c2410a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.996+0000 7f6c2a000640 1 --2- 172.21.15.57:0/1776047183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24107bb0 0x7f6c2410a000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009ca0 tx=0x7f6c1c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 -- 172.21.15.57:0/1776047183 shutdown_connections 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 --2- 172.21.15.57:0/1776047183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24105280 0x7f6c24107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 --2- 172.21.15.57:0/1776047183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24107bb0 0x7f6c2410a000 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 --2- 172.21.15.57:0/1776047183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c24102950 0x7f6c24104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 -- 172.21.15.57:0/1776047183 >> 172.21.15.57:0/1776047183 conn(0x7f6c240fc360 msgr2=0x7f6c240fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 -- 172.21.15.57:0/1776047183 shutdown_connections 2024-04-13T05:17:37.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.997+0000 7f6c2a000640 1 -- 172.21.15.57:0/1776047183 wait complete. 2024-04-13T05:17:38.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 Processor -- start 2024-04-13T05:17:38.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 -- start start 2024-04-13T05:17:38.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c24102950 0x7f6c24198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 0x7f6c241985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24107bb0 0x7f6c2419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2410da10 con 0x7f6c24105280 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2410d890 con 0x7f6c24107bb0 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.998+0000 7f6c2a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2410db90 con 0x7f6c24102950 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 0x7f6c241985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 0x7f6c241985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44758/0 (socket says 172.21.15.57:44758) 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 -- 172.21.15.57:0/3506062545 learned_addr learned my addr 172.21.15.57:0/3506062545 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c28c00640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c24102950 0x7f6c24198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c29600640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24107bb0 0x7f6c2419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 -- 172.21.15.57:0/3506062545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c24102950 msgr2=0x7f6c24198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c24102950 0x7f6c24198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 -- 172.21.15.57:0/3506062545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24107bb0 msgr2=0x7f6c2419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24107bb0 0x7f6c2419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c23e00640 1 -- 172.21.15.57:0/3506062545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2419fcb0 con 0x7f6c24105280 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:37.999+0000 7f6c29600640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24107bb0 0x7f6c2419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:38.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.000+0000 7f6c23e00640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 0x7f6c241985d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000d770 tx=0x7f6c1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:38.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.000+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10017070 con 0x7f6c24105280 2024-04-13T05:17:38.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.000+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c10009c30 con 0x7f6c24105280 2024-04-13T05:17:38.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.000+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2419ffa0 con 0x7f6c24105280 2024-04-13T05:17:38.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.000+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1000f620 con 0x7f6c24105280 2024-04-13T05:17:38.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.000+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c241a0460 con 0x7f6c24105280 2024-04-13T05:17:38.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.001+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c240fde50 con 0x7f6c24105280 2024-04-13T05:17:38.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.004+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c1001a070 con 0x7f6c24105280 2024-04-13T05:17:38.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.006+0000 7f6c21600640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c080a3bb0 0x7f6c080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.006+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6c100d03c0 con 0x7f6c24105280 2024-04-13T05:17:38.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.007+0000 7f6c28c00640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c080a3bb0 0x7f6c080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.007+0000 7f6c28c00640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c080a3bb0 0x7f6c080a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005fd0 tx=0x7f6c14005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:38.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.011+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c10009de0 con 0x7f6c24105280 2024-04-13T05:17:38.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.226+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c24062bc0 con 0x7f6c24105280 2024-04-13T05:17:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.227+0000 7f6c21600640 1 -- 172.21.15.57:0/3506062545 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f6c1009c9a0 con 0x7f6c24105280 2024-04-13T05:17:38.229 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:17:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c080a3bb0 msgr2=0x7f6c080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c080a3bb0 0x7f6c080a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005fd0 tx=0x7f6c14005850 comp rx=0 tx=0).stop 2024-04-13T05:17:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 msgr2=0x7f6c241985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 0x7f6c241985d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000d770 tx=0x7f6c1000dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 shutdown_connections 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c080a3bb0 0x7f6c080a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c24107bb0 0x7f6c2419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24105280 0x7f6c241985d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3506062545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c24102950 0x7f6c24198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 >> 172.21.15.57:0/3506062545 conn(0x7f6c240fc360 msgr2=0x7f6c24103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.230+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 shutdown_connections 2024-04-13T05:17:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.231+0000 7f6c2a000640 1 -- 172.21.15.57:0/3506062545 wait complete. 2024-04-13T05:17:38.253 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:17:38.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:38 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2442310061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:38.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:38 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/11890090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:38.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:38 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2442310061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:38.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:38 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/11890090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:38.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 -- 172.21.15.57:0/2973041406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd41010d0 msgr2=0x7f4bd41014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 --2- 172.21.15.57:0/2973041406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd41010d0 0x7f4bd41014f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009900 tx=0x7f4bc402bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 -- 172.21.15.57:0/2973041406 shutdown_connections 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 --2- 172.21.15.57:0/2973041406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd40ffba0 0x7f4bd4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 --2- 172.21.15.57:0/2973041406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd4100560 0x7f4bd4102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 --2- 172.21.15.57:0/2973041406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd41010d0 0x7f4bd41014f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 -- 172.21.15.57:0/2973041406 >> 172.21.15.57:0/2973041406 conn(0x7f4bd40fb680 msgr2=0x7f4bd40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.491+0000 7f4bd9000640 1 -- 172.21.15.57:0/2973041406 shutdown_connections 2024-04-13T05:17:38.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.492+0000 7f4bd9000640 1 -- 172.21.15.57:0/2973041406 wait complete. 2024-04-13T05:17:38.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.492+0000 7f4bd9000640 1 Processor -- start 2024-04-13T05:17:38.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 -- start start 2024-04-13T05:17:38.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd40ffba0 0x7f4bd419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 0x7f4bd419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd41010d0 0x7f4bd41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410fc80 con 0x7f4bd4100560 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410fb00 con 0x7f4bd41010d0 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd9000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410fe00 con 0x7f4bd40ffba0 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 0x7f4bd419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 0x7f4bd419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44778/0 (socket says 172.21.15.57:44778) 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd2a00640 1 -- 172.21.15.57:0/312941540 learned_addr learned my addr 172.21.15.57:0/312941540 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.493+0000 7f4bd3400640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd40ffba0 0x7f4bd419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd3e00640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd41010d0 0x7f4bd41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd2a00640 1 -- 172.21.15.57:0/312941540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd40ffba0 msgr2=0x7f4bd419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd2a00640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd40ffba0 0x7f4bd419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd2a00640 1 -- 172.21.15.57:0/312941540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd41010d0 msgr2=0x7f4bd41a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd2a00640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd41010d0 0x7f4bd41a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd2a00640 1 -- 172.21.15.57:0/312941540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd41a1fb0 con 0x7f4bd4100560 2024-04-13T05:17:38.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd2a00640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 0x7f4bd419a8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000d770 tx=0x7f4bc000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0017070 con 0x7f4bd4100560 2024-04-13T05:17:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bc0009c30 con 0x7f4bd4100560 2024-04-13T05:17:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.494+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd41a22a0 con 0x7f4bd4100560 2024-04-13T05:17:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.495+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0003d00 con 0x7f4bd4100560 2024-04-13T05:17:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.495+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd41a2840 con 0x7f4bd4100560 2024-04-13T05:17:38.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.496+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd410c430 con 0x7f4bd4100560 2024-04-13T05:17:38.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.499+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4bc0003ea0 con 0x7f4bd4100560 2024-04-13T05:17:38.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:38 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2442310061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:38.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:38 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/11890090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:38.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.500+0000 7f4bcbe00640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb80a3c30 0x7f4bb80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.500+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4bc00d0500 con 0x7f4bd4100560 2024-04-13T05:17:38.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.501+0000 7f4bd3400640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb80a3c30 0x7f4bb80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.501+0000 7f4bd3400640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb80a3c30 0x7f4bb80a60f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006010 tx=0x7f4bc4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:38.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.503+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bc009cbc0 con 0x7f4bd4100560 2024-04-13T05:17:38.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.709+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4bd402d850 con 0x7f4bd4100560 2024-04-13T05:17:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.710+0000 7f4bcbe00640 1 -- 172.21.15.57:0/312941540 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f4bc0009de0 con 0x7f4bd4100560 2024-04-13T05:17:38.712 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb80a3c30 msgr2=0x7f4bb80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb80a3c30 0x7f4bb80a60f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006010 tx=0x7f4bc4000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 msgr2=0x7f4bd419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 0x7f4bd419a8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000d770 tx=0x7f4bc000dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 shutdown_connections 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4bb80a3c30 0x7f4bb80a60f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd41010d0 0x7f4bd41a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4bd4100560 0x7f4bd419a8d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 --2- 172.21.15.57:0/312941540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd40ffba0 0x7f4bd419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 >> 172.21.15.57:0/312941540 conn(0x7f4bd40fb680 msgr2=0x7f4bd40fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:38.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 shutdown_connections 2024-04-13T05:17:38.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.713+0000 7f4bd9000640 1 -- 172.21.15.57:0/312941540 wait complete. 2024-04-13T05:17:38.733 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:17:38.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 -- 172.21.15.57:0/203428752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941019b0 msgr2=0x7f2694101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 --2- 172.21.15.57:0/203428752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941019b0 0x7f2694101e30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f268c009900 tx=0x7f268c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 -- 172.21.15.57:0/203428752 shutdown_connections 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 --2- 172.21.15.57:0/203428752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2694101010 0x7f2694101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 --2- 172.21.15.57:0/203428752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941019b0 0x7f2694101e30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 --2- 172.21.15.57:0/203428752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26941006b0 0x7f2694100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 -- 172.21.15.57:0/203428752 >> 172.21.15.57:0/203428752 conn(0x7f26940fc290 msgr2=0x7f26940fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.946+0000 7f2698e00640 1 -- 172.21.15.57:0/203428752 shutdown_connections 2024-04-13T05:17:38.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.947+0000 7f2698e00640 1 -- 172.21.15.57:0/203428752 wait complete. 2024-04-13T05:17:38.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 Processor -- start 2024-04-13T05:17:38.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 -- start start 2024-04-13T05:17:38.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 0x7f269419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2694101010 0x7f269419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26941019b0 0x7f26941a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f269410fbe0 con 0x7f26941006b0 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f269410fa60 con 0x7f2694101010 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2698e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f269410fd60 con 0x7f26941019b0 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2693400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 0x7f269419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2693400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 0x7f269419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44788/0 (socket says 172.21.15.57:44788) 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.948+0000 7f2693400640 1 -- 172.21.15.57:0/3327330726 learned_addr learned my addr 172.21.15.57:0/3327330726 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2693400640 1 -- 172.21.15.57:0/3327330726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26941019b0 msgr2=0x7f26941a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2692a00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2694101010 0x7f269419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2693400640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26941019b0 0x7f26941a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2693400640 1 -- 172.21.15.57:0/3327330726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2694101010 msgr2=0x7f269419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2693400640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2694101010 0x7f269419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2693400640 1 -- 172.21.15.57:0/3327330726 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26941a1dd0 con 0x7f26941006b0 2024-04-13T05:17:38.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2693400640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 0x7f269419a220 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f268400eb90 tx=0x7f268400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:38.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684004d20 con 0x7f26941006b0 2024-04-13T05:17:38.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f268400ee90 con 0x7f26941006b0 2024-04-13T05:17:38.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.949+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26941a20c0 con 0x7f26941006b0 2024-04-13T05:17:38.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.950+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684010670 con 0x7f26941006b0 2024-04-13T05:17:38.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.950+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26941a25c0 con 0x7f26941006b0 2024-04-13T05:17:38.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.951+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269410b400 con 0x7f26941006b0 2024-04-13T05:17:38.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.953+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2684005020 con 0x7f26941006b0 2024-04-13T05:17:38.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.954+0000 7f268be00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26780a3bb0 0x7f26780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:38.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.954+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f26840d0fb0 con 0x7f26941006b0 2024-04-13T05:17:38.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.954+0000 7f2692a00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26780a3bb0 0x7f26780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:38.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.955+0000 7f2692a00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26780a3bb0 0x7f26780a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f269419b6c0 tx=0x7f267c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:38.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:38.959+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f268409d590 con 0x7f26941006b0 2024-04-13T05:17:39.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.215+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f269402d5e0 con 0x7f26941006b0 2024-04-13T05:17:39.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.216+0000 7f268be00640 1 -- 172.21.15.57:0/3327330726 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11855 (secure 0 0 0) 0x7f268409ce90 con 0x7f26941006b0 2024-04-13T05:17:39.218 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:39.219 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:15:22.311895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:17:39.219 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 24 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.219+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26780a3bb0 msgr2=0x7f26780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.219+0000 7f2698e00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26780a3bb0 0x7f26780a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f269419b6c0 tx=0x7f267c005f90 comp rx=0 tx=0).stop 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.219+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 msgr2=0x7f269419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.219+0000 7f2698e00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 0x7f269419a220 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f268400eb90 tx=0x7f268400c6a0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 shutdown_connections 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26780a3bb0 0x7f26780a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2694101010 0x7f269419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26941019b0 0x7f26941a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 --2- 172.21.15.57:0/3327330726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f26941006b0 0x7f269419a220 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 >> 172.21.15.57:0/3327330726 conn(0x7f26940fc290 msgr2=0x7f26940fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 shutdown_connections 2024-04-13T05:17:39.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.220+0000 7f2698e00640 1 -- 172.21.15.57:0/3327330726 wait complete. 2024-04-13T05:17:39.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:39.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:39 smithi057 ceph-mon[32331]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:39.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3506062545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:39.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/312941540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:39.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:39 smithi189 ceph-mon[35531]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:39.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3506062545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:39.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/312941540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:39.403 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:39 smithi083 ceph-mon[35133]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:39.404 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3506062545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:39.404 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/312941540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:39.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 -- 172.21.15.57:0/903518962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0104040 msgr2=0x7f6de0104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 --2- 172.21.15.57:0/903518962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0104040 0x7f6de0104460 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009900 tx=0x7f6dd002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 -- 172.21.15.57:0/903518962 shutdown_connections 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 --2- 172.21.15.57:0/903518962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de0105bf0 0x7f6de010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 --2- 172.21.15.57:0/903518962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 0x7f6de01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 --2- 172.21.15.57:0/903518962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0104040 0x7f6de0104460 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 -- 172.21.15.57:0/903518962 >> 172.21.15.57:0/903518962 conn(0x7f6de00ff5d0 msgr2=0x7f6de01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 -- 172.21.15.57:0/903518962 shutdown_connections 2024-04-13T05:17:39.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.470+0000 7f6ddf200640 1 -- 172.21.15.57:0/903518962 wait complete. 2024-04-13T05:17:39.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.471+0000 7f6ddf200640 1 Processor -- start 2024-04-13T05:17:39.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.471+0000 7f6ddf200640 1 -- start start 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de0104040 0x7f6de019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddf200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 0x7f6de019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddf200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 0x7f6de01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddf200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010fb90 con 0x7f6de0105bf0 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010fa10 con 0x7f6de0104040 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddf200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010fd10 con 0x7f6de0105230 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6dde800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 0x7f6de01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6dde800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 0x7f6de01a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44800/0 (socket says 172.21.15.57:44800) 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6dde800640 1 -- 172.21.15.57:0/3941231134 learned_addr learned my addr 172.21.15.57:0/3941231134 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddde00640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de0104040 0x7f6de019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.472+0000 7f6ddd400640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 0x7f6de019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dde800640 1 -- 172.21.15.57:0/3941231134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 msgr2=0x7f6de019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dde800640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 0x7f6de019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dde800640 1 -- 172.21.15.57:0/3941231134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de0104040 msgr2=0x7f6de019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dde800640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de0104040 0x7f6de019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dde800640 1 -- 172.21.15.57:0/3941231134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de01a1eb0 con 0x7f6de0105bf0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6ddd400640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 0x7f6de019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dde800640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 0x7f6de01a1840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009fd0 tx=0x7f6dd800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8017070 con 0x7f6de0105bf0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd8003e20 con 0x7f6de0105bf0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de01a21a0 con 0x7f6de0105bf0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8010460 con 0x7f6de0105bf0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.473+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a26f0 con 0x7f6de0105bf0 2024-04-13T05:17:39.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.474+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de010b430 con 0x7f6de0105bf0 2024-04-13T05:17:39.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.476+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6dd8003970 con 0x7f6de0105bf0 2024-04-13T05:17:39.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.478+0000 7f6dd6a00640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dc40a3c30 0x7f6dc40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.478+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6dd80d0650 con 0x7f6de0105bf0 2024-04-13T05:17:39.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.478+0000 7f6ddde00640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dc40a3c30 0x7f6dc40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.479+0000 7f6ddde00640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dc40a3c30 0x7f6dc40a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6dd002c240 tx=0x7f6dd0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.483+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6dd809cc30 con 0x7f6de0105bf0 2024-04-13T05:17:39.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.685+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6de0062bc0 con 0x7f6de0105bf0 2024-04-13T05:17:39.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.689+0000 7f6dd6a00640 1 -- 172.21.15.57:0/3941231134 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f6dd809c530 con 0x7f6de0105bf0 2024-04-13T05:17:39.692 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:39.692 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dc40a3c30 msgr2=0x7f6dc40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dc40a3c30 0x7f6dc40a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6dd002c240 tx=0x7f6dd0005fd0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 msgr2=0x7f6de01a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 0x7f6de01a1840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009fd0 tx=0x7f6dd800b260 comp rx=0 tx=0).stop 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 shutdown_connections 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dc40a3c30 0x7f6dc40a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de0105bf0 0x7f6de01a1840 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de0105230 0x7f6de019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 --2- 172.21.15.57:0/3941231134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6de0104040 0x7f6de019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.692+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 >> 172.21.15.57:0/3941231134 conn(0x7f6de00ff5d0 msgr2=0x7f6de0108600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.693+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 shutdown_connections 2024-04-13T05:17:39.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.693+0000 7f6ddf200640 1 -- 172.21.15.57:0/3941231134 wait complete. 2024-04-13T05:17:39.715 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-13T05:14:53.072497+0000', 'modified': '2024-04-13T05:15:22.311895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34499, 'mds_1': 34457, 'mds_2': 34427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6837/563533422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 563533422}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 563533422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6839/1915044018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 1915044018}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 1915044018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34397': {'gid': 34397, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6833/617814721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6832', 'nonce': 617814721}, {'type': 'v1', 'addr': '172.21.15.83:6833', 'nonce': 617814721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.83:6835/3245344075', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3245344075}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3245344075}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34457': {'gid': 34457, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.83:6837/741118790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 741118790}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 741118790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34499': {'gid': 34499, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6839/3095811120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 3095811120}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 3095811120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34499, 'qdb_cluster': [34427, 34499, 34457]} 2024-04-13T05:17:39.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-13T05:17:39.716 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.932+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1931790765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 msgr2=0x7f9dd8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.932+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1931790765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd8100530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009670 tx=0x7f9dc802bdd0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1931790765 shutdown_connections 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1931790765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 0x7f9dd8101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1931790765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd8101cc0 0x7f9dd8108550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1931790765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd8100530 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1931790765 >> 172.21.15.57:0/1931790765 conn(0x7f9dd80fb680 msgr2=0x7f9dd80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1931790765 shutdown_connections 2024-04-13T05:17:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.933+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1931790765 wait complete. 2024-04-13T05:17:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.934+0000 7f9dd7e00640 1 Processor -- start 2024-04-13T05:17:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.934+0000 7f9dd7e00640 1 -- start start 2024-04-13T05:17:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.934+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.934+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 0x7f9dd819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.934+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd8101cc0 0x7f9dd81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd810d9d0 con 0x7f9dd8100110 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd810d850 con 0x7f9dd8101300 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd810db50 con 0x7f9dd8101cc0 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44812/0 (socket says 172.21.15.57:44812) 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 -- 172.21.15.57:0/1435914254 learned_addr learned my addr 172.21.15.57:0/1435914254 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6000640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 0x7f9dd819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 -- 172.21.15.57:0/1435914254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd8101cc0 msgr2=0x7f9dd81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd8101cc0 0x7f9dd81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 -- 172.21.15.57:0/1435914254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 msgr2=0x7f9dd819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 0x7f9dd819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6a00640 1 -- 172.21.15.57:0/1435914254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd81a1f70 con 0x7f9dd8100110 2024-04-13T05:17:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.935+0000 7f9dd6000640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 0x7f9dd819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.936+0000 7f9dd6a00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd819a350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8034eb0 tx=0x7f9dc8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.936+0000 7f9ddca00640 1 -- 172.21.15.57:0/1435914254 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8039040 con 0x7f9dd8100110 2024-04-13T05:17:39.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.936+0000 7f9ddca00640 1 -- 172.21.15.57:0/1435914254 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dc8044070 con 0x7f9dd8100110 2024-04-13T05:17:39.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.936+0000 7f9ddca00640 1 -- 172.21.15.57:0/1435914254 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8004040 con 0x7f9dd8100110 2024-04-13T05:17:39.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.936+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd81a2200 con 0x7f9dd8100110 2024-04-13T05:17:39.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.936+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd81a26f0 con 0x7f9dd8100110 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 Processor -- start 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 -- start start 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9ddca00640 1 -- 172.21.15.57:0/1435914254 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9dc8034440 con 0x7f9dd8100110 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5f96bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5f9bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556ff5efc990 con 0x556ff5f967b0 2024-04-13T05:17:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ff5ea3530 con 0x556ff5f971a0 2024-04-13T05:17:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9de0e05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556ff5ef8950 con 0x556ff5f9c310 2024-04-13T05:17:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9dd7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5f96bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9dd7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5f96bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44822/0 (socket says 172.21.15.57:44822) 2024-04-13T05:17:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.940+0000 7f9dd7400640 1 -- 172.21.15.57:0/3029550429 learned_addr learned my addr 172.21.15.57:0/3029550429 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd6a00640 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5f9bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd6000640 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd7400640 1 -- 172.21.15.57:0/3029550429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 msgr2=0x556ff5f9e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd7400640 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f9e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd7400640 1 -- 172.21.15.57:0/3029550429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 msgr2=0x556ff5f9bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd7400640 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5f9bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd7400640 1 -- 172.21.15.57:0/3029550429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff5c0f2d0 con 0x556ff5f967b0 2024-04-13T05:17:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd6000640 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.941+0000 7f9dd7400640 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5f96bd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000bb90 tx=0x7f9dd000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63356904022f152a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.942+0000 7f9ddca00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9dbc0a3bb0 0x7f9dbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.942+0000 7f9dc3e00640 1 -- 172.21.15.57:0/3029550429 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0005220 con 0x556ff5f967b0 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.942+0000 7f9dc3e00640 1 -- 172.21.15.57:0/3029550429 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dd0004c50 con 0x556ff5f967b0 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.942+0000 7f9dc3e00640 1 -- 172.21.15.57:0/3029550429 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000e870 con 0x556ff5f967b0 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.942+0000 7f9ddca00640 1 -- 172.21.15.57:0/1435914254 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9dc80f2ba0 con 0x7f9dd8100110 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.942+0000 7f9dd6a00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9dbc0a3bb0 0x7f9dbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 -- 172.21.15.57:0/3029550429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 msgr2=0x556ff5f96bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9dd6a00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9dbc0a3bb0 0x7f9dbc0a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80f0670 tx=0x7f9dc802bdd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5f96bd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000bb90 tx=0x7f9dd000d7b0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 -- 172.21.15.57:0/3029550429 shutdown_connections 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5f9bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 --2- 172.21.15.57:0/3029550429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5f96bd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 -- 172.21.15.57:0/3029550429 >> 172.21.15.57:0/3029550429 conn(0x556ff5f91d20 msgr2=0x556ff5f92120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 -- 172.21.15.57:0/3029550429 shutdown_connections 2024-04-13T05:17:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 -- 172.21.15.57:0/3029550429 wait complete. 2024-04-13T05:17:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.943+0000 7f9de0e05740 1 -- start start 2024-04-13T05:17:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 Processor -- start 2024-04-13T05:17:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5fa0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556ff5f9f2c0 con 0x556ff5f971a0 2024-04-13T05:17:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ff5eb68f0 con 0x556ff5f967b0 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556ff5f9f440 con 0x556ff5f9c310 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9de0e05740 10 client.0 _ll_register_callbacks cb 0x556ff5eb62e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.944+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44832/0 (socket says 172.21.15.57:44832) 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 -- 172.21.15.57:0/1549213782 learned_addr learned my addr 172.21.15.57:0/1549213782 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd7400640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5fa0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd7400640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 -- 172.21.15.57:0/1549213782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 msgr2=0x556ff5fa8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 -- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 msgr2=0x556ff5fa0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5fa0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 -- 172.21.15.57:0/1549213782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff5cd96b0 con 0x556ff5f971a0 2024-04-13T05:17:39.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd7400640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dd6000640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa1230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400e6c0 tx=0x7f9dc400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2310a0a9a850c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.945+0000 7f9dc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9dc0c00640 1 -- 172.21.15.57:0/1549213782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc401c060 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd0004c50 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9dc0c00640 1 -- 172.21.15.57:0/1549213782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9dc4009a20 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9dc0c00640 1 -- 172.21.15.57:0/1549213782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dc4023050 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9de0e05740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556ff5fab620 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9dc0c00640 1 -- 172.21.15.57:0/1549213782 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dd0004c50 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.946+0000 7f9dc0c00640 1 -- 172.21.15.57:0/1549213782 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dc4015340 con 0x556ff5f971a0 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9de0e05740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9de0e05740 10 client.15396 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9de0e05740 20 client.15396 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9de0e05740 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff5fadc80 0x556ff5faf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9de0e05740 4 client.15396 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:17:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff5ea2b40 con 0x556ff5fadc80 2024-04-13T05:17:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.947+0000 7f9dd7400640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff5fadc80 0x556ff5faf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.948+0000 7f9dd7400640 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff5fadc80 0x556ff5faf0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d84c633f9ce19d36 server_cookie=84b8a84bdb819c01 in_seq=0 out_seq=0 2024-04-13T05:17:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.948+0000 7f9dc0c00640 10 client.15396 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.950+0000 7f9dc0c00640 1 -- 172.21.15.57:0/1549213782 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ff5ea2b40 con 0x556ff5fadc80 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.950+0000 7f9dc0c00640 10 client.15396 handle_command_reply: tid=0 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.950+0000 7f9de0e05740 1 client.15396 shutdown 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.950+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 msgr2=0x556ff5fa1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.950+0000 7f9de0e05740 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa1230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400e6c0 tx=0x7f9dc400eb80 comp rx=0 tx=0).stop 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 shutdown_connections 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff5fadc80 0x556ff5faf0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa1230 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 --2- 172.21.15.57:0/1549213782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5fa0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 >> 172.21.15.57:0/1549213782 conn(0x556ff5f91d20 msgr2=0x556ff5f90f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9dc0c00640 0 client.15396 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 shutdown_connections 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 1 -- 172.21.15.57:0/1549213782 wait complete. 2024-04-13T05:17:39.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.951+0000 7f9de0e05740 20 client.15396 trim_cache size 0 max 16384 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 Processor -- start 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 -- start start 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5faf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f8cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556ff5fad430 con 0x556ff5f971a0 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ff5fadb00 con 0x556ff5f967b0 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.971+0000 7f9de0e05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556ff5fae1d0 con 0x556ff5f9c310 2024-04-13T05:17:39.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44840/0 (socket says 172.21.15.57:44840) 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 -- 172.21.15.57:0/2879867373 learned_addr learned my addr 172.21.15.57:0/2879867373 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd7400640 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5faf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 -- 172.21.15.57:0/2879867373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 msgr2=0x556ff5f8cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f8cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 -- 172.21.15.57:0/2879867373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 msgr2=0x556ff5faf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5faf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 -- 172.21.15.57:0/2879867373 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff5f92a90 con 0x556ff5f971a0 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd7400640 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5faf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:39.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.972+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa8230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4015760 tx=0x7f9dc400c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ed7cc8c581abd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.973+0000 7f9dc3400640 1 -- 172.21.15.57:0/2879867373 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40159d0 con 0x556ff5f971a0 2024-04-13T05:17:39.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.973+0000 7f9dc3400640 1 -- 172.21.15.57:0/2879867373 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dc4015b70 con 0x556ff5f971a0 2024-04-13T05:17:39.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.973+0000 7f9dc3400640 1 -- 172.21.15.57:0/2879867373 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4009b00 con 0x556ff5f971a0 2024-04-13T05:17:39.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.973+0000 7f9de0e05740 1 -- 172.21.15.57:0/2879867373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 msgr2=0x556ff5fa8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.973+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa8230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4015760 tx=0x7f9dc400c600 comp rx=0 tx=0).stop 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 -- 172.21.15.57:0/2879867373 shutdown_connections 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5f8cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f971a0 0x556ff5fa8230 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2879867373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f967b0 0x556ff5faf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 -- 172.21.15.57:0/2879867373 >> 172.21.15.57:0/2879867373 conn(0x556ff5f91d20 msgr2=0x556ff5f92100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 -- 172.21.15.57:0/2879867373 shutdown_connections 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 -- 172.21.15.57:0/2879867373 wait complete. 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 -- start start 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.974+0000 7f9de0e05740 1 Processor -- start 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5fa3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5fa3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556ff5faef70 con 0x556ff5f967b0 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ff5fae8a0 con 0x556ff5f971a0 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556ff5faf640 con 0x556ff5f9c310 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9de0e05740 10 client.0 _ll_register_callbacks cb 0x7f9dc400cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9dd7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5fa3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9dd7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5fa3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44848/0 (socket says 172.21.15.57:44848) 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9dd7400640 1 -- 172.21.15.57:0/2073162848 learned_addr learned my addr 172.21.15.57:0/2073162848 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5fa3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.975+0000 7f9dd7400640 1 -- 172.21.15.57:0/2073162848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 msgr2=0x556ff5fa6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dd7400640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dd7400640 1 -- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 msgr2=0x556ff5fa3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dd7400640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5fa3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dd7400640 1 -- 172.21.15.57:0/2073162848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff5fa7560 con 0x556ff5f967b0 2024-04-13T05:17:39.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5fa3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dd7400640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5fa3a80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005d00 tx=0x7f9dd000b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492d2d81c559bd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dc0c00640 1 -- 172.21.15.57:0/2073162848 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004250 con 0x556ff5f967b0 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9dc0c00640 1 -- 172.21.15.57:0/2073162848 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9dd00043f0 con 0x556ff5f967b0 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.976+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc4015b70 con 0x556ff5f967b0 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9dc0c00640 1 -- 172.21.15.57:0/2073162848 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dd00035f0 con 0x556ff5f967b0 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:39.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556ff6013d30 con 0x556ff5f967b0 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9dc0c00640 1 -- 172.21.15.57:0/2073162848 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dc4015b70 con 0x556ff5f967b0 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9dc0c00640 1 -- 172.21.15.57:0/2073162848 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dd000b040 con 0x556ff5f967b0 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 10 client.15402 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 20 client.15402 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff6018790 0x556ff60174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 4 client.15402 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.977+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "status"}) -- 0x556ff5f0a3b0 con 0x556ff6018790 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.978+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff6018790 0x556ff60174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.979+0000 7f9dd6000640 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff6018790 0x556ff60174a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a51bc914f2007be server_cookie=5fea977cc5562a1e in_seq=0 out_seq=0 2024-04-13T05:17:39.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.979+0000 7f9dc0c00640 10 client.15402 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:39.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9dc0c00640 1 -- 172.21.15.57:0/2073162848 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556ff5f0a3b0 con 0x556ff6018790 2024-04-13T05:17:39.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9dc0c00640 10 client.15402 handle_command_reply: tid=0 2024-04-13T05:17:39.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 client.15402 shutdown 2024-04-13T05:17:39.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 msgr2=0x556ff5fa3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5fa3a80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005d00 tx=0x7f9dd000b3d0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 shutdown_connections 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x556ff6018790 0x556ff60174a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff5f971a0 0x556ff5fa3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ff5f9c310 0x556ff5fa6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 --2- 172.21.15.57:0/2073162848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ff5f967b0 0x556ff5fa3a80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9dc0c00640 0 client.15402 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 >> 172.21.15.57:0/2073162848 conn(0x556ff5f91d20 msgr2=0x556ff5f8cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 shutdown_connections 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 1 -- 172.21.15.57:0/2073162848 wait complete. 2024-04-13T05:17:39.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.980+0000 7f9de0e05740 20 client.15402 trim_cache size 0 max 16384 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34427, 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 138.68701968799999, 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 94, 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 201.650072338 2024-04-13T05:17:39.985 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:39.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9dbc0a3bb0 msgr2=0x7f9dbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9dbc0a3bb0 0x7f9dbc0a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80f0670 tx=0x7f9dc802bdd0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 msgr2=0x7f9dd819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:39.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd819a350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8034eb0 tx=0x7f9dc8003f10 comp rx=0 tx=0).stop 2024-04-13T05:17:39.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 shutdown_connections 2024-04-13T05:17:39.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9dbc0a3bb0 0x7f9dbc0a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd8101300 0x7f9dd819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd8101cc0 0x7f9dd81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:39.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 --2- 172.21.15.57:0/1435914254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd8100110 0x7f9dd819a350 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8034eb0 tx=0x7f9dc8003f10 comp rx=0 tx=0).stop 2024-04-13T05:17:39.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 >> 172.21.15.57:0/1435914254 conn(0x7f9dd80fb680 msgr2=0x7f9dd810a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:39.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 shutdown_connections 2024-04-13T05:17:39.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:39.981+0000 7f9dd7e00640 1 -- 172.21.15.57:0/1435914254 wait complete. 2024-04-13T05:17:40.001 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 -- 172.21.15.57:0/3731053280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a410a0b0 msgr2=0x7f47a4110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 --2- 172.21.15.57:0/3731053280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a410a0b0 0x7f47a4110940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f479c009900 tx=0x7f479c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 -- 172.21.15.57:0/3731053280 shutdown_connections 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 --2- 172.21.15.57:0/3731053280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a410a0b0 0x7f47a4110940 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 --2- 172.21.15.57:0/3731053280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47a41096f0 0x7f47a4109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 --2- 172.21.15.57:0/3731053280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a4108500 0x7f47a4108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.218+0000 7f47a3e00640 1 -- 172.21.15.57:0/3731053280 >> 172.21.15.57:0/3731053280 conn(0x7f47a4077490 msgr2=0x7f47a40798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.219+0000 7f47a3e00640 1 -- 172.21.15.57:0/3731053280 shutdown_connections 2024-04-13T05:17:40.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.219+0000 7f47a3e00640 1 -- 172.21.15.57:0/3731053280 wait complete. 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.219+0000 7f47a3e00640 1 Processor -- start 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 -- start start 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47a4108500 0x7f47a419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 0x7f47a419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a410a0b0 0x7f47a41a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4114040 con 0x7f47a41096f0 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4113ec0 con 0x7f47a410a0b0 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41141c0 con 0x7f47a4108500 2024-04-13T05:17:40.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 0x7f47a419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 0x7f47a419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44870/0 (socket says 172.21.15.57:44870) 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.220+0000 7f47a2000640 1 -- 172.21.15.57:0/1442143434 learned_addr learned my addr 172.21.15.57:0/1442143434 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2a00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47a4108500 0x7f47a419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a3400640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a410a0b0 0x7f47a41a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2000640 1 -- 172.21.15.57:0/1442143434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47a4108500 msgr2=0x7f47a419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2000640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47a4108500 0x7f47a419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2000640 1 -- 172.21.15.57:0/1442143434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a410a0b0 msgr2=0x7f47a41a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2000640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a410a0b0 0x7f47a41a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2000640 1 -- 172.21.15.57:0/1442143434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a41a62b0 con 0x7f47a41096f0 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a3400640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a410a0b0 0x7f47a41a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a2000640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 0x7f47a419ebd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f478c00ea80 tx=0x7f478c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a8a00640 1 -- 172.21.15.57:0/1442143434 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c00ce40 con 0x7f47a41096f0 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41a65a0 con 0x7f47a41096f0 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a8a00640 1 -- 172.21.15.57:0/1442143434 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f478c005020 con 0x7f47a41096f0 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.221+0000 7f47a8a00640 1 -- 172.21.15.57:0/1442143434 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c0116d0 con 0x7f47a41096f0 2024-04-13T05:17:40.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.222+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41a6aa0 con 0x7f47a41096f0 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 Processor -- start 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 -- start start 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x561047483bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x561047488ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5610473e9990 con 0x561047489310 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561047390530 con 0x5610474841a0 2024-04-13T05:17:40.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.225+0000 7f47acd8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610473e5950 con 0x5610474837b0 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x561047488ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x561047483bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x561047488ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45576/0 (socket says 172.21.15.57:45576) 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 -- 172.21.15.57:0/4237307015 learned_addr learned my addr 172.21.15.57:0/4237307015 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a8a00640 1 -- 172.21.15.57:0/1442143434 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f478c0051d0 con 0x7f47a41096f0 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 -- 172.21.15.57:0/4237307015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 msgr2=0x561047483bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x561047483bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 -- 172.21.15.57:0/4237307015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 msgr2=0x56104748b7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:17:40.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748b7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.226+0000 7f47a2a00640 1 -- 172.21.15.57:0/4237307015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610470fc2d0 con 0x5610474841a0 2024-04-13T05:17:40.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.227+0000 7f47a2a00640 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x561047488ca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f479400bc90 tx=0x7f479400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3195813ed65b4f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.227+0000 7f4793e00640 1 -- 172.21.15.57:0/4237307015 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794015070 con 0x5610474841a0 2024-04-13T05:17:40.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.227+0000 7f4793e00640 1 -- 172.21.15.57:0/4237307015 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47940035f0 con 0x5610474841a0 2024-04-13T05:17:40.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.227+0000 7f4793e00640 1 -- 172.21.15.57:0/4237307015 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47940116a0 con 0x5610474841a0 2024-04-13T05:17:40.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47a8a00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47840a3bb0 0x7f47840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 -- 172.21.15.57:0/4237307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 msgr2=0x561047488ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x561047488ca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f479400bc90 tx=0x7f479400f730 comp rx=0 tx=0).stop 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47a8a00640 1 -- 172.21.15.57:0/1442143434 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f478c0d0ba0 con 0x7f47a41096f0 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 -- 172.21.15.57:0/4237307015 shutdown_connections 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x561047488ca0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 --2- 172.21.15.57:0/4237307015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x561047483bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 -- 172.21.15.57:0/4237307015 >> 172.21.15.57:0/4237307015 conn(0x56104747ed20 msgr2=0x56104747f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47a3400640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47840a3bb0 0x7f47840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 -- 172.21.15.57:0/4237307015 shutdown_connections 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.228+0000 7f47acd8e740 1 -- 172.21.15.57:0/4237307015 wait complete. 2024-04-13T05:17:40.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 -- start start 2024-04-13T05:17:40.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 Processor -- start 2024-04-13T05:17:40.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47a3400640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47840a3bb0 0x7f47840a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f479c02c170 tx=0x7f479c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x56104748ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x56104748e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047495330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56104748c2c0 con 0x5610474841a0 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610473a38f0 con 0x5610474837b0 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.229+0000 7f47acd8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56104748c440 con 0x561047489310 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47acd8e740 10 client.0 _ll_register_callbacks cb 0x5610473a32e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x56104748e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x56104748e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44886/0 (socket says 172.21.15.57:44886) 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 -- 172.21.15.57:0/565876791 learned_addr learned my addr 172.21.15.57:0/565876791 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2a00640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x56104748ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a3400640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047495330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 -- 172.21.15.57:0/565876791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 msgr2=0x561047495330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047495330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 -- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 msgr2=0x56104748ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x56104748ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 -- 172.21.15.57:0/565876791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610471c66b0 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f47a2000640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x56104748e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f478c098e90 tx=0x7f478c09d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba3b1c8014c82e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.230+0000 7f4790c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f4790c00640 1 -- 172.21.15.57:0/565876791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c01f030 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f4790c00640 1 -- 172.21.15.57:0/565876791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f478c005540 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47940035f0 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f4790c00640 1 -- 172.21.15.57:0/565876791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f478c0a56f0 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f47acd8e740 10 client.15411 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561047498710 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f4790c00640 1 -- 172.21.15.57:0/565876791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47940035f0 con 0x5610474841a0 2024-04-13T05:17:40.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.231+0000 7f4790c00640 1 -- 172.21.15.57:0/565876791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f478c00f040 con 0x5610474841a0 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47acd8e740 10 client.15411 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47acd8e740 10 client.15411 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47acd8e740 20 client.15411 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47acd8e740 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56104749ace0 0x56104749c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47acd8e740 4 client.15411 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104738fb40 con 0x56104749ace0 2024-04-13T05:17:40.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.232+0000 7f47a2a00640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56104749ace0 0x56104749c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.233+0000 7f47a2a00640 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56104749ace0 0x56104749c150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84c8d6dfa804ceda server_cookie=8a00dffa56e0055d in_seq=0 out_seq=0 2024-04-13T05:17:40.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.233+0000 7f4790c00640 10 client.15411 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:40.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.235+0000 7f4790c00640 1 -- 172.21.15.57:0/565876791 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56104738fb40 con 0x56104749ace0 2024-04-13T05:17:40.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.235+0000 7f4790c00640 10 client.15411 handle_command_reply: tid=0 2024-04-13T05:17:40.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.235+0000 7f47acd8e740 1 client.15411 shutdown 2024-04-13T05:17:40.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.235+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 msgr2=0x56104748e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.235+0000 7f47acd8e740 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x56104748e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f478c098e90 tx=0x7f478c09d2b0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 shutdown_connections 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56104749ace0 0x56104749c150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047495330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x56104748e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 --2- 172.21.15.57:0/565876791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x56104748ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f4790c00640 0 client.15411 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 >> 172.21.15.57:0/565876791 conn(0x56104747ed20 msgr2=0x56104747df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 shutdown_connections 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 1 -- 172.21.15.57:0/565876791 wait complete. 2024-04-13T05:17:40.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.236+0000 7f47acd8e740 20 client.15411 trim_cache size 0 max 16384 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 Processor -- start 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 -- start start 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x56104748c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x5610474952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56104749a490 con 0x561047489310 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56104749ab60 con 0x5610474841a0 2024-04-13T05:17:40.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47acd8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56104749b230 con 0x5610474837b0 2024-04-13T05:17:40.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.256+0000 7f47a3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44896/0 (socket says 172.21.15.57:44896) 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 -- 172.21.15.57:0/1269590165 learned_addr learned my addr 172.21.15.57:0/1269590165 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a2a00640 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x56104748c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a2000640 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x5610474952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 -- 172.21.15.57:0/1269590165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 msgr2=0x56104748c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x56104748c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 -- 172.21.15.57:0/1269590165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 msgr2=0x5610474952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x5610474952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 -- 172.21.15.57:0/1269590165 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104747fa90 con 0x561047489310 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.257+0000 7f47a3400640 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748cd20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f479c036040 tx=0x7f479c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4af8f4bb1166890 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f4793400640 1 -- 172.21.15.57:0/1269590165 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c050070 con 0x561047489310 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f4793400640 1 -- 172.21.15.57:0/1269590165 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c046070 con 0x561047489310 2024-04-13T05:17:40.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f4793400640 1 -- 172.21.15.57:0/1269590165 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c004900 con 0x561047489310 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f47acd8e740 1 -- 172.21.15.57:0/1269590165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 msgr2=0x56104748cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f47acd8e740 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748cd20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f479c036040 tx=0x7f479c03f000 comp rx=0 tx=0).stop 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f47acd8e740 1 -- 172.21.15.57:0/1269590165 shutdown_connections 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f47acd8e740 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474841a0 0x5610474952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.258+0000 7f47acd8e740 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561047489310 0x56104748cd20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.259+0000 7f47acd8e740 1 --2- 172.21.15.57:0/1269590165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610474837b0 0x56104748c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.259+0000 7f47acd8e740 1 -- 172.21.15.57:0/1269590165 >> 172.21.15.57:0/1269590165 conn(0x56104747ed20 msgr2=0x561047479da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.259+0000 7f47acd8e740 1 -- 172.21.15.57:0/1269590165 shutdown_connections 2024-04-13T05:17:40.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.259+0000 7f47acd8e740 1 -- 172.21.15.57:0/1269590165 wait complete. 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.259+0000 7f47acd8e740 1 -- start start 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.259+0000 7f47acd8e740 1 Processor -- start 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x5610474909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x561047490ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047493e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56104749bfd0 con 0x5610474841a0 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56104749b900 con 0x5610474837b0 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56104749c6a0 con 0x561047489310 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47acd8e740 10 client.0 _ll_register_callbacks cb 0x7f478c09c5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x561047490ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x561047490ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44900/0 (socket says 172.21.15.57:44900) 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 -- 172.21.15.57:0/315333206 learned_addr learned my addr 172.21.15.57:0/315333206 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2a00640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x5610474909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a3400640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047493e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 -- 172.21.15.57:0/315333206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 msgr2=0x561047493e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047493e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 -- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 msgr2=0x5610474909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x5610474909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.260+0000 7f47a2000640 1 -- 172.21.15.57:0/315333206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561047494460 con 0x5610474841a0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f47a2000640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x561047490ef0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f478c00efd0 tx=0x7f478c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91ac5139498acc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f4790c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f4790c00640 1 -- 172.21.15.57:0/315333206 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c01f070 con 0x5610474841a0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f4790c00640 1 -- 172.21.15.57:0/315333206 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f478c09c0d0 con 0x5610474841a0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479c046070 con 0x5610474841a0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f4790c00640 1 -- 172.21.15.57:0/315333206 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f478c0987d0 con 0x5610474841a0 2024-04-13T05:17:40.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f47acd8e740 10 client.15417 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.261+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561047500c90 con 0x5610474841a0 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f4790c00640 1 -- 172.21.15.57:0/315333206 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c046070 con 0x5610474841a0 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f4790c00640 1 -- 172.21.15.57:0/315333206 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f478c063b00 con 0x5610474841a0 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47acd8e740 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47acd8e740 10 client.15417 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47acd8e740 20 client.15417 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47acd8e740 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5610475057a0 0x5610475044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47acd8e740 4 client.15417 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "status"}) -- 0x5610473f73b0 con 0x5610475057a0 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.262+0000 7f47a2a00640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5610475057a0 0x5610475044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.263+0000 7f47a2a00640 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5610475057a0 0x5610475044b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=985f96f4f5e369a9 server_cookie=27b95f7ad771f4f4 in_seq=0 out_seq=0 2024-04-13T05:17:40.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.263+0000 7f4790c00640 10 client.15417 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f4790c00640 1 -- 172.21.15.57:0/315333206 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5610473f73b0 con 0x5610475057a0 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f4790c00640 10 client.15417 handle_command_reply: tid=0 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 client.15417 shutdown 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 msgr2=0x561047490ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x561047490ef0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f478c00efd0 tx=0x7f478c005540 comp rx=0 tx=0).stop 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 shutdown_connections 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5610475057a0 0x5610475044b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561047489310 0x561047493e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5610474841a0 0x561047490ef0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f4790c00640 0 client.15417 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 --2- 172.21.15.57:0/315333206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610474837b0 0x5610474909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 >> 172.21.15.57:0/315333206 conn(0x56104747ed20 msgr2=0x56104748c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 shutdown_connections 2024-04-13T05:17:40.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.264+0000 7f47acd8e740 1 -- 172.21.15.57:0/315333206 wait complete. 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47acd8e740 20 client.15417 trim_cache size 0 max 16384 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34457, 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 139.99127268000001, 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 94, 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T05:17:40.269 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 189.64559032099999 2024-04-13T05:17:40.270 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:40.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47840a3bb0 msgr2=0x7f47840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47840a3bb0 0x7f47840a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f479c02c170 tx=0x7f479c005a20 comp rx=0 tx=0).stop 2024-04-13T05:17:40.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 msgr2=0x7f47a419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 0x7f47a419ebd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f478c00ea80 tx=0x7f478c00ef40 comp rx=0 tx=0).stop 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 shutdown_connections 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f47840a3bb0 0x7f47840a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47a410a0b0 0x7f47a41a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47a41096f0 0x7f47a419ebd0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 --2- 172.21.15.57:0/1442143434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47a4108500 0x7f47a419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 >> 172.21.15.57:0/1442143434 conn(0x7f47a4077490 msgr2=0x7f47a410ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 shutdown_connections 2024-04-13T05:17:40.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.265+0000 7f47a3e00640 1 -- 172.21.15.57:0/1442143434 wait complete. 2024-04-13T05:17:40.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:40 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3327330726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:40.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:40 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3941231134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:40.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-13T05:17:40.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:40 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3327330726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:40.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:40 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3941231134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:40.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:40 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3327330726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:40.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:40 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3941231134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:40.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.514+0000 7fb58be00640 1 -- 172.21.15.57:0/3016877592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 msgr2=0x7fb58c10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.514+0000 7fb58be00640 1 --2- 172.21.15.57:0/3016877592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c10a340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb584009900 tx=0x7fb58402bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 -- 172.21.15.57:0/3016877592 shutdown_connections 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 --2- 172.21.15.57:0/3016877592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 0x7fb58c1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 --2- 172.21.15.57:0/3016877592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c10a340 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 --2- 172.21.15.57:0/3016877592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 0x7fb58c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 -- 172.21.15.57:0/3016877592 >> 172.21.15.57:0/3016877592 conn(0x7fb58c0fd460 msgr2=0x7fb58c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 -- 172.21.15.57:0/3016877592 shutdown_connections 2024-04-13T05:17:40.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.515+0000 7fb58be00640 1 -- 172.21.15.57:0/3016877592 wait complete. 2024-04-13T05:17:40.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.516+0000 7fb58be00640 1 Processor -- start 2024-04-13T05:17:40.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.516+0000 7fb58be00640 1 -- start start 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.516+0000 7fb58be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 0x7fb58c075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.516+0000 7fb58be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 0x7fb58c0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c10da70 con 0x7fb58c103a20 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c10d8f0 con 0x7fb58c103060 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c10dbf0 con 0x7fb58c101e80 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44924/0 (socket says 172.21.15.57:44924) 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58b400640 1 -- 172.21.15.57:0/1319580282 learned_addr learned my addr 172.21.15.57:0/1319580282 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58aa00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 0x7fb58c075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.517+0000 7fb58a000640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 0x7fb58c0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58b400640 1 -- 172.21.15.57:0/1319580282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 msgr2=0x7fb58c075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58b400640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 0x7fb58c075d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58b400640 1 -- 172.21.15.57:0/1319580282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 msgr2=0x7fb58c0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58b400640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 0x7fb58c0743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58b400640 1 -- 172.21.15.57:0/1319580282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c075180 con 0x7fb58c103a20 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58aa00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 0x7fb58c075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58a000640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 0x7fb58c0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58b400640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c0748f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb584036040 tx=0x7fb584004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb583400640 1 -- 172.21.15.57:0/1319580282 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584043020 con 0x7fb58c103a20 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58c1a21d0 con 0x7fb58c103a20 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.518+0000 7fb583400640 1 -- 172.21.15.57:0/1319580282 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb584004490 con 0x7fb58c103a20 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.519+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58c1a2720 con 0x7fb58c103a20 2024-04-13T05:17:40.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.519+0000 7fb583400640 1 -- 172.21.15.57:0/1319580282 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584003cb0 con 0x7fb58c103a20 2024-04-13T05:17:40.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.522+0000 7fb5943f3740 1 Processor -- start 2024-04-13T05:17:40.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.522+0000 7fb583400640 1 -- 172.21.15.57:0/1319580282 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb584046020 con 0x7fb58c103a20 2024-04-13T05:17:40.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 -- start start 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bcfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bd27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555865b30990 con 0x555865bd0310 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555865ad7530 con 0x555865bca7b0 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb5943f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555865b2c950 con 0x555865bcb1a0 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb58a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bd27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb58a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bd27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44940/0 (socket says 172.21.15.57:44940) 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.523+0000 7fb58a000640 1 -- 172.21.15.57:0/3262755263 learned_addr learned my addr 172.21.15.57:0/3262755263 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58aa00640 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bcfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 -- 172.21.15.57:0/3262755263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 msgr2=0x555865bcfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bcfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 -- 172.21.15.57:0/3262755263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 msgr2=0x555865bcabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 -- 172.21.15.57:0/3262755263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558658432d0 con 0x555865bd0310 2024-04-13T05:17:40.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb58a000640 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bd27a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb578012ac0 tx=0x7fb578009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e988e66e903ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.524+0000 7fb583400640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5700a3bb0 0x7fb5700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb577400640 1 -- 172.21.15.57:0/3262755263 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578003970 con 0x555865bd0310 2024-04-13T05:17:40.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb577400640 1 -- 172.21.15.57:0/3262755263 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578010830 con 0x555865bd0310 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb583400640 1 -- 172.21.15.57:0/1319580282 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb5840f2c30 con 0x7fb58c103a20 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb577400640 1 -- 172.21.15.57:0/3262755263 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578004410 con 0x555865bd0310 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb58aa00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5700a3bb0 0x7fb5700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb58aa00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5700a3bb0 0x7fb5700a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555865bcb5a0 tx=0x7fb57c00c330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb5943f3740 1 -- 172.21.15.57:0/3262755263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 msgr2=0x555865bd27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.525+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bd27a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb578012ac0 tx=0x7fb578009820 comp rx=0 tx=0).stop 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 -- 172.21.15.57:0/3262755263 shutdown_connections 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bd27a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bcfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3262755263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 -- 172.21.15.57:0/3262755263 >> 172.21.15.57:0/3262755263 conn(0x555865bc5d20 msgr2=0x555865bc6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 -- 172.21.15.57:0/3262755263 shutdown_connections 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 -- 172.21.15.57:0/3262755263 wait complete. 2024-04-13T05:17:40.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.526+0000 7fb5943f3740 1 -- start start 2024-04-13T05:17:40.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 Processor -- start 2024-04-13T05:17:40.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bd4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bd52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bdc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555865bd32c0 con 0x555865bd0310 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555865aea8f0 con 0x555865bca7b0 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555865bd3440 con 0x555865bcb1a0 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb5943f3740 10 client.0 _ll_register_callbacks cb 0x555865aea2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb58b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bdc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb58b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bdc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44942/0 (socket says 172.21.15.57:44942) 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.527+0000 7fb58b400640 1 -- 172.21.15.57:0/973708009 learned_addr learned my addr 172.21.15.57:0/973708009 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58aa00640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bd4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58a000640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bd52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58b400640 1 -- 172.21.15.57:0/973708009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 msgr2=0x555865bd52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58b400640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bd52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58b400640 1 -- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 msgr2=0x555865bd4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58b400640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bd4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58b400640 1 -- 172.21.15.57:0/973708009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55586590d6b0 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb58b400640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bdc2a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb58402c210 tx=0x7fb5840be650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6134a07207b6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb56be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb56be00640 1 -- 172.21.15.57:0/973708009 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5840bf210 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb56be00640 1 -- 172.21.15.57:0/973708009 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb5840035f0 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.528+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578010830 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb56be00640 1 -- 172.21.15.57:0/973708009 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5840beb10 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 10 client.15429 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555865bdf610 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb56be00640 1 -- 172.21.15.57:0/973708009 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578010830 con 0x555865bd0310 2024-04-13T05:17:40.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb56be00640 1 -- 172.21.15.57:0/973708009 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5840da070 con 0x555865bd0310 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 10 client.15429 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 20 client.15429 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865be1d30 0x555865be31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 4 client.15429 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.529+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555865ad6b40 con 0x555865be1d30 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.530+0000 7fb58aa00640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865be1d30 0x555865be31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.531+0000 7fb58aa00640 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865be1d30 0x555865be31a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31830563dba270fe server_cookie=210098fbd8759059 in_seq=0 out_seq=0 2024-04-13T05:17:40.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.531+0000 7fb56be00640 10 client.15429 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:40.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb56be00640 1 -- 172.21.15.57:0/973708009 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555865ad6b40 con 0x555865be1d30 2024-04-13T05:17:40.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb56be00640 10 client.15429 handle_command_reply: tid=0 2024-04-13T05:17:40.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 client.15429 shutdown 2024-04-13T05:17:40.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 msgr2=0x555865bdc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bdc2a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb58402c210 tx=0x7fb5840be650 comp rx=0 tx=0).stop 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 shutdown_connections 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865be1d30 0x555865be31a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bd0310 0x555865bdc2a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bcb1a0 0x555865bd52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 --2- 172.21.15.57:0/973708009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bd4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb56be00640 0 client.15429 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 >> 172.21.15.57:0/973708009 conn(0x555865bc5d20 msgr2=0x555865bc4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 shutdown_connections 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 1 -- 172.21.15.57:0/973708009 wait complete. 2024-04-13T05:17:40.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.533+0000 7fb5943f3740 20 client.15429 trim_cache size 0 max 16384 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 Processor -- start 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 -- start start 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bcccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bd0310 0x555865bdc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555865be14e0 con 0x555865bcb1a0 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555865be1bb0 con 0x555865bca7b0 2024-04-13T05:17:40.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb5943f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555865be2280 con 0x555865bd0310 2024-04-13T05:17:40.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb58a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bcccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb58a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bcccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44950/0 (socket says 172.21.15.57:44950) 2024-04-13T05:17:40.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.554+0000 7fb58a000640 1 -- 172.21.15.57:0/2097654293 learned_addr learned my addr 172.21.15.57:0/2097654293 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58aa00640 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58b400640 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bd0310 0x555865bdc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58a000640 1 -- 172.21.15.57:0/2097654293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bd0310 msgr2=0x555865bdc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58a000640 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bd0310 0x555865bdc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58a000640 1 -- 172.21.15.57:0/2097654293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 msgr2=0x555865bcc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58a000640 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58a000640 1 -- 172.21.15.57:0/2097654293 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555865bcfa40 con 0x555865bcb1a0 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb58a000640 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bcccb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb578006010 tx=0x7fb57801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f62da0d4829aae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb576a00640 1 -- 172.21.15.57:0/2097654293 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578004d10 con 0x555865bcb1a0 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb576a00640 1 -- 172.21.15.57:0/2097654293 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578023070 con 0x555865bcb1a0 2024-04-13T05:17:40.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.555+0000 7fb576a00640 1 -- 172.21.15.57:0/2097654293 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5780052a0 con 0x555865bcb1a0 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 -- 172.21.15.57:0/2097654293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 msgr2=0x555865bcccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bcccb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb578006010 tx=0x7fb57801b040 comp rx=0 tx=0).stop 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 -- 172.21.15.57:0/2097654293 shutdown_connections 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bd0310 0x555865bdc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bcccb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 --2- 172.21.15.57:0/2097654293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bca7b0 0x555865bcc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 -- 172.21.15.57:0/2097654293 >> 172.21.15.57:0/2097654293 conn(0x555865bc5d20 msgr2=0x555865bd37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 -- 172.21.15.57:0/2097654293 shutdown_connections 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.556+0000 7fb5943f3740 1 -- 172.21.15.57:0/2097654293 wait complete. 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 -- start start 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 Processor -- start 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bca7b0 0x555865bd75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bd0310 0x555865bdaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555865be3020 con 0x555865bcb1a0 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555865be2950 con 0x555865bd0310 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555865be36f0 con 0x555865bca7b0 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.557+0000 7fb5943f3740 10 client.0 _ll_register_callbacks cb 0x7fb5840baea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bd7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44952/0 (socket says 172.21.15.57:44952) 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 -- 172.21.15.57:0/3397275764 learned_addr learned my addr 172.21.15.57:0/3397275764 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58b400640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bd0310 0x555865bdaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58aa00640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bca7b0 0x555865bd75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 -- 172.21.15.57:0/3397275764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bca7b0 msgr2=0x555865bd75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bca7b0 0x555865bd75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 -- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bd0310 msgr2=0x555865bdaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:17:40.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bd0310 0x555865bdaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 -- 172.21.15.57:0/3397275764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555865bdb050 con 0x555865bcb1a0 2024-04-13T05:17:40.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58aa00640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bca7b0 0x555865bd75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:40.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58b400640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bd0310 0x555865bdaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:40.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb58a000640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bd7ae0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb578002230 tx=0x7fb578004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7af933151165ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.558+0000 7fb56be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb56be00640 1 -- 172.21.15.57:0/3397275764 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57802a670 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb56be00640 1 -- 172.21.15.57:0/3397275764 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb57802a810 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578023070 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb56be00640 1 -- 172.21.15.57:0/3397275764 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb578009ca0 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555865bdc350 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb56be00640 1 -- 172.21.15.57:0/3397275764 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb578023070 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb56be00640 1 -- 172.21.15.57:0/3397275764 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb57800c030 con 0x555865bcb1a0 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 20 client.15435 populate_metadata read hostname 'smithi057' 2024-04-13T05:17:40.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865c4bc50 0x555865c4a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 4 client.15435 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.559+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "status"}) -- 0x555865b3e3b0 con 0x555865c4bc50 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.560+0000 7fb58aa00640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865c4bc50 0x555865c4a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.561+0000 7fb58aa00640 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865c4bc50 0x555865c4a960 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04bbed1988d0eac server_cookie=85baa22d28f23cb9 in_seq=0 out_seq=0 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.561+0000 7fb56be00640 10 client.15435 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb56be00640 1 -- 172.21.15.57:0/3397275764 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555865b3e3b0 con 0x555865c4bc50 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb56be00640 10 client.15435 handle_command_reply: tid=0 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 client.15435 shutdown 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 msgr2=0x555865bd7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bd7ae0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb578002230 tx=0x7fb578004aa0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 shutdown_connections 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555865c4bc50 0x555865c4a960 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555865bd0310 0x555865bdaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555865bcb1a0 0x555865bd7ae0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 --2- 172.21.15.57:0/3397275764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555865bca7b0 0x555865bd75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb56be00640 0 client.15435 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 >> 172.21.15.57:0/3397275764 conn(0x555865bc5d20 msgr2=0x555865bcc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 shutdown_connections 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.562+0000 7fb5943f3740 1 -- 172.21.15.57:0/3397275764 wait complete. 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb5943f3740 20 client.15435 trim_cache size 0 max 16384 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "0a178454-f953-11ee-bc8d-c7b262605968", 2024-04-13T05:17:40.568 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 34499, 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 166.964386676, 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 24, 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 94, 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 173.645609054 2024-04-13T05:17:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5700a3bb0 msgr2=0x7fb5700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5700a3bb0 0x7fb5700a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555865bcb5a0 tx=0x7fb57c00c330 comp rx=0 tx=0).stop 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 msgr2=0x7fb58c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c0748f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb584036040 tx=0x7fb584004270 comp rx=0 tx=0).stop 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 shutdown_connections 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5700a3bb0 0x7fb5700a6070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb58c103060 0x7fb58c0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb58c103a20 0x7fb58c0748f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 --2- 172.21.15.57:0/1319580282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb58c101e80 0x7fb58c075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:40.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 >> 172.21.15.57:0/1319580282 conn(0x7fb58c0fd460 msgr2=0x7fb58c106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:40.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.563+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 shutdown_connections 2024-04-13T05:17:40.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:40.564+0000 7fb58be00640 1 -- 172.21.15.57:0/1319580282 wait complete. 2024-04-13T05:17:40.586 INFO:teuthology.run_tasks:Running task sequential... 2024-04-13T05:17:40.597 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-13T05:17:40.597 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-13T05:17:40.597 INFO:teuthology.task.print:Enabling mgr modules 2024-04-13T05:17:40.597 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-13T05:17:40.614 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-13T05:17:40.615 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-13T05:17:40.615 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-13T05:17:40.615 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-13T05:17:40.615 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:40.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:40.615 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-04-13T05:17:40.671 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-13T05:17:40.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:40.672 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-13T05:17:40.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:40.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2024-04-13T05:17:40.897 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-13T05:17:40.897 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph fs ls 2024-04-13T05:17:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:41 smithi057 ceph-mon[32331]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:41.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:41 smithi189 ceph-mon[35531]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:41.349 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:17:41.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:41 smithi083 ceph-mon[35133]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:17:43.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:43 smithi057 ceph-mon[32331]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:43.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:43 smithi189 ceph-mon[35531]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:43.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:43 smithi083 ceph-mon[35133]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 -- 172.21.15.57:0/3505776815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 msgr2=0x7f5bf0107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 --2- 172.21.15.57:0/3505776815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf0107e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5be00099c0 tx=0x7f5be0009cd0 comp rx=0 tx=0).stop 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 -- 172.21.15.57:0/3505776815 shutdown_connections 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 --2- 172.21.15.57:0/3505776815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0103520 0x7f5bf010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 --2- 172.21.15.57:0/3505776815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf0107e20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 --2- 172.21.15.57:0/3505776815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bf0102b60 0x7f5bf0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 -- 172.21.15.57:0/3505776815 >> 172.21.15.57:0/3505776815 conn(0x7f5bf0079c10 msgr2=0x7f5bf007a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:43.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 -- 172.21.15.57:0/3505776815 shutdown_connections 2024-04-13T05:17:43.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.745+0000 7f5bf5400700 1 -- 172.21.15.57:0/3505776815 wait complete. 2024-04-13T05:17:43.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 Processor -- start 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 -- start start 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b60 0x7f5bf019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bf0103520 0x7f5bf019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf0112cb0 con 0x7f5bf0107a40 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf0112b30 con 0x7f5bf0102b60 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.747+0000 7f5bf5400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf0112e30 con 0x7f5bf0103520 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44976/0 (socket says 172.21.15.57:44976) 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 -- 172.21.15.57:0/1235096083 learned_addr learned my addr 172.21.15.57:0/1235096083 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bee800700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b60 0x7f5bf019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:43.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5be7e00700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bf0103520 0x7f5bf019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:43.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 -- 172.21.15.57:0/1235096083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bf0103520 msgr2=0x7f5bf019b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:43.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bf0103520 0x7f5bf019b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 -- 172.21.15.57:0/1235096083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b60 msgr2=0x7f5bf019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b60 0x7f5bf019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bef200700 1 -- 172.21.15.57:0/1235096083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf01a0290 con 0x7f5bf0107a40 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.748+0000 7f5bee800700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b60 0x7f5bf019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.749+0000 7f5bef200700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf019fb50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5be800cca0 tx=0x7f5be800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.749+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8016020 con 0x7f5bf0107a40 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.749+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5be800ad60 con 0x7f5bf0107a40 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.749+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf01a0580 con 0x7f5bf0107a40 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.749+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80115e0 con 0x7f5bf0107a40 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.749+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf01a7e00 con 0x7f5bf0107a40 2024-04-13T05:17:43.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.750+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf00697a0 con 0x7f5bf0107a40 2024-04-13T05:17:43.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.752+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5be8019020 con 0x7f5bf0107a40 2024-04-13T05:17:43.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.754+0000 7f5be7400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:43.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.754+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5be80d0ca0 con 0x7f5bf0107a40 2024-04-13T05:17:43.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.754+0000 7f5bee800700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:43.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.755+0000 7f5bee800700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005b40 tx=0x7f5be0005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:43.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:43.757+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5be8020020 con 0x7f5bf0107a40 2024-04-13T05:17:44.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.045+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5bf019c5e0 con 0x7f5bf0107a40 2024-04-13T05:17:44.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.046+0000 7f5be7400700 1 -- 172.21.15.57:0/1235096083 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f5bf019c5e0 con 0x7f5bf0107a40 2024-04-13T05:17:44.048 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bd40a3cc0 msgr2=0x7f5bd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005b40 tx=0x7f5be0005ab0 comp rx=0 tx=0).stop 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 msgr2=0x7f5bf019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf019fb50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5be800cca0 tx=0x7f5be800a340 comp rx=0 tx=0).stop 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 shutdown_connections 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bd40a3cc0 0x7f5bd40a6180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bf0107a40 0x7f5bf019fb50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bf0102b60 0x7f5bf019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 --2- 172.21.15.57:0/1235096083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bf0103520 0x7f5bf019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:44.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 >> 172.21.15.57:0/1235096083 conn(0x7f5bf0079c10 msgr2=0x7f5bf01009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:44.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 shutdown_connections 2024-04-13T05:17:44.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:44.049+0000 7f5bf5400700 1 -- 172.21.15.57:0/1235096083 wait complete. 2024-04-13T05:17:44.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:44 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1235096083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-13T05:17:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:44 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1235096083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-13T05:17:44.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:44 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1235096083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-13T05:17:44.890 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-13T05:17:44.891 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-13T05:17:44.891 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-13T05:17:44.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:44.891 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2024-04-13T05:17:44.946 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-13T05:17:44.946 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-13T05:17:44.946 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2024-04-13T05:17:44.946 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-04-13T05:17:44.946 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41709sec preferred_lft 41709sec 2024-04-13T05:17:44.947 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link noprefixroute 2024-04-13T05:17:44.948 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-04-13T05:17:44.948 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-13T05:17:44.948 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2024-04-13T05:17:44.948 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2024-04-13T05:17:44.949 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-13T05:17:44.949 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:17:44.949 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:17:44.949 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2024-04-13T05:17:44.949 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2024-04-13T05:17:44.949 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2024-04-13T05:17:44.950 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-13T05:17:44.950 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:17:45.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:17:45.096 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:45 smithi057 ceph-mon[32331]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:45.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:17:45.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:45.260 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-13T05:17:45.340 INFO:teuthology.orchestra.run.smithi057.stdout:1 2024-04-13T05:17:45.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:45.344 DEBUG:teuthology.orchestra.run.smithi057:> route 2024-04-13T05:17:45.395 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2024-04-13T05:17:45.395 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-13T05:17:45.395 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-13T05:17:45.395 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-13T05:17:45.395 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-13T05:17:45.396 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:17:45.397 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:17:45.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-13T05:17:45.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-13T05:17:45.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-13T05:17:45.397 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:17:45.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:45 smithi083 ceph-mon[35133]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:45.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:17:45.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:45 smithi189 ceph-mon[35531]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:17:45.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:17:45.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:45.662 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-04-13T05:17:45.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:45.718 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2024-04-13T05:17:45.774 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:17:45.774 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:17:45.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:45.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-13T05:17:45.775 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:17:45.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:17:45.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:17:45.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-13T05:17:45.997 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:17:46.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:17:46.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:17:46.333 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-13T05:17:46.333 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-04-13T05:17:46.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2024-04-13T05:17:46.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2024-04-13T05:17:46.333 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-04-13T05:17:46.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-13T05:17:46.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:17:46.544 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-13T05:17:46.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T05:17:46.544 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:46.595 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-13T05:17:46.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-13T05:17:46.595 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:46.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-04-13T05:17:46.779 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-04-13T05:17:46.837 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T05:17:46.838 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6851f3d5cad808da345862ffed2203ee2c439f121f2ac3db87bb2c5457b1ce80/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/eee32290b21af5b97799ed3b97942259138b04d6a361a6949b56fbc33ff47406/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ba38b59613f94e78d8047d50e6da3db99e177f5d78bb8531a90381148c82eb45/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/86ab784d697c2d6a710126b875258c1c5a46e596ffb190580fdb9bf6e15cbc46/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bc0adc14b06059c566e30aae0619eb2c6bb7c1cf05d5b6007ea150b67749e6a0/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/88f5e5177d007b1027576a24101334ffb000d2df9af20c9f436e68dc9282c4b0/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/b7bd757dedb3c03736477ef5fb575a52b3e9f6193e0d3927122420700d88449f/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e95c76b032ca6ef0528bd5d53e3d1a2bbef7697ac98e87aeff39bd3e73bc4da9/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/5fc78578520fb786ab26f5309544797a8902e2ed889908f6fc3ba1a61097a768/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/7897aeadb114fe2014e7c81b3367e0bceac12315f7fdebc7f217e8f1da1b05fa/merged 2024-04-13T05:17:46.839 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-04-13T05:17:46.840 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:46.840 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:46.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:46.841 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-04-13T05:17:46.887 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-13T05:17:46.888 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07 --client_fs=cephfs) 2024-04-13T05:17:46.932 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-04-13T05:17:47.013 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T05:17:47.058 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-04-13T05:17:47.058 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-04-13T05:17:47.059 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6851f3d5cad808da345862ffed2203ee2c439f121f2ac3db87bb2c5457b1ce80/merged 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/eee32290b21af5b97799ed3b97942259138b04d6a361a6949b56fbc33ff47406/merged 2024-04-13T05:17:47.060 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ba38b59613f94e78d8047d50e6da3db99e177f5d78bb8531a90381148c82eb45/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/86ab784d697c2d6a710126b875258c1c5a46e596ffb190580fdb9bf6e15cbc46/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bc0adc14b06059c566e30aae0619eb2c6bb7c1cf05d5b6007ea150b67749e6a0/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/88f5e5177d007b1027576a24101334ffb000d2df9af20c9f436e68dc9282c4b0/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/b7bd757dedb3c03736477ef5fb575a52b3e9f6193e0d3927122420700d88449f/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e95c76b032ca6ef0528bd5d53e3d1a2bbef7697ac98e87aeff39bd3e73bc4da9/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/5fc78578520fb786ab26f5309544797a8902e2ed889908f6fc3ba1a61097a768/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/7897aeadb114fe2014e7c81b3367e0bceac12315f7fdebc7f217e8f1da1b05fa/merged 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:47.061 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:47.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:47.062 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-04-13T05:17:47.128 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2024-04-13T05:17:47.126+0000 7f04cbc93480 -1 init, newargv = 0x55a8fbf9aea0 newargc=15 2024-04-13T05:17:47.129 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[87665]: starting ceph client 2024-04-13T05:17:47.158 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[87665]: starting fuse 2024-04-13T05:17:47.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:47 smithi057 ceph-mon[32331]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:47.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:47 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:17:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:47 smithi083 ceph-mon[35133]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:47 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:17:47.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:47 smithi189 ceph-mon[35531]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:17:47.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:47 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:17:48.252 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-04-13T05:17:48.342 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-04-13T05:17:48.386 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-04-13T05:17:48.387 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/6851f3d5cad808da345862ffed2203ee2c439f121f2ac3db87bb2c5457b1ce80/merged 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/eee32290b21af5b97799ed3b97942259138b04d6a361a6949b56fbc33ff47406/merged 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/ba38b59613f94e78d8047d50e6da3db99e177f5d78bb8531a90381148c82eb45/merged 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/86ab784d697c2d6a710126b875258c1c5a46e596ffb190580fdb9bf6e15cbc46/merged 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bc0adc14b06059c566e30aae0619eb2c6bb7c1cf05d5b6007ea150b67749e6a0/merged 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/88f5e5177d007b1027576a24101334ffb000d2df9af20c9f436e68dc9282c4b0/merged 2024-04-13T05:17:48.388 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/b7bd757dedb3c03736477ef5fb575a52b3e9f6193e0d3927122420700d88449f/merged 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e95c76b032ca6ef0528bd5d53e3d1a2bbef7697ac98e87aeff39bd3e73bc4da9/merged 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/5fc78578520fb786ab26f5309544797a8902e2ed889908f6fc3ba1a61097a768/merged 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/7897aeadb114fe2014e7c81b3367e0bceac12315f7fdebc7f217e8f1da1b05fa/merged 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:17:48.389 INFO:teuthology.orchestra.run.smithi057.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:48.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:48.390 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-04-13T05:17:48.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:48.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:48 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:48.478 INFO:teuthology.orchestra.run.smithi057.stdout:119 2024-04-13T05:17:48.478 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> sudo stdin-killer -- python3 -c ' 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> import re 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> import os 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2024-04-13T05:17:48.479 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> return f 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2024-04-13T05:17:48.480 DEBUG:teuthology.orchestra.run.smithi057:> ' 2024-04-13T05:17:48.500 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:48 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:48 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:48.647 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.87665.asok 2024-04-13T05:17:48.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-13T05:17:48.664 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87665.asok 2024-04-13T05:17:48.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:48.664 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87665.asok status 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "218a691417ef57c5d22a8d5cf3c5f41f7b542838", 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 19.0.0-2884-g218a6914 (218a691417ef57c5d22a8d5cf3c5f41f7b542838) squid (dev)", 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-13T05:17:48.886 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "87665", 2024-04-13T05:17:48.887 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07" 2024-04-13T05:17:48.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:17:48.887 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2024-04-13T05:17:48.887 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2024-04-13T05:17:48.887 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 15447, 2024-04-13T05:17:48.887 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 15447 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 4178242768 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 4178242768 2024-04-13T05:17:48.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.15447 192.168.0.1:0/4178242768", 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/4178242768", 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 5, 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 24, 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 94, 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2024-04-13T05:17:48.889 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:17:48.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:48.901 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2024-04-13T05:17:48.901 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:48.943 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2024-04-13T05:17:48.944 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:48.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:17:48.944 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-13T05:17:49.087 INFO:teuthology.run_tasks:Running task exec... 2024-04-13T05:17:49.097 INFO:teuthology.task.exec:Executing custom commands... 2024-04-13T05:17:49.098 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T05:17:49.098 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-13T05:17:49.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 -- 172.21.15.57:0/472068719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1006b0 msgr2=0x7f910c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 --2- 172.21.15.57:0/472068719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1006b0 0x7f910c100ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9100009900 tx=0x7f910002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 -- 172.21.15.57:0/472068719 shutdown_connections 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 --2- 172.21.15.57:0/472068719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f910c101010 0x7f910c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 --2- 172.21.15.57:0/472068719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c1019b0 0x7f910c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 --2- 172.21.15.57:0/472068719 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1006b0 0x7f910c100ad0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 -- 172.21.15.57:0/472068719 >> 172.21.15.57:0/472068719 conn(0x7f910c0fc2b0 msgr2=0x7f910c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 -- 172.21.15.57:0/472068719 shutdown_connections 2024-04-13T05:17:49.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.294+0000 7f9112000640 1 -- 172.21.15.57:0/472068719 wait complete. 2024-04-13T05:17:49.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.295+0000 7f9112000640 1 Processor -- start 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.295+0000 7f9112000640 1 -- start start 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9112000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f910c1006b0 0x7f910c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9112000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 0x7f910c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9112000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1019b0 0x7f910c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9112000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f910c10fba0 con 0x7f910c1019b0 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9112000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f910c10fa20 con 0x7f910c1006b0 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9112000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f910c10fd20 con 0x7f910c101010 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9111600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1019b0 0x7f910c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f910be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 0x7f910c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f910be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 0x7f910c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:43308/0 (socket says 172.21.15.57:43308) 2024-04-13T05:17:49.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f910be00640 1 -- 172.21.15.57:0/1785826519 learned_addr learned my addr 172.21.15.57:0/1785826519 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.296+0000 7f9110c00640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f910c1006b0 0x7f910c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f910be00640 1 -- 172.21.15.57:0/1785826519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f910c1006b0 msgr2=0x7f910c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f910be00640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f910c1006b0 0x7f910c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f910be00640 1 -- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1019b0 msgr2=0x7f910c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f910be00640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1019b0 0x7f910c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f910be00640 1 -- 172.21.15.57:0/1785826519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c1a1e80 con 0x7f910c101010 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f910be00640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 0x7f910c19a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90f800d950 tx=0x7f90f800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.297+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8017070 con 0x7f910c101010 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.298+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c1a2170 con 0x7f910c101010 2024-04-13T05:17:49.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.298+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90f8009c30 con 0x7f910c101010 2024-04-13T05:17:49.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.298+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f800f620 con 0x7f910c101010 2024-04-13T05:17:49.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.298+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c1a2710 con 0x7f910c101010 2024-04-13T05:17:49.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.299+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c10b610 con 0x7f910c101010 2024-04-13T05:17:49.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.302+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f90f80109a0 con 0x7f910c101010 2024-04-13T05:17:49.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.304+0000 7f9109600640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f90f40a3ba0 0x7f90f40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:49.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.304+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f90f80d0380 con 0x7f910c101010 2024-04-13T05:17:49.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.304+0000 7f9110c00640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f90f40a3ba0 0x7f90f40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:49.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.305+0000 7f9110c00640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f90f40a3ba0 0x7f90f40a6060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9100000c00 tx=0x7f910002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:49.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.318+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90f8009de0 con 0x7f910c101010 2024-04-13T05:17:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:49 smithi083 ceph-mon[35133]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:49.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:49 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:49 smithi057 ceph-mon[32331]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:49 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:49.574+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f910c02d5e0 con 0x7f910c101010 2024-04-13T05:17:49.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:49 smithi189 ceph-mon[35531]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:17:49.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:49.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:49 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.177+0000 7f9109600640 1 -- 172.21.15.57:0/1785826519 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f90f809c310 con 0x7f910c101010 2024-04-13T05:17:50.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f90f40a3ba0 msgr2=0x7f90f40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f90f40a3ba0 0x7f90f40a6060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9100000c00 tx=0x7f910002c7e0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 msgr2=0x7f910c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 0x7f910c19a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90f800d950 tx=0x7f90f800de10 comp rx=0 tx=0).stop 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 shutdown_connections 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f90f40a3ba0 0x7f90f40a6060 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f910c1019b0 0x7f910c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f910c101010 0x7f910c19a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.180+0000 7f9112000640 1 --2- 172.21.15.57:0/1785826519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f910c1006b0 0x7f910c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.181+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 >> 172.21.15.57:0/1785826519 conn(0x7f910c0fc2b0 msgr2=0x7f910c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.181+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 shutdown_connections 2024-04-13T05:17:50.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.181+0000 7f9112000640 1 -- 172.21.15.57:0/1785826519 wait complete. 2024-04-13T05:17:50.202 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-13T05:17:50.213 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-13T05:17:50.254 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 -- 172.21.15.57:0/226342437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254102f80 msgr2=0x7f3254103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 --2- 172.21.15.57:0/226342437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254102f80 0x7f3254103400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3240009900 tx=0x7f324002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 -- 172.21.15.57:0/226342437 shutdown_connections 2024-04-13T05:17:50.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 --2- 172.21.15.57:0/226342437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254103940 0x7f325410a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 --2- 172.21.15.57:0/226342437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254102f80 0x7f3254103400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 --2- 172.21.15.57:0/226342437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101e30 0x7f3254102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 -- 172.21.15.57:0/226342437 >> 172.21.15.57:0/226342437 conn(0x7f32540fd3a0 msgr2=0x7f32540ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 -- 172.21.15.57:0/226342437 shutdown_connections 2024-04-13T05:17:50.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.482+0000 7f3258c00640 1 -- 172.21.15.57:0/226342437 wait complete. 2024-04-13T05:17:50.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.483+0000 7f3258c00640 1 Processor -- start 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.483+0000 7f3258c00640 1 -- start start 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.483+0000 7f3258c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254101e30 0x7f3254075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.483+0000 7f3258c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254102f80 0x7f32540743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3258c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 0x7f32540748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3258c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f325410d9b0 con 0x7f3254103940 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3258c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f325410d830 con 0x7f3254102f80 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3258c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f325410db30 con 0x7f3254101e30 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 0x7f32540748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254101e30 0x7f3254075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 0x7f32540748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44732/0 (socket says 172.21.15.57:44732) 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 -- 172.21.15.57:0/3129569804 learned_addr learned my addr 172.21.15.57:0/3129569804 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3252a00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254102f80 0x7f32540743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 -- 172.21.15.57:0/3129569804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254101e30 msgr2=0x7f3254075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254101e30 0x7f3254075d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 -- 172.21.15.57:0/3129569804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254102f80 msgr2=0x7f32540743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254102f80 0x7f32540743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.484+0000 7f3253e00640 1 -- 172.21.15.57:0/3129569804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3254075180 con 0x7f3254103940 2024-04-13T05:17:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f3252a00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254102f80 0x7f32540743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:50.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f3253e00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 0x7f32540748f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f324c009fd0 tx=0x7f324c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c017070 con 0x7f3254103940 2024-04-13T05:17:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f324c003e20 con 0x7f3254103940 2024-04-13T05:17:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0106c0 con 0x7f3254103940 2024-04-13T05:17:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32541a2210 con 0x7f3254103940 2024-04-13T05:17:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.485+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32541a2710 con 0x7f3254103940 2024-04-13T05:17:50.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.487+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32541091b0 con 0x7f3254103940 2024-04-13T05:17:50.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.489+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f324c003970 con 0x7f3254103940 2024-04-13T05:17:50.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.491+0000 7f324be00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32380a3bb0 0x7f32380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.491+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f324c0d04d0 con 0x7f3254103940 2024-04-13T05:17:50.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.491+0000 7f3253400640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32380a3bb0 0x7f32380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.492+0000 7f3253400640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32380a3bb0 0x7f32380a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f324400bbe0 tx=0x7f324400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:50.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.496+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f324c09cab0 con 0x7f3254103940 2024-04-13T05:17:50.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:50 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1785826519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-13T05:17:50.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:50 smithi083 ceph-mon[35133]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-13T05:17:50.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:50 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:50 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:50 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1785826519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-13T05:17:50.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:50 smithi057 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-13T05:17:50.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:50 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:50 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:50 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1785826519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-13T05:17:50.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:50 smithi189 ceph-mon[35531]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-13T05:17:50.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:50 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:50 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:50.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.704+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3254062bc0 con 0x7f3254103940 2024-04-13T05:17:50.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.709+0000 7f324be00640 1 -- 172.21.15.57:0/3129569804 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f324c0047e0 con 0x7f3254103940 2024-04-13T05:17:50.712 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:50.712 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:50.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32380a3bb0 msgr2=0x7f32380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32380a3bb0 0x7f32380a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f324400bbe0 tx=0x7f324400b390 comp rx=0 tx=0).stop 2024-04-13T05:17:50.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 msgr2=0x7f32540748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 0x7f32540748f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f324c009fd0 tx=0x7f324c00b260 comp rx=0 tx=0).stop 2024-04-13T05:17:50.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 shutdown_connections 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f32380a3bb0 0x7f32380a6070 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254102f80 0x7f32540743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3254103940 0x7f32540748f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 --2- 172.21.15.57:0/3129569804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254101e30 0x7f3254075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 >> 172.21.15.57:0/3129569804 conn(0x7f32540fd3a0 msgr2=0x7f3254106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.712+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 shutdown_connections 2024-04-13T05:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.713+0000 7f3258c00640 1 -- 172.21.15.57:0/3129569804 wait complete. 2024-04-13T05:17:50.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:17:50.737 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 -- 172.21.15.57:0/1431280156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8076980 msgr2=0x7f5cc8076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 --2- 172.21.15.57:0/1431280156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8076980 0x7f5cc8076da0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009900 tx=0x7f5cb802bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 -- 172.21.15.57:0/1431280156 shutdown_connections 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 --2- 172.21.15.57:0/1431280156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc8077bc0 0x7f5cc8078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 --2- 172.21.15.57:0/1431280156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc8078560 0x7f5cc8078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 --2- 172.21.15.57:0/1431280156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8076980 0x7f5cc8076da0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 -- 172.21.15.57:0/1431280156 >> 172.21.15.57:0/1431280156 conn(0x7f5cc80ff7b0 msgr2=0x7f5cc8101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:50.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 -- 172.21.15.57:0/1431280156 shutdown_connections 2024-04-13T05:17:50.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.963+0000 7f5cce600640 1 -- 172.21.15.57:0/1431280156 wait complete. 2024-04-13T05:17:50.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 Processor -- start 2024-04-13T05:17:50.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 -- start start 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc8076980 0x7f5cc819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 0x7f5cc819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc8078560 0x7f5cc81a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8114180 con 0x7f5cc8077bc0 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8114000 con 0x7f5cc8076980 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.965+0000 7f5cce600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8114300 con 0x7f5cc8078560 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 0x7f5cc819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 0x7f5cc819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44756/0 (socket says 172.21.15.57:44756) 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 -- 172.21.15.57:0/1439097735 learned_addr learned my addr 172.21.15.57:0/1439097735 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc7400640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc8076980 0x7f5cc819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc7e00640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc8078560 0x7f5cc81a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 -- 172.21.15.57:0/1439097735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc8078560 msgr2=0x7f5cc81a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc8078560 0x7f5cc81a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 -- 172.21.15.57:0/1439097735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc8076980 msgr2=0x7f5cc819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc8076980 0x7f5cc819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:50.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.966+0000 7f5cc6a00640 1 -- 172.21.15.57:0/1439097735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81a6410 con 0x7f5cc8077bc0 2024-04-13T05:17:50.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.967+0000 7f5cc6a00640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 0x7f5cc819ed30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e9f0 tx=0x7f5cb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:50.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.967+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400cd80 con 0x7f5cc8077bc0 2024-04-13T05:17:50.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.967+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb40108e0 con 0x7f5cc8077bc0 2024-04-13T05:17:50.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.967+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc81a6700 con 0x7f5cc8077bc0 2024-04-13T05:17:50.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.967+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4018920 con 0x7f5cc8077bc0 2024-04-13T05:17:50.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.967+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81a6c50 con 0x7f5cc8077bc0 2024-04-13T05:17:50.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.968+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc8110b70 con 0x7f5cc8077bc0 2024-04-13T05:17:50.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.971+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5cb4010a90 con 0x7f5cc8077bc0 2024-04-13T05:17:50.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.974+0000 7f5cbfe00640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5cac0a3bb0 0x7f5cac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:50.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.974+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5cb40d0c40 con 0x7f5cc8077bc0 2024-04-13T05:17:50.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.974+0000 7f5cc7400640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5cac0a3bb0 0x7f5cac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:50.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.975+0000 7f5cc7400640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5cac0a3bb0 0x7f5cac0a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8006010 tx=0x7f5cb8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:50.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:50.977+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cb409d220 con 0x7f5cc8077bc0 2024-04-13T05:17:51.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.190+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5cc8062bc0 con 0x7f5cc8077bc0 2024-04-13T05:17:51.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.190+0000 7f5cbfe00640 1 -- 172.21.15.57:0/1439097735 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f5cb409cb20 con 0x7f5cc8077bc0 2024-04-13T05:17:51.192 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:17:51.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5cac0a3bb0 msgr2=0x7f5cac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5cac0a3bb0 0x7f5cac0a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8006010 tx=0x7f5cb8000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 msgr2=0x7f5cc819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 0x7f5cc819ed30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e9f0 tx=0x7f5cb400eeb0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 shutdown_connections 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5cac0a3bb0 0x7f5cac0a6070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cc8078560 0x7f5cc81a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc8077bc0 0x7f5cc819ed30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 --2- 172.21.15.57:0/1439097735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cc8076980 0x7f5cc819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 >> 172.21.15.57:0/1439097735 conn(0x7f5cc80ff7b0 msgr2=0x7f5cc81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 shutdown_connections 2024-04-13T05:17:51.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.193+0000 7f5cce600640 1 -- 172.21.15.57:0/1439097735 wait complete. 2024-04-13T05:17:51.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:17:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 -- 172.21.15.57:0/4111266036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78105280 msgr2=0x7f5a78107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/4111266036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78105280 0x7f5a78107670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009900 tx=0x7f5a6002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 -- 172.21.15.57:0/4111266036 shutdown_connections 2024-04-13T05:17:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/4111266036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78107bb0 0x7f5a7810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/4111266036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78105280 0x7f5a78107670 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/4111266036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a78102950 0x7f5a78104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 -- 172.21.15.57:0/4111266036 >> 172.21.15.57:0/4111266036 conn(0x7f5a780fc360 msgr2=0x7f5a780fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 -- 172.21.15.57:0/4111266036 shutdown_connections 2024-04-13T05:17:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.420+0000 7f5a7cc00640 1 -- 172.21.15.57:0/4111266036 wait complete. 2024-04-13T05:17:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.421+0000 7f5a7cc00640 1 Processor -- start 2024-04-13T05:17:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.421+0000 7f5a7cc00640 1 -- start start 2024-04-13T05:17:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a7cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a78102950 0x7f5a7819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a7cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78105280 0x7f5a7819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a7cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 0x7f5a781a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a7cc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810da00 con 0x7f5a78107bb0 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a7cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810d880 con 0x7f5a78105280 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a7cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810db80 con 0x7f5a78102950 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 0x7f5a781a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a78102950 0x7f5a7819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 0x7f5a781a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44784/0 (socket says 172.21.15.57:44784) 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 -- 172.21.15.57:0/3637865666 learned_addr learned my addr 172.21.15.57:0/3637865666 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a76a00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78105280 0x7f5a7819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 -- 172.21.15.57:0/3637865666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a78102950 msgr2=0x7f5a7819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a78102950 0x7f5a7819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 -- 172.21.15.57:0/3637865666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78105280 msgr2=0x7f5a7819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78105280 0x7f5a7819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.422+0000 7f5a77e00640 1 -- 172.21.15.57:0/3637865666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a781a1f70 con 0x7f5a78107bb0 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a76a00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78105280 0x7f5a7819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a77e00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 0x7f5a781a1900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000eaa0 tx=0x7f5a7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000cdb0 con 0x7f5a78107bb0 2024-04-13T05:17:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a70005020 con 0x7f5a78107bb0 2024-04-13T05:17:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a781a2260 con 0x7f5a78107bb0 2024-04-13T05:17:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a700116d0 con 0x7f5a78107bb0 2024-04-13T05:17:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.423+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a781a2760 con 0x7f5a78107bb0 2024-04-13T05:17:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.424+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a78109700 con 0x7f5a78107bb0 2024-04-13T05:17:51.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.426+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a700051d0 con 0x7f5a78107bb0 2024-04-13T05:17:51.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.428+0000 7f5a6fe00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a5c0a3bb0 0x7f5a5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.428+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5a700d09f0 con 0x7f5a78107bb0 2024-04-13T05:17:51.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.428+0000 7f5a77400640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a5c0a3bb0 0x7f5a5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.429+0000 7f5a77400640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a5c0a3bb0 0x7f5a5c0a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009bd0 tx=0x7f5a68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:51.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.432+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a70016030 con 0x7f5a78107bb0 2024-04-13T05:17:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-13T05:17:51.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-13T05:17:51.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:17:51.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3129569804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:51.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:17:51.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:17:51.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-13T05:17:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-13T05:17:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:17:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3129569804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:17:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:17:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3129569804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:17:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:17:51.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.639+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5a78062bc0 con 0x7f5a78107bb0 2024-04-13T05:17:51.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.640+0000 7f5a6fe00640 1 -- 172.21.15.57:0/3637865666 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f5a7009cfd0 con 0x7f5a78107bb0 2024-04-13T05:17:51.642 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a5c0a3bb0 msgr2=0x7f5a5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a5c0a3bb0 0x7f5a5c0a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009bd0 tx=0x7f5a68005f90 comp rx=0 tx=0).stop 2024-04-13T05:17:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 msgr2=0x7f5a781a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 0x7f5a781a1900 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000eaa0 tx=0x7f5a7000ef60 comp rx=0 tx=0).stop 2024-04-13T05:17:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 shutdown_connections 2024-04-13T05:17:51.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5a5c0a3bb0 0x7f5a5c0a6070 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a78105280 0x7f5a7819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a78107bb0 0x7f5a781a1900 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 --2- 172.21.15.57:0/3637865666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a78102950 0x7f5a7819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 >> 172.21.15.57:0/3637865666 conn(0x7f5a780fc360 msgr2=0x7f5a781020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 shutdown_connections 2024-04-13T05:17:51.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.643+0000 7f5a7cc00640 1 -- 172.21.15.57:0/3637865666 wait complete. 2024-04-13T05:17:51.664 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:17:51.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 -- 172.21.15.57:0/2143034053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0101e80 msgr2=0x7fd3e01022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/2143034053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0101e80 0x7fd3e01022a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009900 tx=0x7fd3d402bc40 comp rx=0 tx=0).stop 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 -- 172.21.15.57:0/2143034053 shutdown_connections 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/2143034053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0103060 0x7fd3e01034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/2143034053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3e0103a20 0x7fd3e010a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/2143034053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0101e80 0x7fd3e01022a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 -- 172.21.15.57:0/2143034053 >> 172.21.15.57:0/2143034053 conn(0x7fd3e00fd460 msgr2=0x7fd3e00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 -- 172.21.15.57:0/2143034053 shutdown_connections 2024-04-13T05:17:51.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.896+0000 7fd3e6a00640 1 -- 172.21.15.57:0/2143034053 wait complete. 2024-04-13T05:17:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.897+0000 7fd3e6a00640 1 Processor -- start 2024-04-13T05:17:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.897+0000 7fd3e6a00640 1 -- start start 2024-04-13T05:17:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0101e80 0x7fd3e0075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3e0103060 0x7fd3e00743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 0x7fd3e00748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e010da50 con 0x7fd3e0103a20 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e010d8d0 con 0x7fd3e0101e80 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e010dbd0 con 0x7fd3e0103060 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 0x7fd3e00748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 0x7fd3e00748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44806/0 (socket says 172.21.15.57:44806) 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e6000640 1 -- 172.21.15.57:0/1675755303 learned_addr learned my addr 172.21.15.57:0/1675755303 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e5600640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0101e80 0x7fd3e0075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.898+0000 7fd3e4c00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3e0103060 0x7fd3e00743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6000640 1 -- 172.21.15.57:0/1675755303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3e0103060 msgr2=0x7fd3e00743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6000640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3e0103060 0x7fd3e00743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6000640 1 -- 172.21.15.57:0/1675755303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0101e80 msgr2=0x7fd3e0075d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:51.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6000640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0101e80 0x7fd3e0075d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6000640 1 -- 172.21.15.57:0/1675755303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e0075180 con 0x7fd3e0103a20 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e5600640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0101e80 0x7fd3e0075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6000640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 0x7fd3e00748f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800eaa0 tx=0x7fd3d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800cdb0 con 0x7fd3e0103a20 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3d8005020 con 0x7fd3e0103a20 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80116d0 con 0x7fd3e0103a20 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e01a2240 con 0x7fd3e0103a20 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.899+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e01a2740 con 0x7fd3e0103a20 2024-04-13T05:17:51.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.901+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e01092a0 con 0x7fd3e0103a20 2024-04-13T05:17:51.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.903+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd3d80051d0 con 0x7fd3e0103a20 2024-04-13T05:17:51.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.904+0000 7fd3de000640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd3c80a3c30 0x7fd3c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:51.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.904+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd3d80d0a50 con 0x7fd3e0103a20 2024-04-13T05:17:51.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.904+0000 7fd3e5600640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd3c80a3c30 0x7fd3c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:51.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.905+0000 7fd3e5600640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd3c80a3c30 0x7fd3c80a60f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd3d402c250 tx=0x7fd3d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:51.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:51.908+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3d8016030 con 0x7fd3e0103a20 2024-04-13T05:17:52.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.119+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3e002d800 con 0x7fd3e0103a20 2024-04-13T05:17:52.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.120+0000 7fd3de000640 1 -- 172.21.15.57:0/1675755303 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fd3d809d030 con 0x7fd3e0103a20 2024-04-13T05:17:52.122 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:17:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd3c80a3c30 msgr2=0x7fd3c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd3c80a3c30 0x7fd3c80a60f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd3d402c250 tx=0x7fd3d4005fd0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 msgr2=0x7fd3e00748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 0x7fd3e00748f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800eaa0 tx=0x7fd3d800ef60 comp rx=0 tx=0).stop 2024-04-13T05:17:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 shutdown_connections 2024-04-13T05:17:52.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd3c80a3c30 0x7fd3c80a60f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e0103a20 0x7fd3e00748f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3e0103060 0x7fd3e00743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 --2- 172.21.15.57:0/1675755303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3e0101e80 0x7fd3e0075d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 >> 172.21.15.57:0/1675755303 conn(0x7fd3e00fd460 msgr2=0x7fd3e0106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 shutdown_connections 2024-04-13T05:17:52.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.123+0000 7fd3e6a00640 1 -- 172.21.15.57:0/1675755303 wait complete. 2024-04-13T05:17:52.146 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.370+0000 7f7b7a800640 1 -- 172.21.15.57:0/1802593959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b74108500 msgr2=0x7f7b74108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.370+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1802593959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b74108500 0x7f7b74108920 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009900 tx=0x7f7b6802bc40 comp rx=0 tx=0).stop 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 -- 172.21.15.57:0/1802593959 shutdown_connections 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1802593959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b741096f0 0x7f7b74109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1802593959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b7410a0b0 0x7f7b74110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1802593959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b74108500 0x7f7b74108920 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 -- 172.21.15.57:0/1802593959 >> 172.21.15.57:0/1802593959 conn(0x7f7b74077490 msgr2=0x7f7b740798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:52.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 -- 172.21.15.57:0/1802593959 shutdown_connections 2024-04-13T05:17:52.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.371+0000 7f7b7a800640 1 -- 172.21.15.57:0/1802593959 wait complete. 2024-04-13T05:17:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.372+0000 7f7b7a800640 1 Processor -- start 2024-04-13T05:17:52.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.372+0000 7f7b7a800640 1 -- start start 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b7a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b74108500 0x7f7b7419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b7a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 0x7f7b7419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b7a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b7410a0b0 0x7f7b741a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b7a800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74114090 con 0x7f7b741096f0 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b7a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74113f10 con 0x7f7b7410a0b0 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b7a800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74114210 con 0x7f7b74108500 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 0x7f7b7419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 0x7f7b7419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44818/0 (socket says 172.21.15.57:44818) 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 -- 172.21.15.57:0/1107439292 learned_addr learned my addr 172.21.15.57:0/1107439292 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b79400640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b74108500 0x7f7b7419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b79e00640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b7410a0b0 0x7f7b741a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 -- 172.21.15.57:0/1107439292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b74108500 msgr2=0x7f7b7419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b74108500 0x7f7b7419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 -- 172.21.15.57:0/1107439292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b7410a0b0 msgr2=0x7f7b741a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b7410a0b0 0x7f7b741a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b78a00640 1 -- 172.21.15.57:0/1107439292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741a6280 con 0x7f7b741096f0 2024-04-13T05:17:52.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.373+0000 7f7b79e00640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b7410a0b0 0x7f7b741a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:52.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.374+0000 7f7b78a00640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 0x7f7b7419eba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c920 tx=0x7f7b6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.374+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60007c70 con 0x7f7b741096f0 2024-04-13T05:17:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.374+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b60007e10 con 0x7f7b741096f0 2024-04-13T05:17:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.374+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60018820 con 0x7f7b741096f0 2024-04-13T05:17:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.374+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a6570 con 0x7f7b741096f0 2024-04-13T05:17:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.374+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a6a70 con 0x7f7b741096f0 2024-04-13T05:17:52.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.375+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b7410f7c0 con 0x7f7b741096f0 2024-04-13T05:17:52.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.377+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b600189c0 con 0x7f7b741096f0 2024-04-13T05:17:52.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.379+0000 7f7b72000640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7b5c0a3c30 0x7f7b5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.380+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7b600d09a0 con 0x7f7b741096f0 2024-04-13T05:17:52.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.380+0000 7f7b79400640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7b5c0a3c30 0x7f7b5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.381+0000 7f7b79400640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7b5c0a3c30 0x7f7b5c0a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802c250 tx=0x7f7b68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:52.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.383+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b60014030 con 0x7f7b741096f0 2024-04-13T05:17:52.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:52 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1439097735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:52 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3637865666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:52 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1675755303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:52 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1439097735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:52 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3637865666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:52 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1675755303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:52 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1439097735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:52 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3637865666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:52 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1675755303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:52.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.598+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7b74062bc0 con 0x7f7b741096f0 2024-04-13T05:17:52.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.599+0000 7f7b72000640 1 -- 172.21.15.57:0/1107439292 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f7b6009cf80 con 0x7f7b741096f0 2024-04-13T05:17:52.601 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:52.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7b5c0a3c30 msgr2=0x7f7b5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7b5c0a3c30 0x7f7b5c0a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802c250 tx=0x7f7b68000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 msgr2=0x7f7b7419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 0x7f7b7419eba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c920 tx=0x7f7b6000cde0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 shutdown_connections 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7b5c0a3c30 0x7f7b5c0a60f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b7410a0b0 0x7f7b741a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b741096f0 0x7f7b7419eba0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.602+0000 7f7b7a800640 1 --2- 172.21.15.57:0/1107439292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b74108500 0x7f7b7419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.603+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 >> 172.21.15.57:0/1107439292 conn(0x7f7b74077490 msgr2=0x7f7b7410c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.603+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 shutdown_connections 2024-04-13T05:17:52.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.603+0000 7f7b7a800640 1 -- 172.21.15.57:0/1107439292 wait complete. 2024-04-13T05:17:52.625 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 -- 172.21.15.57:0/92943262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03281019b0 msgr2=0x7f0328101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 --2- 172.21.15.57:0/92943262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03281019b0 0x7f0328101e30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0320009ca0 tx=0x7f032002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 -- 172.21.15.57:0/92943262 shutdown_connections 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 --2- 172.21.15.57:0/92943262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03281019b0 0x7f0328101e30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 --2- 172.21.15.57:0/92943262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0328101010 0x7f0328101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 --2- 172.21.15.57:0/92943262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281006b0 0x7f0328100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.854+0000 7f032e000640 1 -- 172.21.15.57:0/92943262 >> 172.21.15.57:0/92943262 conn(0x7f03280fc290 msgr2=0x7f03280fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.855+0000 7f032e000640 1 -- 172.21.15.57:0/92943262 shutdown_connections 2024-04-13T05:17:52.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.855+0000 7f032e000640 1 -- 172.21.15.57:0/92943262 wait complete. 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 Processor -- start 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 -- start start 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03281006b0 0x7f032819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 0x7f032819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281019b0 0x7f03281a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f032810fb90 con 0x7f0328101010 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f032810fa10 con 0x7f03281019b0 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.856+0000 7f032e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f032810fd10 con 0x7f03281006b0 2024-04-13T05:17:52.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 0x7f032819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f032cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03281006b0 0x7f032819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 0x7f032819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44846/0 (socket says 172.21.15.57:44846) 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 -- 172.21.15.57:0/2448410826 learned_addr learned my addr 172.21.15.57:0/2448410826 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f032d600640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281019b0 0x7f03281a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 -- 172.21.15.57:0/2448410826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03281006b0 msgr2=0x7f032819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03281006b0 0x7f032819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 -- 172.21.15.57:0/2448410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281019b0 msgr2=0x7f03281a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281019b0 0x7f03281a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:52.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.857+0000 7f0327e00640 1 -- 172.21.15.57:0/2448410826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281a1e20 con 0x7f0328101010 2024-04-13T05:17:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.858+0000 7f0327e00640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 0x7f032819a740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f031800d770 tx=0x7f031800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.858+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318017070 con 0x7f0328101010 2024-04-13T05:17:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.858+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03180054d0 con 0x7f0328101010 2024-04-13T05:17:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.858+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318004cf0 con 0x7f0328101010 2024-04-13T05:17:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.858+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03281a2110 con 0x7f0328101010 2024-04-13T05:17:52.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.858+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a2610 con 0x7f0328101010 2024-04-13T05:17:52.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.859+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032810b400 con 0x7f0328101010 2024-04-13T05:17:52.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.861+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0318004e90 con 0x7f0328101010 2024-04-13T05:17:52.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.863+0000 7f0325600640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f030c0a3c30 0x7f030c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:52.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.864+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f03180d0150 con 0x7f0328101010 2024-04-13T05:17:52.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.864+0000 7f032cc00640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f030c0a3c30 0x7f030c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:52.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.864+0000 7f032cc00640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f030c0a3c30 0x7f030c0a60f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0310005d00 tx=0x7f031000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:52.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:52.868+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f031809c730 con 0x7f0328101010 2024-04-13T05:17:53.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.078+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0328062bf0 con 0x7f0328101010 2024-04-13T05:17:53.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.080+0000 7f0325600640 1 -- 172.21.15.57:0/2448410826 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f031809c030 con 0x7f0328101010 2024-04-13T05:17:53.083 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:53.083 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:53.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f030c0a3c30 msgr2=0x7f030c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f030c0a3c30 0x7f030c0a60f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0310005d00 tx=0x7f031000b410 comp rx=0 tx=0).stop 2024-04-13T05:17:53.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 msgr2=0x7f032819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 0x7f032819a740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f031800d770 tx=0x7f031800dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 shutdown_connections 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f030c0a3c30 0x7f030c0a60f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03281019b0 0x7f03281a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0328101010 0x7f032819a740 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 --2- 172.21.15.57:0/2448410826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03281006b0 0x7f032819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.084+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 >> 172.21.15.57:0/2448410826 conn(0x7f03280fc290 msgr2=0x7f03280fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:53.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.085+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 shutdown_connections 2024-04-13T05:17:53.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.085+0000 7f032e000640 1 -- 172.21.15.57:0/2448410826 wait complete. 2024-04-13T05:17:53.109 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:17:53.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:17:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 -- 172.21.15.57:0/2765841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 msgr2=0x7f6df4078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 --2- 172.21.15.57:0/2765841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df4078020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 -- 172.21.15.57:0/2765841229 shutdown_connections 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 --2- 172.21.15.57:0/2765841229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4078560 0x7f6df4078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 --2- 172.21.15.57:0/2765841229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df4078020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 --2- 172.21.15.57:0/2765841229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4076980 0x7f6df4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 -- 172.21.15.57:0/2765841229 >> 172.21.15.57:0/2765841229 conn(0x7f6df40ff690 msgr2=0x7f6df4101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 -- 172.21.15.57:0/2765841229 shutdown_connections 2024-04-13T05:17:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.330+0000 7f6df9a00640 1 -- 172.21.15.57:0/2765841229 wait complete. 2024-04-13T05:17:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.331+0000 7f6df9a00640 1 Processor -- start 2024-04-13T05:17:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.331+0000 7f6df9a00640 1 -- start start 2024-04-13T05:17:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4076980 0x7f6df419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4078560 0x7f6df41a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4114050 con 0x7f6df4077bc0 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4113ed0 con 0x7f6df4078560 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41141d0 con 0x7f6df4076980 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44860/0 (socket says 172.21.15.57:44860) 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 -- 172.21.15.57:0/1909470584 learned_addr learned my addr 172.21.15.57:0/1909470584 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3e00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4076980 0x7f6df419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9000640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4078560 0x7f6df41a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 -- 172.21.15.57:0/1909470584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4076980 msgr2=0x7f6df419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4076980 0x7f6df419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 -- 172.21.15.57:0/1909470584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4078560 msgr2=0x7f6df41a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4078560 0x7f6df41a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df3400640 1 -- 172.21.15.57:0/1909470584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41a6300 con 0x7f6df4077bc0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.332+0000 7f6df9000640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4078560 0x7f6df41a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.333+0000 7f6df3400640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df419ec70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6de0036040 tx=0x7f6de00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.333+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0043020 con 0x7f6df4077bc0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.333+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df41a6590 con 0x7f6df4077bc0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.333+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6de00054c0 con 0x7f6df4077bc0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.333+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0003d70 con 0x7f6df4077bc0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.333+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41a6ae0 con 0x7f6df4077bc0 2024-04-13T05:17:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.335+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df4069150 con 0x7f6df4077bc0 2024-04-13T05:17:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.337+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6de0046020 con 0x7f6df4077bc0 2024-04-13T05:17:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.338+0000 7f6df0c00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dd80a3bb0 0x7f6dd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.338+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6de00f2d90 con 0x7f6df4077bc0 2024-04-13T05:17:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.339+0000 7f6df3e00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dd80a3bb0 0x7f6dd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.340+0000 7f6df3e00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dd80a3bb0 0x7f6dd80a6070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005e10 tx=0x7f6de4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.343+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6de004d020 con 0x7f6df4077bc0 2024-04-13T05:17:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:53 smithi083 ceph-mon[35133]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-13T05:17:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:53 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1107439292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:53 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2448410826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:53.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:53 smithi057 ceph-mon[32331]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-13T05:17:53.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:53 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1107439292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:53.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:53 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2448410826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:53.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.551+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6df402d5e0 con 0x7f6df4077bc0 2024-04-13T05:17:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.552+0000 7f6df0c00640 1 -- 172.21.15.57:0/1909470584 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f6de00bf370 con 0x7f6df4077bc0 2024-04-13T05:17:53.554 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.554+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dd80a3bb0 msgr2=0x7f6dd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.554+0000 7f6df9a00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dd80a3bb0 0x7f6dd80a6070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005e10 tx=0x7f6de4005da0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.554+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 msgr2=0x7f6df419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.554+0000 7f6df9a00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df419ec70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6de0036040 tx=0x7f6de00042f0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 shutdown_connections 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6dd80a3bb0 0x7f6dd80a6070 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df4078560 0x7f6df41a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6df4077bc0 0x7f6df419ec70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 --2- 172.21.15.57:0/1909470584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4076980 0x7f6df419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 >> 172.21.15.57:0/1909470584 conn(0x7f6df40ff690 msgr2=0x7f6df4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 shutdown_connections 2024-04-13T05:17:53.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.555+0000 7f6df9a00640 1 -- 172.21.15.57:0/1909470584 wait complete. 2024-04-13T05:17:53.577 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:17:53.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:53 smithi189 ceph-mon[35531]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-13T05:17:53.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:53 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1107439292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:53.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:53 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2448410826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:53.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2756267401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0078560 msgr2=0x7f8bc0078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2756267401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0078560 0x7f8bc0078e00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009ca0 tx=0x7f8bb802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2756267401 shutdown_connections 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2756267401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0078560 0x7f8bc0078e00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2756267401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0077bc0 0x7f8bc0078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2756267401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc0076980 0x7f8bc0076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2756267401 >> 172.21.15.57:0/2756267401 conn(0x7f8bc00ff7b0 msgr2=0x7f8bc0101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2756267401 shutdown_connections 2024-04-13T05:17:53.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.805+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2756267401 wait complete. 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.806+0000 7f8bc4c00640 1 Processor -- start 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 -- start start 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 0x7f8bc019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc0077bc0 0x7f8bc019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0078560 0x7f8bc01a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0114160 con 0x7f8bc0076980 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0113fe0 con 0x7f8bc0077bc0 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bc4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc01142e0 con 0x7f8bc0078560 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bbde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 0x7f8bc019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bbde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 0x7f8bc019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44882/0 (socket says 172.21.15.57:44882) 2024-04-13T05:17:53.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.807+0000 7f8bbde00640 1 -- 172.21.15.57:0/2837804910 learned_addr learned my addr 172.21.15.57:0/2837804910 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbd400640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc0077bc0 0x7f8bc019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbe800640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0078560 0x7f8bc01a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbde00640 1 -- 172.21.15.57:0/2837804910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0078560 msgr2=0x7f8bc01a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbde00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0078560 0x7f8bc01a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbde00640 1 -- 172.21.15.57:0/2837804910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc0077bc0 msgr2=0x7f8bc019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbde00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc0077bc0 0x7f8bc019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbde00640 1 -- 172.21.15.57:0/2837804910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc01a6380 con 0x7f8bc0076980 2024-04-13T05:17:53.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bbde00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 0x7f8bc019e760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000d950 tx=0x7f8bb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:53.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0017070 con 0x7f8bc0076980 2024-04-13T05:17:53.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bb0009c30 con 0x7f8bc0076980 2024-04-13T05:17:53.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.808+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc01a6670 con 0x7f8bc0076980 2024-04-13T05:17:53.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.809+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000f620 con 0x7f8bc0076980 2024-04-13T05:17:53.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.809+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc01a6c10 con 0x7f8bc0076980 2024-04-13T05:17:53.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.810+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc0110d00 con 0x7f8bc0076980 2024-04-13T05:17:53.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.812+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bb00109a0 con 0x7f8bc0076980 2024-04-13T05:17:53.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.813+0000 7f8bb6a00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8ba40a3c30 0x7f8ba40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:53.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.813+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8bb00d0460 con 0x7f8bc0076980 2024-04-13T05:17:53.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.814+0000 7f8bbd400640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8ba40a3c30 0x7f8ba40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:53.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.815+0000 7f8bbd400640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8ba40a3c30 0x7f8ba40a60f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005e10 tx=0x7f8bac005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:53.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:53.818+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bb0009de0 con 0x7f8bc0076980 2024-04-13T05:17:54.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.027+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8bc0062bf0 con 0x7f8bc0076980 2024-04-13T05:17:54.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.028+0000 7f8bb6a00640 1 -- 172.21.15.57:0/2837804910 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f8bb009ca40 con 0x7f8bc0076980 2024-04-13T05:17:54.030 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8ba40a3c30 msgr2=0x7f8ba40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8ba40a3c30 0x7f8ba40a60f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bac005e10 tx=0x7f8bac005da0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 msgr2=0x7f8bc019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 0x7f8bc019e760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000d950 tx=0x7f8bb000de10 comp rx=0 tx=0).stop 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 shutdown_connections 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8ba40a3c30 0x7f8ba40a60f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bc0077bc0 0x7f8bc019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0078560 0x7f8bc01a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 --2- 172.21.15.57:0/2837804910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8bc0076980 0x7f8bc019e760 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 >> 172.21.15.57:0/2837804910 conn(0x7f8bc00ff7b0 msgr2=0x7f8bc0100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:54.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 shutdown_connections 2024-04-13T05:17:54.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.031+0000 7f8bc4c00640 1 -- 172.21.15.57:0/2837804910 wait complete. 2024-04-13T05:17:54.052 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:17:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 -- 172.21.15.57:0/3127356588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 msgr2=0x7fbb64110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 --2- 172.21.15.57:0/3127356588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb64110940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009ca0 tx=0x7fbb5c02c100 comp rx=0 tx=0).stop 2024-04-13T05:17:54.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 -- 172.21.15.57:0/3127356588 shutdown_connections 2024-04-13T05:17:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 --2- 172.21.15.57:0/3127356588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb64110940 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 --2- 172.21.15.57:0/3127356588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 0x7fbb64109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 --2- 172.21.15.57:0/3127356588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 0x7fbb64108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 -- 172.21.15.57:0/3127356588 >> 172.21.15.57:0/3127356588 conn(0x7fbb64077490 msgr2=0x7fbb640798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 -- 172.21.15.57:0/3127356588 shutdown_connections 2024-04-13T05:17:54.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.286+0000 7fbb63e00640 1 -- 172.21.15.57:0/3127356588 wait complete. 2024-04-13T05:17:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.287+0000 7fbb63e00640 1 Processor -- start 2024-04-13T05:17:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.287+0000 7fbb63e00640 1 -- start start 2024-04-13T05:17:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.287+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 0x7fbb6419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.287+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 0x7fbb6419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb641a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64114010 con 0x7fbb6410a0b0 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64113e90 con 0x7fbb64108500 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64114190 con 0x7fbb641096f0 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb641a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb641a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44902/0 (socket says 172.21.15.57:44902) 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 -- 172.21.15.57:0/59450550 learned_addr learned my addr 172.21.15.57:0/59450550 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb62000640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 0x7fbb6419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb62a00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 0x7fbb6419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 -- 172.21.15.57:0/59450550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 msgr2=0x7fbb6419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 0x7fbb6419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 -- 172.21.15.57:0/59450550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 msgr2=0x7fbb6419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 0x7fbb6419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb63400640 1 -- 172.21.15.57:0/59450550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641a61b0 con 0x7fbb6410a0b0 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb62000640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 0x7fbb6419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:54.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.288+0000 7fbb62a00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 0x7fbb6419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.289+0000 7fbb63400640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb641a5b40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c032380 tx=0x7fbb5c035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.289+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c039040 con 0x7fbb6410a0b0 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.289+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb5c044070 con 0x7fbb6410a0b0 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.289+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c004670 con 0x7fbb6410a0b0 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.289+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb641a6440 con 0x7fbb6410a0b0 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.289+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641a68e0 con 0x7fbb6410a0b0 2024-04-13T05:17:54.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.290+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6410f750 con 0x7fbb6410a0b0 2024-04-13T05:17:54.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.292+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbb5c003970 con 0x7fbb6410a0b0 2024-04-13T05:17:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.293+0000 7fbb5b400640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbb480a3c30 0x7fbb480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.294+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbb5c0f3180 con 0x7fbb6410a0b0 2024-04-13T05:17:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.294+0000 7fbb62a00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbb480a3c30 0x7fbb480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.295+0000 7fbb62a00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbb480a3c30 0x7fbb480a60f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbb64109550 tx=0x7fbb54005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:54.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.298+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb5c0bf760 con 0x7fbb6410a0b0 2024-04-13T05:17:54.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:54 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1909470584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:54.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:54 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2837804910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:54.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.507+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb64062bc0 con 0x7fbb6410a0b0 2024-04-13T05:17:54.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.508+0000 7fbb5b400640 1 -- 172.21.15.57:0/59450550 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fbb5c0098b0 con 0x7fbb6410a0b0 2024-04-13T05:17:54.510 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:17:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbb480a3c30 msgr2=0x7fbb480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbb480a3c30 0x7fbb480a60f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbb64109550 tx=0x7fbb54005db0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 msgr2=0x7fbb641a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb641a5b40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c032380 tx=0x7fbb5c035e20 comp rx=0 tx=0).stop 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 shutdown_connections 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbb480a3c30 0x7fbb480a60f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb6410a0b0 0x7fbb641a5b40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb641096f0 0x7fbb6419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 --2- 172.21.15.57:0/59450550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb64108500 0x7fbb6419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 >> 172.21.15.57:0/59450550 conn(0x7fbb64077490 msgr2=0x7fbb6410c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 shutdown_connections 2024-04-13T05:17:54.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.511+0000 7fbb63e00640 1 -- 172.21.15.57:0/59450550 wait complete. 2024-04-13T05:17:54.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:54 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1909470584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:54.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:54 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2837804910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:54.534 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:17:54.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:54 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1909470584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:54.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:54 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2837804910' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.761+0000 7f17a5a00640 1 -- 172.21.15.57:0/1778127922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 msgr2=0x7f17a0108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.761+0000 7f17a5a00640 1 --2- 172.21.15.57:0/1778127922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a0108a40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 -- 172.21.15.57:0/1778127922 shutdown_connections 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 --2- 172.21.15.57:0/1778127922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a0109810 0x7f17a0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 --2- 172.21.15.57:0/1778127922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a010a1d0 0x7f17a0110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 --2- 172.21.15.57:0/1778127922 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a0108a40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 -- 172.21.15.57:0/1778127922 >> 172.21.15.57:0/1778127922 conn(0x7f17a00775c0 msgr2=0x7f17a0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 -- 172.21.15.57:0/1778127922 shutdown_connections 2024-04-13T05:17:54.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.762+0000 7f17a5a00640 1 -- 172.21.15.57:0/1778127922 wait complete. 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 Processor -- start 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 -- start start 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a0109810 0x7f17a019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a010a1d0 0x7f17a01a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0114150 con 0x7f17a0108620 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0113fd0 con 0x7f17a010a1d0 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.763+0000 7f17a5a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01142d0 con 0x7f17a0109810 2024-04-13T05:17:54.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44914/0 (socket says 172.21.15.57:44914) 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 -- 172.21.15.57:0/4070736226 learned_addr learned my addr 172.21.15.57:0/4070736226 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f1797e00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a0109810 0x7f17a019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179f400640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a010a1d0 0x7f17a01a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 -- 172.21.15.57:0/4070736226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a0109810 msgr2=0x7f17a019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a0109810 0x7f17a019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 -- 172.21.15.57:0/4070736226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a010a1d0 msgr2=0x7f17a01a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a010a1d0 0x7f17a01a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179ea00640 1 -- 172.21.15.57:0/4070736226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a01a64b0 con 0x7f17a0108620 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.764+0000 7f179f400640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a010a1d0 0x7f17a01a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.765+0000 7f179ea00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a019e890 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1790000c00 tx=0x7f1790004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.765+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790043020 con 0x7f17a0108620 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.765+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1790004aa0 con 0x7f17a0108620 2024-04-13T05:17:54.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.765+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a01a6740 con 0x7f17a0108620 2024-04-13T05:17:54.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.765+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790003d00 con 0x7f17a0108620 2024-04-13T05:17:54.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.765+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a01a6ce0 con 0x7f17a0108620 2024-04-13T05:17:54.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.766+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a010fa60 con 0x7f17a0108620 2024-04-13T05:17:54.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.768+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1790046020 con 0x7f17a0108620 2024-04-13T05:17:54.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.770+0000 7f1797400640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f17840a3c30 0x7f17840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:54.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.770+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f17900f2dc0 con 0x7f17a0108620 2024-04-13T05:17:54.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.770+0000 7f1797e00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f17840a3c30 0x7f17840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:54.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.771+0000 7f1797e00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f17840a3c30 0x7f17840a60f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1788005fd0 tx=0x7f1788005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:54.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.774+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f179004d020 con 0x7f17a0108620 2024-04-13T05:17:54.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.979+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f17a0062bc0 con 0x7f17a0108620 2024-04-13T05:17:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.980+0000 7f1797400640 1 -- 172.21.15.57:0/4070736226 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f17900bf3a0 con 0x7f17a0108620 2024-04-13T05:17:54.982 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f17840a3c30 msgr2=0x7f17840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f17840a3c30 0x7f17840a60f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1788005fd0 tx=0x7f1788005850 comp rx=0 tx=0).stop 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 msgr2=0x7f17a019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a019e890 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1790000c00 tx=0x7f1790004270 comp rx=0 tx=0).stop 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 shutdown_connections 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f17840a3c30 0x7f17840a60f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17a010a1d0 0x7f17a01a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a0109810 0x7f17a019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 --2- 172.21.15.57:0/4070736226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17a0108620 0x7f17a019e890 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 >> 172.21.15.57:0/4070736226 conn(0x7f17a00775c0 msgr2=0x7f17a010cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 shutdown_connections 2024-04-13T05:17:54.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:54.983+0000 7f17a5a00640 1 -- 172.21.15.57:0/4070736226 wait complete. 2024-04-13T05:17:55.004 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-13T05:17:55.004 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.226+0000 7ff1d4a00640 1 -- 172.21.15.57:0/436026583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109630 msgr2=0x7ff1d0109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.226+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/436026583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109630 0x7ff1d0109ab0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009900 tx=0x7ff1b802bc50 comp rx=0 tx=0).stop 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 -- 172.21.15.57:0/436026583 shutdown_connections 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/436026583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d0109ff0 0x7ff1d0110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/436026583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109630 0x7ff1d0109ab0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/436026583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d0108440 0x7ff1d0108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 -- 172.21.15.57:0/436026583 >> 172.21.15.57:0/436026583 conn(0x7ff1d0077400 msgr2=0x7ff1d0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 -- 172.21.15.57:0/436026583 shutdown_connections 2024-04-13T05:17:55.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.227+0000 7ff1d4a00640 1 -- 172.21.15.57:0/436026583 wait complete. 2024-04-13T05:17:55.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 Processor -- start 2024-04-13T05:17:55.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 -- start start 2024-04-13T05:17:55.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d0108440 0x7ff1d019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d0109630 0x7ff1d019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 0x7ff1d01a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0113f80 con 0x7ff1d0109ff0 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0113e00 con 0x7ff1d0109630 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.228+0000 7ff1d4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0114100 con 0x7ff1d0108440 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 0x7ff1d01a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 0x7ff1d01a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33892/0 (socket says 172.21.15.57:33892) 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 -- 172.21.15.57:0/476964192 learned_addr learned my addr 172.21.15.57:0/476964192 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cde00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d0109630 0x7ff1d019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1ce800640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d0108440 0x7ff1d019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 -- 172.21.15.57:0/476964192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d0108440 msgr2=0x7ff1d019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d0108440 0x7ff1d019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 -- 172.21.15.57:0/476964192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d0109630 msgr2=0x7ff1d019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d0109630 0x7ff1d019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.229+0000 7ff1cf200640 1 -- 172.21.15.57:0/476964192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d01a61e0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.230+0000 7ff1cf200640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 0x7ff1d01a5b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800cce0 tx=0x7ff1c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:55.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.230+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800f450 con 0x7ff1d0109ff0 2024-04-13T05:17:55.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.230+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1c800faa0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.230+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d01a64d0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.230+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8018b70 con 0x7ff1d0109ff0 2024-04-13T05:17:55.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.230+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d01a69d0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.231+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d010f700 con 0x7ff1d0109ff0 2024-04-13T05:17:55.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.233+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff1c8018d10 con 0x7ff1d0109ff0 2024-04-13T05:17:55.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.234+0000 7ff1c7400640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff1b40a3bb0 0x7ff1b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.235+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff1c80d0cf0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.235+0000 7ff1ce800640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff1b40a3bb0 0x7ff1b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.236+0000 7ff1ce800640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff1b40a3bb0 0x7ff1b40a6070 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00058c0 tx=0x7ff1c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:55.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.239+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1c8014030 con 0x7ff1d0109ff0 2024-04-13T05:17:55.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:55 smithi057 ceph-mon[32331]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-13T05:17:55.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:55 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/59450550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:55.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:55 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4070736226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.488+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1d002d5e0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.490+0000 7ff1c7400640 1 -- 172.21.15.57:0/476964192 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7ff1c809d2d0 con 0x7ff1d0109ff0 2024-04-13T05:17:55.492 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:55.492 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:17:49.577177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:17:55.493 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-04-13T05:17:55.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff1b40a3bb0 msgr2=0x7ff1b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff1b40a3bb0 0x7ff1b40a6070 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00058c0 tx=0x7ff1c0005850 comp rx=0 tx=0).stop 2024-04-13T05:17:55.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 msgr2=0x7ff1d01a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 0x7ff1d01a5b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800cce0 tx=0x7ff1c800a620 comp rx=0 tx=0).stop 2024-04-13T05:17:55.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 shutdown_connections 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff1b40a3bb0 0x7ff1b40a6070 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d0109630 0x7ff1d019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1d0109ff0 0x7ff1d01a5b70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 --2- 172.21.15.57:0/476964192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d0108440 0x7ff1d019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 >> 172.21.15.57:0/476964192 conn(0x7ff1d0077400 msgr2=0x7ff1d010c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 shutdown_connections 2024-04-13T05:17:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.493+0000 7ff1d4a00640 1 -- 172.21.15.57:0/476964192 wait complete. 2024-04-13T05:17:55.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:55 smithi083 ceph-mon[35133]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-13T05:17:55.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:55 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/59450550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:55.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:55 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4070736226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:55.515 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-13T05:17:55.515 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-13T05:17:55.515 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T05:17:55.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:55 smithi189 ceph-mon[35531]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-13T05:17:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:55 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/59450550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:55 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4070736226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:17:55.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 -- 172.21.15.57:0/1843856161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078077bc0 msgr2=0x7f8078078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 --2- 172.21.15.57:0/1843856161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078077bc0 0x7f8078078020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8064009900 tx=0x7f806402bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:55.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 -- 172.21.15.57:0/1843856161 shutdown_connections 2024-04-13T05:17:55.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 --2- 172.21.15.57:0/1843856161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078078560 0x7f8078078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 --2- 172.21.15.57:0/1843856161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078077bc0 0x7f8078078020 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 --2- 172.21.15.57:0/1843856161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078076980 0x7f8078076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 -- 172.21.15.57:0/1843856161 >> 172.21.15.57:0/1843856161 conn(0x7f80780ff7b0 msgr2=0x7f8078101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:55.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 -- 172.21.15.57:0/1843856161 shutdown_connections 2024-04-13T05:17:55.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.731+0000 7f8080000640 1 -- 172.21.15.57:0/1843856161 wait complete. 2024-04-13T05:17:55.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.732+0000 7f8080000640 1 Processor -- start 2024-04-13T05:17:55.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 -- start start 2024-04-13T05:17:55.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 0x7f807819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078077bc0 0x7f807819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078078560 0x7f80781a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8078114180 con 0x7f8078076980 2024-04-13T05:17:55.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8078114000 con 0x7f8078078560 2024-04-13T05:17:55.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f8080000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8078114300 con 0x7f8078077bc0 2024-04-13T05:17:55.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f807d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 0x7f807819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f807d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 0x7f807819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33904/0 (socket says 172.21.15.57:33904) 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.733+0000 7f807d600640 1 -- 172.21.15.57:0/2367582287 learned_addr learned my addr 172.21.15.57:0/2367582287 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807e000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078078560 0x7f80781a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807cc00640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078077bc0 0x7f807819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807d600640 1 -- 172.21.15.57:0/2367582287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078077bc0 msgr2=0x7f807819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807d600640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078077bc0 0x7f807819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807d600640 1 -- 172.21.15.57:0/2367582287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078078560 msgr2=0x7f80781a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807d600640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078078560 0x7f80781a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807d600640 1 -- 172.21.15.57:0/2367582287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80781a6480 con 0x7f8078076980 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807cc00640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078077bc0 0x7f807819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:55.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.734+0000 7f807d600640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 0x7f807819e860 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f806c00d770 tx=0x7f806c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:55.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.735+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c017070 con 0x7f8078076980 2024-04-13T05:17:55.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.735+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f806c0054d0 con 0x7f8078076980 2024-04-13T05:17:55.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.735+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c004cf0 con 0x7f8078076980 2024-04-13T05:17:55.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.735+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781a6770 con 0x7f8078076980 2024-04-13T05:17:55.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.735+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80781a6c70 con 0x7f8078076980 2024-04-13T05:17:55.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.736+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8078110de0 con 0x7f8078076980 2024-04-13T05:17:55.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.738+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f806c004e90 con 0x7f8078076980 2024-04-13T05:17:55.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.739+0000 7f8076000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80600a3c30 0x7f80600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:55.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.740+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f806c0d02a0 con 0x7f8078076980 2024-04-13T05:17:55.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.740+0000 7f807cc00640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80600a3c30 0x7f80600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:55.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.741+0000 7f807cc00640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80600a3c30 0x7f80600a60f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80640096c0 tx=0x7f8064000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:55.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.744+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f806c09c880 con 0x7f8078076980 2024-04-13T05:17:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.950+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8078062400 con 0x7f80600a3c30 2024-04-13T05:17:55.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.983+0000 7f8076000640 1 -- 172.21.15.57:0/2367582287 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340546 (secure 0 0 0) 0x7f8078062400 con 0x7f80600a3c30 2024-04-13T05:17:55.990 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:55.992 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:17:55.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80600a3c30 msgr2=0x7f80600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80600a3c30 0x7f80600a60f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80640096c0 tx=0x7f8064000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 msgr2=0x7f807819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 0x7f807819e860 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f806c00d770 tx=0x7f806c00dc30 comp rx=0 tx=0).stop 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 shutdown_connections 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80600a3c30 0x7f80600a60f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078078560 0x7f80781a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8078077bc0 0x7f807819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 --2- 172.21.15.57:0/2367582287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8078076980 0x7f807819e860 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.989+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 >> 172.21.15.57:0/2367582287 conn(0x7f80780ff7b0 msgr2=0x7f8078100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.990+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 shutdown_connections 2024-04-13T05:17:55.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:55.990+0000 7f8080000640 1 -- 172.21.15.57:0/2367582287 wait complete. 2024-04-13T05:17:56.013 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-13T05:17:55.674518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663411,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1070,"num_read_kb":661,"num_write":363,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367948,"kb_used_data":14636,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573876,"statfs":{"total":1151940427776,"available":1151563649024,"internally_reserved":0,"allocated":14987264,"data_stored":8281919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004497"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600757+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:16:41.600757+0000","last_peered":"2024-04-13T05:16:41.600757+0000","last_clean":"2024-04-13T05:16:41.600757+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:16:41.600757+0000","last_undegraded":"2024-04-13T05:16:41.600757+0000","last_fullsized":"2024-04-13T05:16:41.600757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595560+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:16:41.595560+0000","last_peered":"2024-04-13T05:16:41.595560+0000","last_clean":"2024-04-13T05:16:41.595560+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:16:41.595560+0000","last_undegraded":"2024-04-13T05:16:41.595560+0000","last_fullsized":"2024-04-13T05:16:41.595560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601071+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:16:41.601071+0000","last_peered":"2024-04-13T05:16:41.601071+0000","last_clean":"2024-04-13T05:16:41.601071+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:16:41.601071+0000","last_undegraded":"2024-04-13T05:16:41.601071+0000","last_fullsized":"2024-04-13T05:16:41.601071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.573856+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:16:41.573856+0000","last_peered":"2024-04-13T05:16:41.573856+0000","last_clean":"2024-04-13T05:16:41.573856+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:16:41.573856+0000","last_undegraded":"2024-04-13T05:16:41.573856+0000","last_fullsized":"2024-04-13T05:16:41.573856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.580277+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:16:41.580277+0000","last_peered":"2024-04-13T05:16:41.580277+0000","last_clean":"2024-04-13T05:16:41.580277+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:16:41.580277+0000","last_undegraded":"2024-04-13T05:16:41.580277+0000","last_fullsized":"2024-04-13T05:16:41.580277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591873+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:16:41.591873+0000","last_peered":"2024-04-13T05:16:41.591873+0000","last_clean":"2024-04-13T05:16:41.591873+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:16:41.591873+0000","last_undegraded":"2024-04-13T05:16:41.591873+0000","last_fullsized":"2024-04-13T05:16:41.591873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593324+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:16:41.593324+0000","last_peered":"2024-04-13T05:16:41.593324+0000","last_clean":"2024-04-13T05:16:41.593324+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:16:41.593324+0000","last_undegraded":"2024-04-13T05:16:41.593324+0000","last_fullsized":"2024-04-13T05:16:41.593324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597594+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:16:41.597594+0000","last_peered":"2024-04-13T05:16:41.597594+0000","last_clean":"2024-04-13T05:16:41.597594+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:16:41.597594+0000","last_undegraded":"2024-04-13T05:16:41.597594+0000","last_fullsized":"2024-04-13T05:16:41.597594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600436+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:16:41.600436+0000","last_peered":"2024-04-13T05:16:41.600436+0000","last_clean":"2024-04-13T05:16:41.600436+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:16:41.600436+0000","last_undegraded":"2024-04-13T05:16:41.600436+0000","last_fullsized":"2024-04-13T05:16:41.600436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596758+0000","last_change":"2024-04-13T05:16:14.217489+0000","last_active":"2024-04-13T05:16:41.596758+0000","last_peered":"2024-04-13T05:16:41.596758+0000","last_clean":"2024-04-13T05:16:41.596758+0000","last_became_active":"2024-04-13T05:16:14.216651+0000","last_became_peered":"2024-04-13T05:16:14.216651+0000","last_unstale":"2024-04-13T05:16:41.596758+0000","last_undegraded":"2024-04-13T05:16:41.596758+0000","last_fullsized":"2024-04-13T05:16:41.596758+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:55:10.808460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601929+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:16:41.601929+0000","last_peered":"2024-04-13T05:16:41.601929+0000","last_clean":"2024-04-13T05:16:41.601929+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:16:41.601929+0000","last_undegraded":"2024-04-13T05:16:41.601929+0000","last_fullsized":"2024-04-13T05:16:41.601929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587123+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:16:41.587123+0000","last_peered":"2024-04-13T05:16:41.587123+0000","last_clean":"2024-04-13T05:16:41.587123+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:16:41.587123+0000","last_undegraded":"2024-04-13T05:16:41.587123+0000","last_fullsized":"2024-04-13T05:16:41.587123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590389+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:16:41.590389+0000","last_peered":"2024-04-13T05:16:41.590389+0000","last_clean":"2024-04-13T05:16:41.590389+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:16:41.590389+0000","last_undegraded":"2024-04-13T05:16:41.590389+0000","last_fullsized":"2024-04-13T05:16:41.590389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597106+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:16:41.597106+0000","last_peered":"2024-04-13T05:16:41.597106+0000","last_clean":"2024-04-13T05:16:41.597106+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:16:41.597106+0000","last_undegraded":"2024-04-13T05:16:41.597106+0000","last_fullsized":"2024-04-13T05:16:41.597106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591785+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:16:41.591785+0000","last_peered":"2024-04-13T05:16:41.591785+0000","last_clean":"2024-04-13T05:16:41.591785+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:16:41.591785+0000","last_undegraded":"2024-04-13T05:16:41.591785+0000","last_fullsized":"2024-04-13T05:16:41.591785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601120+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:16:41.601120+0000","last_peered":"2024-04-13T05:16:41.601120+0000","last_clean":"2024-04-13T05:16:41.601120+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:16:41.601120+0000","last_undegraded":"2024-04-13T05:16:41.601120+0000","last_fullsized":"2024-04-13T05:16:41.601120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600809+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:16:41.600809+0000","last_peered":"2024-04-13T05:16:41.600809+0000","last_clean":"2024-04-13T05:16:41.600809+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:16:41.600809+0000","last_undegraded":"2024-04-13T05:16:41.600809+0000","last_fullsized":"2024-04-13T05:16:41.600809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593257+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:16:41.593257+0000","last_peered":"2024-04-13T05:16:41.593257+0000","last_clean":"2024-04-13T05:16:41.593257+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:16:41.593257+0000","last_undegraded":"2024-04-13T05:16:41.593257+0000","last_fullsized":"2024-04-13T05:16:41.593257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574834+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:16:41.574834+0000","last_peered":"2024-04-13T05:16:41.574834+0000","last_clean":"2024-04-13T05:16:41.574834+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:16:41.574834+0000","last_undegraded":"2024-04-13T05:16:41.574834+0000","last_fullsized":"2024-04-13T05:16:41.574834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207323+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:16:14.207323+0000","last_peered":"2024-04-13T05:16:14.207323+0000","last_clean":"2024-04-13T05:16:14.207323+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:16:14.207323+0000","last_undegraded":"2024-04-13T05:16:14.207323+0000","last_fullsized":"2024-04-13T05:16:14.207323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208557+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:16:14.208557+0000","last_peered":"2024-04-13T05:16:14.208557+0000","last_clean":"2024-04-13T05:16:14.208557+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:16:14.208557+0000","last_undegraded":"2024-04-13T05:16:14.208557+0000","last_fullsized":"2024-04-13T05:16:14.208557+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596482+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:16:41.596482+0000","last_peered":"2024-04-13T05:16:41.596482+0000","last_clean":"2024-04-13T05:16:41.596482+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:16:41.596482+0000","last_undegraded":"2024-04-13T05:16:41.596482+0000","last_fullsized":"2024-04-13T05:16:41.596482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207004+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:16:14.207004+0000","last_peered":"2024-04-13T05:16:14.207004+0000","last_clean":"2024-04-13T05:16:14.207004+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:16:14.207004+0000","last_undegraded":"2024-04-13T05:16:14.207004+0000","last_fullsized":"2024-04-13T05:16:14.207004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206186+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:16:14.206186+0000","last_peered":"2024-04-13T05:16:14.206186+0000","last_clean":"2024-04-13T05:16:14.206186+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:16:14.206186+0000","last_undegraded":"2024-04-13T05:16:14.206186+0000","last_fullsized":"2024-04-13T05:16:14.206186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600939+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:16:41.600939+0000","last_peered":"2024-04-13T05:16:41.600939+0000","last_clean":"2024-04-13T05:16:41.600939+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:16:41.600939+0000","last_undegraded":"2024-04-13T05:16:41.600939+0000","last_fullsized":"2024-04-13T05:16:41.600939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207399+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:16:14.207399+0000","last_peered":"2024-04-13T05:16:14.207399+0000","last_clean":"2024-04-13T05:16:14.207399+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:16:14.207399+0000","last_undegraded":"2024-04-13T05:16:14.207399+0000","last_fullsized":"2024-04-13T05:16:14.207399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215166+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:16:14.215166+0000","last_peered":"2024-04-13T05:16:14.215166+0000","last_clean":"2024-04-13T05:16:14.215166+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:16:14.215166+0000","last_undegraded":"2024-04-13T05:16:14.215166+0000","last_fullsized":"2024-04-13T05:16:14.215166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600165+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:16:41.600165+0000","last_peered":"2024-04-13T05:16:41.600165+0000","last_clean":"2024-04-13T05:16:41.600165+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:16:41.600165+0000","last_undegraded":"2024-04-13T05:16:41.600165+0000","last_fullsized":"2024-04-13T05:16:41.600165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207127+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:16:14.207127+0000","last_peered":"2024-04-13T05:16:14.207127+0000","last_clean":"2024-04-13T05:16:14.207127+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:16:14.207127+0000","last_undegraded":"2024-04-13T05:16:14.207127+0000","last_fullsized":"2024-04-13T05:16:14.207127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711648+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:16:14.711648+0000","last_peered":"2024-04-13T05:16:14.711648+0000","last_clean":"2024-04-13T05:16:14.711648+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:16:14.711648+0000","last_undegraded":"2024-04-13T05:16:14.711648+0000","last_fullsized":"2024-04-13T05:16:14.711648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601248+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:16:41.601248+0000","last_peered":"2024-04-13T05:16:41.601248+0000","last_clean":"2024-04-13T05:16:41.601248+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:16:41.601248+0000","last_undegraded":"2024-04-13T05:16:41.601248+0000","last_fullsized":"2024-04-13T05:16:41.601248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208068+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:16:14.208068+0000","last_peered":"2024-04-13T05:16:14.208068+0000","last_clean":"2024-04-13T05:16:14.208068+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:16:14.208068+0000","last_undegraded":"2024-04-13T05:16:14.208068+0000","last_fullsized":"2024-04-13T05:16:14.208068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:51.796889+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:17:51.796889+0000","last_peered":"2024-04-13T05:17:51.796889+0000","last_clean":"2024-04-13T05:17:51.796889+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:17:51.796889+0000","last_undegraded":"2024-04-13T05:17:51.796889+0000","last_fullsized":"2024-04-13T05:17:51.796889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590755+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:16:41.590755+0000","last_peered":"2024-04-13T05:16:41.590755+0000","last_clean":"2024-04-13T05:16:41.590755+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:16:41.590755+0000","last_undegraded":"2024-04-13T05:16:41.590755+0000","last_fullsized":"2024-04-13T05:16:41.590755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207679+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:16:14.207679+0000","last_peered":"2024-04-13T05:16:14.207679+0000","last_clean":"2024-04-13T05:16:14.207679+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:16:14.207679+0000","last_undegraded":"2024-04-13T05:16:14.207679+0000","last_fullsized":"2024-04-13T05:16:14.207679+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.212785+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:16:14.212785+0000","last_peered":"2024-04-13T05:16:14.212785+0000","last_clean":"2024-04-13T05:16:14.212785+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:16:14.212785+0000","last_undegraded":"2024-04-13T05:16:14.212785+0000","last_fullsized":"2024-04-13T05:16:14.212785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597193+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:16:41.597193+0000","last_peered":"2024-04-13T05:16:41.597193+0000","last_clean":"2024-04-13T05:16:41.597193+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:16:41.597193+0000","last_undegraded":"2024-04-13T05:16:41.597193+0000","last_fullsized":"2024-04-13T05:16:41.597193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215483+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:16:14.215483+0000","last_peered":"2024-04-13T05:16:14.215483+0000","last_clean":"2024-04-13T05:16:14.215483+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:16:14.215483+0000","last_undegraded":"2024-04-13T05:16:14.215483+0000","last_fullsized":"2024-04-13T05:16:14.215483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.500836+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:16:14.500836+0000","last_peered":"2024-04-13T05:16:14.500836+0000","last_clean":"2024-04-13T05:16:14.500836+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:16:14.500836+0000","last_undegraded":"2024-04-13T05:16:14.500836+0000","last_fullsized":"2024-04-13T05:16:14.500836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595783+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:16:41.595783+0000","last_peered":"2024-04-13T05:16:41.595783+0000","last_clean":"2024-04-13T05:16:41.595783+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:16:41.595783+0000","last_undegraded":"2024-04-13T05:16:41.595783+0000","last_fullsized":"2024-04-13T05:16:41.595783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548990+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:16:14.548990+0000","last_peered":"2024-04-13T05:16:14.548990+0000","last_clean":"2024-04-13T05:16:14.548990+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:16:14.548990+0000","last_undegraded":"2024-04-13T05:16:14.548990+0000","last_fullsized":"2024-04-13T05:16:14.548990+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488274+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:16:14.488274+0000","last_peered":"2024-04-13T05:16:14.488274+0000","last_clean":"2024-04-13T05:16:14.488274+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:16:14.488274+0000","last_undegraded":"2024-04-13T05:16:14.488274+0000","last_fullsized":"2024-04-13T05:16:14.488274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595439+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:16:41.595439+0000","last_peered":"2024-04-13T05:16:41.595439+0000","last_clean":"2024-04-13T05:16:41.595439+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:16:41.595439+0000","last_undegraded":"2024-04-13T05:16:41.595439+0000","last_fullsized":"2024-04-13T05:16:41.595439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205352+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:16:14.205352+0000","last_peered":"2024-04-13T05:16:14.205352+0000","last_clean":"2024-04-13T05:16:14.205352+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:16:14.205352+0000","last_undegraded":"2024-04-13T05:16:14.205352+0000","last_fullsized":"2024-04-13T05:16:14.205352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":188,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:52.798933+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:17:52.798933+0000","last_peered":"2024-04-13T05:17:52.798933+0000","last_clean":"2024-04-13T05:17:52.798933+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:17:52.798933+0000","last_undegraded":"2024-04-13T05:17:52.798933+0000","last_fullsized":"2024-04-13T05:17:52.798933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599498+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:16:41.599498+0000","last_peered":"2024-04-13T05:16:41.599498+0000","last_clean":"2024-04-13T05:16:41.599498+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:16:41.599498+0000","last_undegraded":"2024-04-13T05:16:41.599498+0000","last_fullsized":"2024-04-13T05:16:41.599498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207859+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:16:14.207859+0000","last_peered":"2024-04-13T05:16:14.207859+0000","last_clean":"2024-04-13T05:16:14.207859+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:16:14.207859+0000","last_undegraded":"2024-04-13T05:16:14.207859+0000","last_fullsized":"2024-04-13T05:16:14.207859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712372+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:16:14.712372+0000","last_peered":"2024-04-13T05:16:14.712372+0000","last_clean":"2024-04-13T05:16:14.712372+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:16:14.712372+0000","last_undegraded":"2024-04-13T05:16:14.712372+0000","last_fullsized":"2024-04-13T05:16:14.712372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590058+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:16:41.590058+0000","last_peered":"2024-04-13T05:16:41.590058+0000","last_clean":"2024-04-13T05:16:41.590058+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:16:41.590058+0000","last_undegraded":"2024-04-13T05:16:41.590058+0000","last_fullsized":"2024-04-13T05:16:41.590058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207958+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:16:14.207958+0000","last_peered":"2024-04-13T05:16:14.207958+0000","last_clean":"2024-04-13T05:16:14.207958+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:16:14.207958+0000","last_undegraded":"2024-04-13T05:16:14.207958+0000","last_fullsized":"2024-04-13T05:16:14.207958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489335+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:16:14.489335+0000","last_peered":"2024-04-13T05:16:14.489335+0000","last_clean":"2024-04-13T05:16:14.489335+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:16:14.489335+0000","last_undegraded":"2024-04-13T05:16:14.489335+0000","last_fullsized":"2024-04-13T05:16:14.489335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601469+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:16:41.601469+0000","last_peered":"2024-04-13T05:16:41.601469+0000","last_clean":"2024-04-13T05:16:41.601469+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:16:41.601469+0000","last_undegraded":"2024-04-13T05:16:41.601469+0000","last_fullsized":"2024-04-13T05:16:41.601469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208081+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:16:14.208081+0000","last_peered":"2024-04-13T05:16:14.208081+0000","last_clean":"2024-04-13T05:16:14.208081+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:16:14.208081+0000","last_undegraded":"2024-04-13T05:16:14.208081+0000","last_fullsized":"2024-04-13T05:16:14.208081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":386,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:53.802110+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:17:53.802110+0000","last_peered":"2024-04-13T05:17:53.802110+0000","last_clean":"2024-04-13T05:17:53.802110+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:17:53.802110+0000","last_undegraded":"2024-04-13T05:17:53.802110+0000","last_fullsized":"2024-04-13T05:17:53.802110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67681,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":226,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595726+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:16:41.595726+0000","last_peered":"2024-04-13T05:16:41.595726+0000","last_clean":"2024-04-13T05:16:41.595726+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:16:41.595726+0000","last_undegraded":"2024-04-13T05:16:41.595726+0000","last_fullsized":"2024-04-13T05:16:41.595726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.490155+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:16:14.490155+0000","last_peered":"2024-04-13T05:16:14.490155+0000","last_clean":"2024-04-13T05:16:14.490155+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:16:14.490155+0000","last_undegraded":"2024-04-13T05:16:14.490155+0000","last_fullsized":"2024-04-13T05:16:14.490155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207832+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:16:14.207832+0000","last_peered":"2024-04-13T05:16:14.207832+0000","last_clean":"2024-04-13T05:16:14.207832+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:16:14.207832+0000","last_undegraded":"2024-04-13T05:16:14.207832+0000","last_fullsized":"2024-04-13T05:16:14.207832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591125+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:16:41.591125+0000","last_peered":"2024-04-13T05:16:41.591125+0000","last_clean":"2024-04-13T05:16:41.591125+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:16:41.591125+0000","last_undegraded":"2024-04-13T05:16:41.591125+0000","last_fullsized":"2024-04-13T05:16:41.591125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214510+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:16:14.214510+0000","last_peered":"2024-04-13T05:16:14.214510+0000","last_clean":"2024-04-13T05:16:14.214510+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:16:14.214510+0000","last_undegraded":"2024-04-13T05:16:14.214510+0000","last_fullsized":"2024-04-13T05:16:14.214510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488900+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:16:14.488900+0000","last_peered":"2024-04-13T05:16:14.488900+0000","last_clean":"2024-04-13T05:16:14.488900+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:16:14.488900+0000","last_undegraded":"2024-04-13T05:16:14.488900+0000","last_fullsized":"2024-04-13T05:16:14.488900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596593+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:16:41.596593+0000","last_peered":"2024-04-13T05:16:41.596593+0000","last_clean":"2024-04-13T05:16:41.596593+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:16:41.596593+0000","last_undegraded":"2024-04-13T05:16:41.596593+0000","last_fullsized":"2024-04-13T05:16:41.596593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208701+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:16:14.208701+0000","last_peered":"2024-04-13T05:16:14.208701+0000","last_clean":"2024-04-13T05:16:14.208701+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:16:14.208701+0000","last_undegraded":"2024-04-13T05:16:14.208701+0000","last_fullsized":"2024-04-13T05:16:14.208701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:51.804836+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:17:51.804836+0000","last_peered":"2024-04-13T05:17:51.804836+0000","last_clean":"2024-04-13T05:17:51.804836+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:17:51.804836+0000","last_undegraded":"2024-04-13T05:17:51.804836+0000","last_fullsized":"2024-04-13T05:17:51.804836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593652+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:16:41.593652+0000","last_peered":"2024-04-13T05:16:41.593652+0000","last_clean":"2024-04-13T05:16:41.593652+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:16:41.593652+0000","last_undegraded":"2024-04-13T05:16:41.593652+0000","last_fullsized":"2024-04-13T05:16:41.593652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.502070+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:16:14.502070+0000","last_peered":"2024-04-13T05:16:14.502070+0000","last_clean":"2024-04-13T05:16:14.502070+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:16:14.502070+0000","last_undegraded":"2024-04-13T05:16:14.502070+0000","last_fullsized":"2024-04-13T05:16:14.502070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709866+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:16:14.709866+0000","last_peered":"2024-04-13T05:16:14.709866+0000","last_clean":"2024-04-13T05:16:14.709866+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:16:14.709866+0000","last_undegraded":"2024-04-13T05:16:14.709866+0000","last_fullsized":"2024-04-13T05:16:14.709866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600587+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:16:41.600587+0000","last_peered":"2024-04-13T05:16:41.600587+0000","last_clean":"2024-04-13T05:16:41.600587+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:16:41.600587+0000","last_undegraded":"2024-04-13T05:16:41.600587+0000","last_fullsized":"2024-04-13T05:16:41.600587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488458+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:16:14.488458+0000","last_peered":"2024-04-13T05:16:14.488458+0000","last_clean":"2024-04-13T05:16:14.488458+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:16:14.488458+0000","last_undegraded":"2024-04-13T05:16:14.488458+0000","last_fullsized":"2024-04-13T05:16:14.488458+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710704+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:16:14.710704+0000","last_peered":"2024-04-13T05:16:14.710704+0000","last_clean":"2024-04-13T05:16:14.710704+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:16:14.710704+0000","last_undegraded":"2024-04-13T05:16:14.710704+0000","last_fullsized":"2024-04-13T05:16:14.710704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584084+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:16:41.584084+0000","last_peered":"2024-04-13T05:16:41.584084+0000","last_clean":"2024-04-13T05:16:41.584084+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:16:41.584084+0000","last_undegraded":"2024-04-13T05:16:41.584084+0000","last_fullsized":"2024-04-13T05:16:41.584084+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711265+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:16:14.711265+0000","last_peered":"2024-04-13T05:16:14.711265+0000","last_clean":"2024-04-13T05:16:14.711265+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:16:14.711265+0000","last_undegraded":"2024-04-13T05:16:14.711265+0000","last_fullsized":"2024-04-13T05:16:14.711265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208277+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:16:14.208277+0000","last_peered":"2024-04-13T05:16:14.208277+0000","last_clean":"2024-04-13T05:16:14.208277+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:16:14.208277+0000","last_undegraded":"2024-04-13T05:16:14.208277+0000","last_fullsized":"2024-04-13T05:16:14.208277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597318+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:16:41.597318+0000","last_peered":"2024-04-13T05:16:41.597318+0000","last_clean":"2024-04-13T05:16:41.597318+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:16:41.597318+0000","last_undegraded":"2024-04-13T05:16:41.597318+0000","last_fullsized":"2024-04-13T05:16:41.597318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501299+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:16:14.501299+0000","last_peered":"2024-04-13T05:16:14.501299+0000","last_clean":"2024-04-13T05:16:14.501299+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:16:14.501299+0000","last_undegraded":"2024-04-13T05:16:14.501299+0000","last_fullsized":"2024-04-13T05:16:14.501299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711835+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:16:14.711835+0000","last_peered":"2024-04-13T05:16:14.711835+0000","last_clean":"2024-04-13T05:16:14.711835+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:16:14.711835+0000","last_undegraded":"2024-04-13T05:16:14.711835+0000","last_fullsized":"2024-04-13T05:16:14.711835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591498+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:16:41.591498+0000","last_peered":"2024-04-13T05:16:41.591498+0000","last_clean":"2024-04-13T05:16:41.591498+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:16:41.591498+0000","last_undegraded":"2024-04-13T05:16:41.591498+0000","last_fullsized":"2024-04-13T05:16:41.591498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501784+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:16:14.501784+0000","last_peered":"2024-04-13T05:16:14.501784+0000","last_clean":"2024-04-13T05:16:14.501784+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:16:14.501784+0000","last_undegraded":"2024-04-13T05:16:14.501784+0000","last_fullsized":"2024-04-13T05:16:14.501784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587661+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:16:41.587661+0000","last_peered":"2024-04-13T05:16:41.587661+0000","last_clean":"2024-04-13T05:16:41.587661+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:16:41.587661+0000","last_undegraded":"2024-04-13T05:16:41.587661+0000","last_fullsized":"2024-04-13T05:16:41.587661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":190,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:52.806787+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:17:52.806787+0000","last_peered":"2024-04-13T05:17:52.806787+0000","last_clean":"2024-04-13T05:17:52.806787+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:17:52.806787+0000","last_undegraded":"2024-04-13T05:17:52.806787+0000","last_fullsized":"2024-04-13T05:17:52.806787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712596+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:16:14.712596+0000","last_peered":"2024-04-13T05:16:14.712596+0000","last_clean":"2024-04-13T05:16:14.712596+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:16:14.712596+0000","last_undegraded":"2024-04-13T05:16:14.712596+0000","last_fullsized":"2024-04-13T05:16:14.712596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.602040+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:16:41.602040+0000","last_peered":"2024-04-13T05:16:41.602040+0000","last_clean":"2024-04-13T05:16:41.602040+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:16:41.602040+0000","last_undegraded":"2024-04-13T05:16:41.602040+0000","last_fullsized":"2024-04-13T05:16:41.602040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207400+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:16:14.207400+0000","last_peered":"2024-04-13T05:16:14.207400+0000","last_clean":"2024-04-13T05:16:14.207400+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:16:14.207400+0000","last_undegraded":"2024-04-13T05:16:14.207400+0000","last_fullsized":"2024-04-13T05:16:14.207400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711063+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:16:14.711063+0000","last_peered":"2024-04-13T05:16:14.711063+0000","last_clean":"2024-04-13T05:16:14.711063+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:16:14.711063+0000","last_undegraded":"2024-04-13T05:16:14.711063+0000","last_fullsized":"2024-04-13T05:16:14.711063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.490158+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:16:14.490158+0000","last_peered":"2024-04-13T05:16:14.490158+0000","last_clean":"2024-04-13T05:16:14.490158+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:16:14.490158+0000","last_undegraded":"2024-04-13T05:16:14.490158+0000","last_fullsized":"2024-04-13T05:16:14.490158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574537+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:16:41.574537+0000","last_peered":"2024-04-13T05:16:41.574537+0000","last_clean":"2024-04-13T05:16:41.574537+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:16:41.574537+0000","last_undegraded":"2024-04-13T05:16:41.574537+0000","last_fullsized":"2024-04-13T05:16:41.574537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711427+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:16:14.711427+0000","last_peered":"2024-04-13T05:16:14.711427+0000","last_clean":"2024-04-13T05:16:14.711427+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:16:14.711427+0000","last_undegraded":"2024-04-13T05:16:14.711427+0000","last_fullsized":"2024-04-13T05:16:14.711427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214710+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:16:14.214710+0000","last_peered":"2024-04-13T05:16:14.214710+0000","last_clean":"2024-04-13T05:16:14.214710+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:16:14.214710+0000","last_undegraded":"2024-04-13T05:16:14.214710+0000","last_fullsized":"2024-04-13T05:16:14.214710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711158+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:16:14.711158+0000","last_peered":"2024-04-13T05:16:14.711158+0000","last_clean":"2024-04-13T05:16:14.711158+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:16:14.711158+0000","last_undegraded":"2024-04-13T05:16:14.711158+0000","last_fullsized":"2024-04-13T05:16:14.711158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574744+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:16:41.574744+0000","last_peered":"2024-04-13T05:16:41.574744+0000","last_clean":"2024-04-13T05:16:41.574744+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:16:41.574744+0000","last_undegraded":"2024-04-13T05:16:41.574744+0000","last_fullsized":"2024-04-13T05:16:41.574744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205197+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:16:14.205197+0000","last_peered":"2024-04-13T05:16:14.205197+0000","last_clean":"2024-04-13T05:16:14.205197+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:16:14.205197+0000","last_undegraded":"2024-04-13T05:16:14.205197+0000","last_fullsized":"2024-04-13T05:16:14.205197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:05.240813+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:17:05.240813+0000","last_peered":"2024-04-13T05:17:05.240813+0000","last_clean":"2024-04-13T05:17:05.240813+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:17:05.240813+0000","last_undegraded":"2024-04-13T05:17:05.240813+0000","last_fullsized":"2024-04-13T05:17:05.240813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595268+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:16:41.595268+0000","last_peered":"2024-04-13T05:16:41.595268+0000","last_clean":"2024-04-13T05:16:41.595268+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:16:41.595268+0000","last_undegraded":"2024-04-13T05:16:41.595268+0000","last_fullsized":"2024-04-13T05:16:41.595268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207010+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:16:14.207010+0000","last_peered":"2024-04-13T05:16:14.207010+0000","last_clean":"2024-04-13T05:16:14.207010+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:16:14.207010+0000","last_undegraded":"2024-04-13T05:16:14.207010+0000","last_fullsized":"2024-04-13T05:16:14.207010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.209094+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:16:14.209094+0000","last_peered":"2024-04-13T05:16:14.209094+0000","last_clean":"2024-04-13T05:16:14.209094+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:16:14.209094+0000","last_undegraded":"2024-04-13T05:16:14.209094+0000","last_fullsized":"2024-04-13T05:16:14.209094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596065+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:16:41.596065+0000","last_peered":"2024-04-13T05:16:41.596065+0000","last_clean":"2024-04-13T05:16:41.596065+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:16:41.596065+0000","last_undegraded":"2024-04-13T05:16:41.596065+0000","last_fullsized":"2024-04-13T05:16:41.596065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208632+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:16:14.208632+0000","last_peered":"2024-04-13T05:16:14.208632+0000","last_clean":"2024-04-13T05:16:14.208632+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:16:14.208632+0000","last_undegraded":"2024-04-13T05:16:14.208632+0000","last_fullsized":"2024-04-13T05:16:14.208632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709476+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:16:14.709476+0000","last_peered":"2024-04-13T05:16:14.709476+0000","last_clean":"2024-04-13T05:16:14.709476+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:16:14.709476+0000","last_undegraded":"2024-04-13T05:16:14.709476+0000","last_fullsized":"2024-04-13T05:16:14.709476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586590+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:16:41.586590+0000","last_peered":"2024-04-13T05:16:41.586590+0000","last_clean":"2024-04-13T05:16:41.586590+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:16:41.586590+0000","last_undegraded":"2024-04-13T05:16:41.586590+0000","last_fullsized":"2024-04-13T05:16:41.586590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208596+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:16:14.208596+0000","last_peered":"2024-04-13T05:16:14.208596+0000","last_clean":"2024-04-13T05:16:14.208596+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:16:14.208596+0000","last_undegraded":"2024-04-13T05:16:14.208596+0000","last_fullsized":"2024-04-13T05:16:14.208596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205469+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:16:14.205469+0000","last_peered":"2024-04-13T05:16:14.205469+0000","last_clean":"2024-04-13T05:16:14.205469+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:16:14.205469+0000","last_undegraded":"2024-04-13T05:16:14.205469+0000","last_fullsized":"2024-04-13T05:16:14.205469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591578+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:16:41.591578+0000","last_peered":"2024-04-13T05:16:41.591578+0000","last_clean":"2024-04-13T05:16:41.591578+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:16:41.591578+0000","last_undegraded":"2024-04-13T05:16:41.591578+0000","last_fullsized":"2024-04-13T05:16:41.591578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207219+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:16:14.207219+0000","last_peered":"2024-04-13T05:16:14.207219+0000","last_clean":"2024-04-13T05:16:14.207219+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:16:14.207219+0000","last_undegraded":"2024-04-13T05:16:14.207219+0000","last_fullsized":"2024-04-13T05:16:14.207219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:25.449178+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:17:25.449178+0000","last_peered":"2024-04-13T05:17:25.449178+0000","last_clean":"2024-04-13T05:17:25.449178+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:17:25.449178+0000","last_undegraded":"2024-04-13T05:17:25.449178+0000","last_fullsized":"2024-04-13T05:17:25.449178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597776+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:16:41.597776+0000","last_peered":"2024-04-13T05:16:41.597776+0000","last_clean":"2024-04-13T05:16:41.597776+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:16:41.597776+0000","last_undegraded":"2024-04-13T05:16:41.597776+0000","last_fullsized":"2024-04-13T05:16:41.597776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208228+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:16:14.208228+0000","last_peered":"2024-04-13T05:16:14.208228+0000","last_clean":"2024-04-13T05:16:14.208228+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:16:14.208228+0000","last_undegraded":"2024-04-13T05:16:14.208228+0000","last_fullsized":"2024-04-13T05:16:14.208228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711142+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:16:14.711142+0000","last_peered":"2024-04-13T05:16:14.711142+0000","last_clean":"2024-04-13T05:16:14.711142+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:16:14.711142+0000","last_undegraded":"2024-04-13T05:16:14.711142+0000","last_fullsized":"2024-04-13T05:16:14.711142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.602016+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:16:41.602016+0000","last_peered":"2024-04-13T05:16:41.602016+0000","last_clean":"2024-04-13T05:16:41.602016+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:16:41.602016+0000","last_undegraded":"2024-04-13T05:16:41.602016+0000","last_fullsized":"2024-04-13T05:16:41.602016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709227+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:16:14.709227+0000","last_peered":"2024-04-13T05:16:14.709227+0000","last_clean":"2024-04-13T05:16:14.709227+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:16:14.709227+0000","last_undegraded":"2024-04-13T05:16:14.709227+0000","last_fullsized":"2024-04-13T05:16:14.709227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208151+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:16:14.208151+0000","last_peered":"2024-04-13T05:16:14.208151+0000","last_clean":"2024-04-13T05:16:14.208151+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:16:14.208151+0000","last_undegraded":"2024-04-13T05:16:14.208151+0000","last_fullsized":"2024-04-13T05:16:14.208151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601798+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:16:41.601798+0000","last_peered":"2024-04-13T05:16:41.601798+0000","last_clean":"2024-04-13T05:16:41.601798+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:16:41.601798+0000","last_undegraded":"2024-04-13T05:16:41.601798+0000","last_fullsized":"2024-04-13T05:16:41.601798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548391+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:16:14.548391+0000","last_peered":"2024-04-13T05:16:14.548391+0000","last_clean":"2024-04-13T05:16:14.548391+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:16:14.548391+0000","last_undegraded":"2024-04-13T05:16:14.548391+0000","last_fullsized":"2024-04-13T05:16:14.548391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489672+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:16:14.489672+0000","last_peered":"2024-04-13T05:16:14.489672+0000","last_clean":"2024-04-13T05:16:14.489672+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:16:14.489672+0000","last_undegraded":"2024-04-13T05:16:14.489672+0000","last_fullsized":"2024-04-13T05:16:14.489672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585990+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:16:41.585990+0000","last_peered":"2024-04-13T05:16:41.585990+0000","last_clean":"2024-04-13T05:16:41.585990+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:16:41.585990+0000","last_undegraded":"2024-04-13T05:16:41.585990+0000","last_fullsized":"2024-04-13T05:16:41.585990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.487926+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:16:14.487926+0000","last_peered":"2024-04-13T05:16:14.487926+0000","last_clean":"2024-04-13T05:16:14.487926+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:16:14.487926+0000","last_undegraded":"2024-04-13T05:16:14.487926+0000","last_fullsized":"2024-04-13T05:16:14.487926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501896+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:16:14.501896+0000","last_peered":"2024-04-13T05:16:14.501896+0000","last_clean":"2024-04-13T05:16:14.501896+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:16:14.501896+0000","last_undegraded":"2024-04-13T05:16:14.501896+0000","last_fullsized":"2024-04-13T05:16:14.501896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594768+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:16:41.594768+0000","last_peered":"2024-04-13T05:16:41.594768+0000","last_clean":"2024-04-13T05:16:41.594768+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:16:41.594768+0000","last_undegraded":"2024-04-13T05:16:41.594768+0000","last_fullsized":"2024-04-13T05:16:41.594768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584585+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:16:41.584585+0000","last_peered":"2024-04-13T05:16:41.584585+0000","last_clean":"2024-04-13T05:16:41.584585+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:16:41.584585+0000","last_undegraded":"2024-04-13T05:16:41.584585+0000","last_fullsized":"2024-04-13T05:16:41.584585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587156+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:16:41.587156+0000","last_peered":"2024-04-13T05:16:41.587156+0000","last_clean":"2024-04-13T05:16:41.587156+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:16:41.587156+0000","last_undegraded":"2024-04-13T05:16:41.587156+0000","last_fullsized":"2024-04-13T05:16:41.587156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.592009+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:16:41.592009+0000","last_peered":"2024-04-13T05:16:41.592009+0000","last_clean":"2024-04-13T05:16:41.592009+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:16:41.592009+0000","last_undegraded":"2024-04-13T05:16:41.592009+0000","last_fullsized":"2024-04-13T05:16:41.592009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591014+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:16:41.591014+0000","last_peered":"2024-04-13T05:16:41.591014+0000","last_clean":"2024-04-13T05:16:41.591014+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:16:41.591014+0000","last_undegraded":"2024-04-13T05:16:41.591014+0000","last_fullsized":"2024-04-13T05:16:41.591014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586928+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:16:41.586928+0000","last_peered":"2024-04-13T05:16:41.586928+0000","last_clean":"2024-04-13T05:16:41.586928+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:16:41.586928+0000","last_undegraded":"2024-04-13T05:16:41.586928+0000","last_fullsized":"2024-04-13T05:16:41.586928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571520+0000","last_change":"2024-04-13T05:16:14.714804+0000","last_active":"2024-04-13T05:16:41.571520+0000","last_peered":"2024-04-13T05:16:41.571520+0000","last_clean":"2024-04-13T05:16:41.571520+0000","last_became_active":"2024-04-13T05:16:14.713542+0000","last_became_peered":"2024-04-13T05:16:14.713542+0000","last_unstale":"2024-04-13T05:16:41.571520+0000","last_undegraded":"2024-04-13T05:16:41.571520+0000","last_fullsized":"2024-04-13T05:16:41.571520+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:21:57.719452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600113+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:16:41.600113+0000","last_peered":"2024-04-13T05:16:41.600113+0000","last_clean":"2024-04-13T05:16:41.600113+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:16:41.600113+0000","last_undegraded":"2024-04-13T05:16:41.600113+0000","last_fullsized":"2024-04-13T05:16:41.600113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.570115+0000","last_change":"2024-04-13T05:16:14.713320+0000","last_active":"2024-04-13T05:16:41.570115+0000","last_peered":"2024-04-13T05:16:41.570115+0000","last_clean":"2024-04-13T05:16:41.570115+0000","last_became_active":"2024-04-13T05:16:14.712507+0000","last_became_peered":"2024-04-13T05:16:14.712507+0000","last_unstale":"2024-04-13T05:16:41.570115+0000","last_undegraded":"2024-04-13T05:16:41.570115+0000","last_fullsized":"2024-04-13T05:16:41.570115+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:07.206767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.598327+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:16:41.598327+0000","last_peered":"2024-04-13T05:16:41.598327+0000","last_clean":"2024-04-13T05:16:41.598327+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:16:41.598327+0000","last_undegraded":"2024-04-13T05:16:41.598327+0000","last_fullsized":"2024-04-13T05:16:41.598327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596446+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:16:41.596446+0000","last_peered":"2024-04-13T05:16:41.596446+0000","last_clean":"2024-04-13T05:16:41.596446+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:16:41.596446+0000","last_undegraded":"2024-04-13T05:16:41.596446+0000","last_fullsized":"2024-04-13T05:16:41.596446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600578+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:16:41.600578+0000","last_peered":"2024-04-13T05:16:41.600578+0000","last_clean":"2024-04-13T05:16:41.600578+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:16:41.600578+0000","last_undegraded":"2024-04-13T05:16:41.600578+0000","last_fullsized":"2024-04-13T05:16:41.600578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593910+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:16:41.593910+0000","last_peered":"2024-04-13T05:16:41.593910+0000","last_clean":"2024-04-13T05:16:41.593910+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:16:41.593910+0000","last_undegraded":"2024-04-13T05:16:41.593910+0000","last_fullsized":"2024-04-13T05:16:41.593910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571621+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:16:41.571621+0000","last_peered":"2024-04-13T05:16:41.571621+0000","last_clean":"2024-04-13T05:16:41.571621+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:16:41.571621+0000","last_undegraded":"2024-04-13T05:16:41.571621+0000","last_fullsized":"2024-04-13T05:16:41.571621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72809,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":944,"num_read_kb":552,"num_write":180,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32900,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712252,"statfs":{"total":95995035648,"available":95961346048,"internally_reserved":0,"allocated":1765376,"data_stored":1182287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808898,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":530645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":525887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1123505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":589428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":523113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1155072,"data_stored":591047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1118068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:17:56.019 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T05:17:56.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:56 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/476964192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 -- 172.21.15.57:0/1160280684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109630 msgr2=0x7f2b58109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 --2- 172.21.15.57:0/1160280684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109630 0x7f2b58109ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009900 tx=0x7f2b4002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 -- 172.21.15.57:0/1160280684 shutdown_connections 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 --2- 172.21.15.57:0/1160280684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58109ff0 0x7f2b58110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 --2- 172.21.15.57:0/1160280684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109630 0x7f2b58109ab0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 --2- 172.21.15.57:0/1160280684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58108440 0x7f2b58108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 -- 172.21.15.57:0/1160280684 >> 172.21.15.57:0/1160280684 conn(0x7f2b58077400 msgr2=0x7f2b58079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.280+0000 7f2b57200640 1 -- 172.21.15.57:0/1160280684 shutdown_connections 2024-04-13T05:17:56.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.281+0000 7f2b57200640 1 -- 172.21.15.57:0/1160280684 wait complete. 2024-04-13T05:17:56.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.281+0000 7f2b57200640 1 Processor -- start 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 -- start start 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58108440 0x7f2b5819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58109630 0x7f2b5819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 0x7f2b581a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58113fd0 con 0x7f2b58109ff0 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58113e50 con 0x7f2b58108440 2024-04-13T05:17:56.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b57200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58114150 con 0x7f2b58109630 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b56800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 0x7f2b581a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b56800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 0x7f2b581a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33912/0 (socket says 172.21.15.57:33912) 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.282+0000 7f2b56800640 1 -- 172.21.15.57:0/1581436383 learned_addr learned my addr 172.21.15.57:0/1581436383 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b4d400640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58109630 0x7f2b5819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b55e00640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58108440 0x7f2b5819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b56800640 1 -- 172.21.15.57:0/1581436383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58109630 msgr2=0x7f2b5819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b56800640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58109630 0x7f2b5819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b56800640 1 -- 172.21.15.57:0/1581436383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58108440 msgr2=0x7f2b5819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b56800640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58108440 0x7f2b5819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b56800640 1 -- 172.21.15.57:0/1581436383 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b581a6250 con 0x7f2b58109ff0 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b4d400640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58109630 0x7f2b5819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b55e00640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58108440 0x7f2b5819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b56800640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 0x7f2b581a5be0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000eaa0 tx=0x7f2b5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000cdb0 con 0x7f2b58109ff0 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b50005020 con 0x7f2b58109ff0 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500116d0 con 0x7f2b58109ff0 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b581a6540 con 0x7f2b58109ff0 2024-04-13T05:17:56.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.283+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581a6a90 con 0x7f2b58109ff0 2024-04-13T05:17:56.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.284+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b5810f760 con 0x7f2b58109ff0 2024-04-13T05:17:56.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.287+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2b500051d0 con 0x7f2b58109ff0 2024-04-13T05:17:56.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.288+0000 7f2b4f400640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2b3c0a3c30 0x7f2b3c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.288+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2b500d09d0 con 0x7f2b58109ff0 2024-04-13T05:17:56.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.288+0000 7f2b55e00640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2b3c0a3c30 0x7f2b3c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.289+0000 7f2b55e00640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2b3c0a3c30 0x7f2b3c0a60f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b58109490 tx=0x7f2b48005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:56.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.292+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b50016030 con 0x7f2b58109ff0 2024-04-13T05:17:56.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.496+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b58062400 con 0x7f2b3c0a3c30 2024-04-13T05:17:56.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:56 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/476964192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:56.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.511+0000 7f2b4f400640 1 -- 172.21.15.57:0/1581436383 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340546 (secure 0 0 0) 0x7f2b58062400 con 0x7f2b3c0a3c30 2024-04-13T05:17:56.518 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:56.520 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.518+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2b3c0a3c30 msgr2=0x7f2b3c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2b3c0a3c30 0x7f2b3c0a60f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b58109490 tx=0x7f2b48005e70 comp rx=0 tx=0).stop 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 msgr2=0x7f2b581a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 0x7f2b581a5be0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000eaa0 tx=0x7f2b5000ef60 comp rx=0 tx=0).stop 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 shutdown_connections 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2b3c0a3c30 0x7f2b3c0a60f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b58109ff0 0x7f2b581a5be0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b58109630 0x7f2b5819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 --2- 172.21.15.57:0/1581436383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58108440 0x7f2b5819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 >> 172.21.15.57:0/1581436383 conn(0x7f2b58077400 msgr2=0x7f2b5810c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:56.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 shutdown_connections 2024-04-13T05:17:56.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.519+0000 7f2b57200640 1 -- 172.21.15.57:0/1581436383 wait complete. 2024-04-13T05:17:56.539 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-13T05:17:55.674518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663411,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1070,"num_read_kb":661,"num_write":363,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367948,"kb_used_data":14636,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573876,"statfs":{"total":1151940427776,"available":1151563649024,"internally_reserved":0,"allocated":14987264,"data_stored":8281919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004497"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600757+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:16:41.600757+0000","last_peered":"2024-04-13T05:16:41.600757+0000","last_clean":"2024-04-13T05:16:41.600757+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:16:41.600757+0000","last_undegraded":"2024-04-13T05:16:41.600757+0000","last_fullsized":"2024-04-13T05:16:41.600757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595560+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:16:41.595560+0000","last_peered":"2024-04-13T05:16:41.595560+0000","last_clean":"2024-04-13T05:16:41.595560+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:16:41.595560+0000","last_undegraded":"2024-04-13T05:16:41.595560+0000","last_fullsized":"2024-04-13T05:16:41.595560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601071+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:16:41.601071+0000","last_peered":"2024-04-13T05:16:41.601071+0000","last_clean":"2024-04-13T05:16:41.601071+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:16:41.601071+0000","last_undegraded":"2024-04-13T05:16:41.601071+0000","last_fullsized":"2024-04-13T05:16:41.601071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.573856+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:16:41.573856+0000","last_peered":"2024-04-13T05:16:41.573856+0000","last_clean":"2024-04-13T05:16:41.573856+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:16:41.573856+0000","last_undegraded":"2024-04-13T05:16:41.573856+0000","last_fullsized":"2024-04-13T05:16:41.573856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.580277+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:16:41.580277+0000","last_peered":"2024-04-13T05:16:41.580277+0000","last_clean":"2024-04-13T05:16:41.580277+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:16:41.580277+0000","last_undegraded":"2024-04-13T05:16:41.580277+0000","last_fullsized":"2024-04-13T05:16:41.580277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591873+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:16:41.591873+0000","last_peered":"2024-04-13T05:16:41.591873+0000","last_clean":"2024-04-13T05:16:41.591873+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:16:41.591873+0000","last_undegraded":"2024-04-13T05:16:41.591873+0000","last_fullsized":"2024-04-13T05:16:41.591873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593324+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:16:41.593324+0000","last_peered":"2024-04-13T05:16:41.593324+0000","last_clean":"2024-04-13T05:16:41.593324+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:16:41.593324+0000","last_undegraded":"2024-04-13T05:16:41.593324+0000","last_fullsized":"2024-04-13T05:16:41.593324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597594+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:16:41.597594+0000","last_peered":"2024-04-13T05:16:41.597594+0000","last_clean":"2024-04-13T05:16:41.597594+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:16:41.597594+0000","last_undegraded":"2024-04-13T05:16:41.597594+0000","last_fullsized":"2024-04-13T05:16:41.597594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600436+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:16:41.600436+0000","last_peered":"2024-04-13T05:16:41.600436+0000","last_clean":"2024-04-13T05:16:41.600436+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:16:41.600436+0000","last_undegraded":"2024-04-13T05:16:41.600436+0000","last_fullsized":"2024-04-13T05:16:41.600436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596758+0000","last_change":"2024-04-13T05:16:14.217489+0000","last_active":"2024-04-13T05:16:41.596758+0000","last_peered":"2024-04-13T05:16:41.596758+0000","last_clean":"2024-04-13T05:16:41.596758+0000","last_became_active":"2024-04-13T05:16:14.216651+0000","last_became_peered":"2024-04-13T05:16:14.216651+0000","last_unstale":"2024-04-13T05:16:41.596758+0000","last_undegraded":"2024-04-13T05:16:41.596758+0000","last_fullsized":"2024-04-13T05:16:41.596758+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:55:10.808460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601929+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:16:41.601929+0000","last_peered":"2024-04-13T05:16:41.601929+0000","last_clean":"2024-04-13T05:16:41.601929+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:16:41.601929+0000","last_undegraded":"2024-04-13T05:16:41.601929+0000","last_fullsized":"2024-04-13T05:16:41.601929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587123+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:16:41.587123+0000","last_peered":"2024-04-13T05:16:41.587123+0000","last_clean":"2024-04-13T05:16:41.587123+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:16:41.587123+0000","last_undegraded":"2024-04-13T05:16:41.587123+0000","last_fullsized":"2024-04-13T05:16:41.587123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590389+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:16:41.590389+0000","last_peered":"2024-04-13T05:16:41.590389+0000","last_clean":"2024-04-13T05:16:41.590389+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:16:41.590389+0000","last_undegraded":"2024-04-13T05:16:41.590389+0000","last_fullsized":"2024-04-13T05:16:41.590389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597106+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:16:41.597106+0000","last_peered":"2024-04-13T05:16:41.597106+0000","last_clean":"2024-04-13T05:16:41.597106+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:16:41.597106+0000","last_undegraded":"2024-04-13T05:16:41.597106+0000","last_fullsized":"2024-04-13T05:16:41.597106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591785+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:16:41.591785+0000","last_peered":"2024-04-13T05:16:41.591785+0000","last_clean":"2024-04-13T05:16:41.591785+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:16:41.591785+0000","last_undegraded":"2024-04-13T05:16:41.591785+0000","last_fullsized":"2024-04-13T05:16:41.591785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601120+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:16:41.601120+0000","last_peered":"2024-04-13T05:16:41.601120+0000","last_clean":"2024-04-13T05:16:41.601120+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:16:41.601120+0000","last_undegraded":"2024-04-13T05:16:41.601120+0000","last_fullsized":"2024-04-13T05:16:41.601120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600809+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:16:41.600809+0000","last_peered":"2024-04-13T05:16:41.600809+0000","last_clean":"2024-04-13T05:16:41.600809+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:16:41.600809+0000","last_undegraded":"2024-04-13T05:16:41.600809+0000","last_fullsized":"2024-04-13T05:16:41.600809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593257+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:16:41.593257+0000","last_peered":"2024-04-13T05:16:41.593257+0000","last_clean":"2024-04-13T05:16:41.593257+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:16:41.593257+0000","last_undegraded":"2024-04-13T05:16:41.593257+0000","last_fullsized":"2024-04-13T05:16:41.593257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574834+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:16:41.574834+0000","last_peered":"2024-04-13T05:16:41.574834+0000","last_clean":"2024-04-13T05:16:41.574834+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:16:41.574834+0000","last_undegraded":"2024-04-13T05:16:41.574834+0000","last_fullsized":"2024-04-13T05:16:41.574834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207323+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:16:14.207323+0000","last_peered":"2024-04-13T05:16:14.207323+0000","last_clean":"2024-04-13T05:16:14.207323+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:16:14.207323+0000","last_undegraded":"2024-04-13T05:16:14.207323+0000","last_fullsized":"2024-04-13T05:16:14.207323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208557+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:16:14.208557+0000","last_peered":"2024-04-13T05:16:14.208557+0000","last_clean":"2024-04-13T05:16:14.208557+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:16:14.208557+0000","last_undegraded":"2024-04-13T05:16:14.208557+0000","last_fullsized":"2024-04-13T05:16:14.208557+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596482+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:16:41.596482+0000","last_peered":"2024-04-13T05:16:41.596482+0000","last_clean":"2024-04-13T05:16:41.596482+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:16:41.596482+0000","last_undegraded":"2024-04-13T05:16:41.596482+0000","last_fullsized":"2024-04-13T05:16:41.596482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207004+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:16:14.207004+0000","last_peered":"2024-04-13T05:16:14.207004+0000","last_clean":"2024-04-13T05:16:14.207004+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:16:14.207004+0000","last_undegraded":"2024-04-13T05:16:14.207004+0000","last_fullsized":"2024-04-13T05:16:14.207004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206186+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:16:14.206186+0000","last_peered":"2024-04-13T05:16:14.206186+0000","last_clean":"2024-04-13T05:16:14.206186+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:16:14.206186+0000","last_undegraded":"2024-04-13T05:16:14.206186+0000","last_fullsized":"2024-04-13T05:16:14.206186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600939+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:16:41.600939+0000","last_peered":"2024-04-13T05:16:41.600939+0000","last_clean":"2024-04-13T05:16:41.600939+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:16:41.600939+0000","last_undegraded":"2024-04-13T05:16:41.600939+0000","last_fullsized":"2024-04-13T05:16:41.600939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207399+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:16:14.207399+0000","last_peered":"2024-04-13T05:16:14.207399+0000","last_clean":"2024-04-13T05:16:14.207399+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:16:14.207399+0000","last_undegraded":"2024-04-13T05:16:14.207399+0000","last_fullsized":"2024-04-13T05:16:14.207399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215166+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:16:14.215166+0000","last_peered":"2024-04-13T05:16:14.215166+0000","last_clean":"2024-04-13T05:16:14.215166+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:16:14.215166+0000","last_undegraded":"2024-04-13T05:16:14.215166+0000","last_fullsized":"2024-04-13T05:16:14.215166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600165+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:16:41.600165+0000","last_peered":"2024-04-13T05:16:41.600165+0000","last_clean":"2024-04-13T05:16:41.600165+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:16:41.600165+0000","last_undegraded":"2024-04-13T05:16:41.600165+0000","last_fullsized":"2024-04-13T05:16:41.600165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207127+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:16:14.207127+0000","last_peered":"2024-04-13T05:16:14.207127+0000","last_clean":"2024-04-13T05:16:14.207127+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:16:14.207127+0000","last_undegraded":"2024-04-13T05:16:14.207127+0000","last_fullsized":"2024-04-13T05:16:14.207127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711648+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:16:14.711648+0000","last_peered":"2024-04-13T05:16:14.711648+0000","last_clean":"2024-04-13T05:16:14.711648+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:16:14.711648+0000","last_undegraded":"2024-04-13T05:16:14.711648+0000","last_fullsized":"2024-04-13T05:16:14.711648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601248+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:16:41.601248+0000","last_peered":"2024-04-13T05:16:41.601248+0000","last_clean":"2024-04-13T05:16:41.601248+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:16:41.601248+0000","last_undegraded":"2024-04-13T05:16:41.601248+0000","last_fullsized":"2024-04-13T05:16:41.601248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208068+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:16:14.208068+0000","last_peered":"2024-04-13T05:16:14.208068+0000","last_clean":"2024-04-13T05:16:14.208068+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:16:14.208068+0000","last_undegraded":"2024-04-13T05:16:14.208068+0000","last_fullsized":"2024-04-13T05:16:14.208068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:51.796889+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:17:51.796889+0000","last_peered":"2024-04-13T05:17:51.796889+0000","last_clean":"2024-04-13T05:17:51.796889+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:17:51.796889+0000","last_undegraded":"2024-04-13T05:17:51.796889+0000","last_fullsized":"2024-04-13T05:17:51.796889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590755+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:16:41.590755+0000","last_peered":"2024-04-13T05:16:41.590755+0000","last_clean":"2024-04-13T05:16:41.590755+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:16:41.590755+0000","last_undegraded":"2024-04-13T05:16:41.590755+0000","last_fullsized":"2024-04-13T05:16:41.590755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207679+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:16:14.207679+0000","last_peered":"2024-04-13T05:16:14.207679+0000","last_clean":"2024-04-13T05:16:14.207679+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:16:14.207679+0000","last_undegraded":"2024-04-13T05:16:14.207679+0000","last_fullsized":"2024-04-13T05:16:14.207679+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.212785+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:16:14.212785+0000","last_peered":"2024-04-13T05:16:14.212785+0000","last_clean":"2024-04-13T05:16:14.212785+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:16:14.212785+0000","last_undegraded":"2024-04-13T05:16:14.212785+0000","last_fullsized":"2024-04-13T05:16:14.212785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597193+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:16:41.597193+0000","last_peered":"2024-04-13T05:16:41.597193+0000","last_clean":"2024-04-13T05:16:41.597193+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:16:41.597193+0000","last_undegraded":"2024-04-13T05:16:41.597193+0000","last_fullsized":"2024-04-13T05:16:41.597193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215483+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:16:14.215483+0000","last_peered":"2024-04-13T05:16:14.215483+0000","last_clean":"2024-04-13T05:16:14.215483+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:16:14.215483+0000","last_undegraded":"2024-04-13T05:16:14.215483+0000","last_fullsized":"2024-04-13T05:16:14.215483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.500836+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:16:14.500836+0000","last_peered":"2024-04-13T05:16:14.500836+0000","last_clean":"2024-04-13T05:16:14.500836+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:16:14.500836+0000","last_undegraded":"2024-04-13T05:16:14.500836+0000","last_fullsized":"2024-04-13T05:16:14.500836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595783+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:16:41.595783+0000","last_peered":"2024-04-13T05:16:41.595783+0000","last_clean":"2024-04-13T05:16:41.595783+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:16:41.595783+0000","last_undegraded":"2024-04-13T05:16:41.595783+0000","last_fullsized":"2024-04-13T05:16:41.595783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548990+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:16:14.548990+0000","last_peered":"2024-04-13T05:16:14.548990+0000","last_clean":"2024-04-13T05:16:14.548990+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:16:14.548990+0000","last_undegraded":"2024-04-13T05:16:14.548990+0000","last_fullsized":"2024-04-13T05:16:14.548990+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488274+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:16:14.488274+0000","last_peered":"2024-04-13T05:16:14.488274+0000","last_clean":"2024-04-13T05:16:14.488274+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:16:14.488274+0000","last_undegraded":"2024-04-13T05:16:14.488274+0000","last_fullsized":"2024-04-13T05:16:14.488274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595439+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:16:41.595439+0000","last_peered":"2024-04-13T05:16:41.595439+0000","last_clean":"2024-04-13T05:16:41.595439+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:16:41.595439+0000","last_undegraded":"2024-04-13T05:16:41.595439+0000","last_fullsized":"2024-04-13T05:16:41.595439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205352+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:16:14.205352+0000","last_peered":"2024-04-13T05:16:14.205352+0000","last_clean":"2024-04-13T05:16:14.205352+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:16:14.205352+0000","last_undegraded":"2024-04-13T05:16:14.205352+0000","last_fullsized":"2024-04-13T05:16:14.205352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":188,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:52.798933+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:17:52.798933+0000","last_peered":"2024-04-13T05:17:52.798933+0000","last_clean":"2024-04-13T05:17:52.798933+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:17:52.798933+0000","last_undegraded":"2024-04-13T05:17:52.798933+0000","last_fullsized":"2024-04-13T05:17:52.798933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599498+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:16:41.599498+0000","last_peered":"2024-04-13T05:16:41.599498+0000","last_clean":"2024-04-13T05:16:41.599498+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:16:41.599498+0000","last_undegraded":"2024-04-13T05:16:41.599498+0000","last_fullsized":"2024-04-13T05:16:41.599498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207859+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:16:14.207859+0000","last_peered":"2024-04-13T05:16:14.207859+0000","last_clean":"2024-04-13T05:16:14.207859+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:16:14.207859+0000","last_undegraded":"2024-04-13T05:16:14.207859+0000","last_fullsized":"2024-04-13T05:16:14.207859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712372+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:16:14.712372+0000","last_peered":"2024-04-13T05:16:14.712372+0000","last_clean":"2024-04-13T05:16:14.712372+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:16:14.712372+0000","last_undegraded":"2024-04-13T05:16:14.712372+0000","last_fullsized":"2024-04-13T05:16:14.712372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590058+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:16:41.590058+0000","last_peered":"2024-04-13T05:16:41.590058+0000","last_clean":"2024-04-13T05:16:41.590058+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:16:41.590058+0000","last_undegraded":"2024-04-13T05:16:41.590058+0000","last_fullsized":"2024-04-13T05:16:41.590058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207958+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:16:14.207958+0000","last_peered":"2024-04-13T05:16:14.207958+0000","last_clean":"2024-04-13T05:16:14.207958+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:16:14.207958+0000","last_undegraded":"2024-04-13T05:16:14.207958+0000","last_fullsized":"2024-04-13T05:16:14.207958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489335+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:16:14.489335+0000","last_peered":"2024-04-13T05:16:14.489335+0000","last_clean":"2024-04-13T05:16:14.489335+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:16:14.489335+0000","last_undegraded":"2024-04-13T05:16:14.489335+0000","last_fullsized":"2024-04-13T05:16:14.489335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601469+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:16:41.601469+0000","last_peered":"2024-04-13T05:16:41.601469+0000","last_clean":"2024-04-13T05:16:41.601469+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:16:41.601469+0000","last_undegraded":"2024-04-13T05:16:41.601469+0000","last_fullsized":"2024-04-13T05:16:41.601469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208081+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:16:14.208081+0000","last_peered":"2024-04-13T05:16:14.208081+0000","last_clean":"2024-04-13T05:16:14.208081+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:16:14.208081+0000","last_undegraded":"2024-04-13T05:16:14.208081+0000","last_fullsized":"2024-04-13T05:16:14.208081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":386,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:53.802110+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:17:53.802110+0000","last_peered":"2024-04-13T05:17:53.802110+0000","last_clean":"2024-04-13T05:17:53.802110+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:17:53.802110+0000","last_undegraded":"2024-04-13T05:17:53.802110+0000","last_fullsized":"2024-04-13T05:17:53.802110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67681,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":226,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595726+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:16:41.595726+0000","last_peered":"2024-04-13T05:16:41.595726+0000","last_clean":"2024-04-13T05:16:41.595726+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:16:41.595726+0000","last_undegraded":"2024-04-13T05:16:41.595726+0000","last_fullsized":"2024-04-13T05:16:41.595726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.490155+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:16:14.490155+0000","last_peered":"2024-04-13T05:16:14.490155+0000","last_clean":"2024-04-13T05:16:14.490155+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:16:14.490155+0000","last_undegraded":"2024-04-13T05:16:14.490155+0000","last_fullsized":"2024-04-13T05:16:14.490155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207832+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:16:14.207832+0000","last_peered":"2024-04-13T05:16:14.207832+0000","last_clean":"2024-04-13T05:16:14.207832+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:16:14.207832+0000","last_undegraded":"2024-04-13T05:16:14.207832+0000","last_fullsized":"2024-04-13T05:16:14.207832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591125+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:16:41.591125+0000","last_peered":"2024-04-13T05:16:41.591125+0000","last_clean":"2024-04-13T05:16:41.591125+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:16:41.591125+0000","last_undegraded":"2024-04-13T05:16:41.591125+0000","last_fullsized":"2024-04-13T05:16:41.591125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214510+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:16:14.214510+0000","last_peered":"2024-04-13T05:16:14.214510+0000","last_clean":"2024-04-13T05:16:14.214510+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:16:14.214510+0000","last_undegraded":"2024-04-13T05:16:14.214510+0000","last_fullsized":"2024-04-13T05:16:14.214510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488900+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:16:14.488900+0000","last_peered":"2024-04-13T05:16:14.488900+0000","last_clean":"2024-04-13T05:16:14.488900+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:16:14.488900+0000","last_undegraded":"2024-04-13T05:16:14.488900+0000","last_fullsized":"2024-04-13T05:16:14.488900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596593+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:16:41.596593+0000","last_peered":"2024-04-13T05:16:41.596593+0000","last_clean":"2024-04-13T05:16:41.596593+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:16:41.596593+0000","last_undegraded":"2024-04-13T05:16:41.596593+0000","last_fullsized":"2024-04-13T05:16:41.596593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208701+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:16:14.208701+0000","last_peered":"2024-04-13T05:16:14.208701+0000","last_clean":"2024-04-13T05:16:14.208701+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:16:14.208701+0000","last_undegraded":"2024-04-13T05:16:14.208701+0000","last_fullsized":"2024-04-13T05:16:14.208701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:51.804836+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:17:51.804836+0000","last_peered":"2024-04-13T05:17:51.804836+0000","last_clean":"2024-04-13T05:17:51.804836+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:17:51.804836+0000","last_undegraded":"2024-04-13T05:17:51.804836+0000","last_fullsized":"2024-04-13T05:17:51.804836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593652+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:16:41.593652+0000","last_peered":"2024-04-13T05:16:41.593652+0000","last_clean":"2024-04-13T05:16:41.593652+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:16:41.593652+0000","last_undegraded":"2024-04-13T05:16:41.593652+0000","last_fullsized":"2024-04-13T05:16:41.593652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.502070+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:16:14.502070+0000","last_peered":"2024-04-13T05:16:14.502070+0000","last_clean":"2024-04-13T05:16:14.502070+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:16:14.502070+0000","last_undegraded":"2024-04-13T05:16:14.502070+0000","last_fullsized":"2024-04-13T05:16:14.502070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709866+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:16:14.709866+0000","last_peered":"2024-04-13T05:16:14.709866+0000","last_clean":"2024-04-13T05:16:14.709866+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:16:14.709866+0000","last_undegraded":"2024-04-13T05:16:14.709866+0000","last_fullsized":"2024-04-13T05:16:14.709866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600587+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:16:41.600587+0000","last_peered":"2024-04-13T05:16:41.600587+0000","last_clean":"2024-04-13T05:16:41.600587+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:16:41.600587+0000","last_undegraded":"2024-04-13T05:16:41.600587+0000","last_fullsized":"2024-04-13T05:16:41.600587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488458+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:16:14.488458+0000","last_peered":"2024-04-13T05:16:14.488458+0000","last_clean":"2024-04-13T05:16:14.488458+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:16:14.488458+0000","last_undegraded":"2024-04-13T05:16:14.488458+0000","last_fullsized":"2024-04-13T05:16:14.488458+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710704+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:16:14.710704+0000","last_peered":"2024-04-13T05:16:14.710704+0000","last_clean":"2024-04-13T05:16:14.710704+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:16:14.710704+0000","last_undegraded":"2024-04-13T05:16:14.710704+0000","last_fullsized":"2024-04-13T05:16:14.710704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584084+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:16:41.584084+0000","last_peered":"2024-04-13T05:16:41.584084+0000","last_clean":"2024-04-13T05:16:41.584084+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:16:41.584084+0000","last_undegraded":"2024-04-13T05:16:41.584084+0000","last_fullsized":"2024-04-13T05:16:41.584084+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711265+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:16:14.711265+0000","last_peered":"2024-04-13T05:16:14.711265+0000","last_clean":"2024-04-13T05:16:14.711265+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:16:14.711265+0000","last_undegraded":"2024-04-13T05:16:14.711265+0000","last_fullsized":"2024-04-13T05:16:14.711265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208277+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:16:14.208277+0000","last_peered":"2024-04-13T05:16:14.208277+0000","last_clean":"2024-04-13T05:16:14.208277+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:16:14.208277+0000","last_undegraded":"2024-04-13T05:16:14.208277+0000","last_fullsized":"2024-04-13T05:16:14.208277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597318+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:16:41.597318+0000","last_peered":"2024-04-13T05:16:41.597318+0000","last_clean":"2024-04-13T05:16:41.597318+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:16:41.597318+0000","last_undegraded":"2024-04-13T05:16:41.597318+0000","last_fullsized":"2024-04-13T05:16:41.597318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501299+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:16:14.501299+0000","last_peered":"2024-04-13T05:16:14.501299+0000","last_clean":"2024-04-13T05:16:14.501299+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:16:14.501299+0000","last_undegraded":"2024-04-13T05:16:14.501299+0000","last_fullsized":"2024-04-13T05:16:14.501299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711835+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:16:14.711835+0000","last_peered":"2024-04-13T05:16:14.711835+0000","last_clean":"2024-04-13T05:16:14.711835+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:16:14.711835+0000","last_undegraded":"2024-04-13T05:16:14.711835+0000","last_fullsized":"2024-04-13T05:16:14.711835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591498+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:16:41.591498+0000","last_peered":"2024-04-13T05:16:41.591498+0000","last_clean":"2024-04-13T05:16:41.591498+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:16:41.591498+0000","last_undegraded":"2024-04-13T05:16:41.591498+0000","last_fullsized":"2024-04-13T05:16:41.591498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501784+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:16:14.501784+0000","last_peered":"2024-04-13T05:16:14.501784+0000","last_clean":"2024-04-13T05:16:14.501784+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:16:14.501784+0000","last_undegraded":"2024-04-13T05:16:14.501784+0000","last_fullsized":"2024-04-13T05:16:14.501784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587661+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:16:41.587661+0000","last_peered":"2024-04-13T05:16:41.587661+0000","last_clean":"2024-04-13T05:16:41.587661+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:16:41.587661+0000","last_undegraded":"2024-04-13T05:16:41.587661+0000","last_fullsized":"2024-04-13T05:16:41.587661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":190,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:52.806787+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:17:52.806787+0000","last_peered":"2024-04-13T05:17:52.806787+0000","last_clean":"2024-04-13T05:17:52.806787+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:17:52.806787+0000","last_undegraded":"2024-04-13T05:17:52.806787+0000","last_fullsized":"2024-04-13T05:17:52.806787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712596+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:16:14.712596+0000","last_peered":"2024-04-13T05:16:14.712596+0000","last_clean":"2024-04-13T05:16:14.712596+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:16:14.712596+0000","last_undegraded":"2024-04-13T05:16:14.712596+0000","last_fullsized":"2024-04-13T05:16:14.712596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.602040+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:16:41.602040+0000","last_peered":"2024-04-13T05:16:41.602040+0000","last_clean":"2024-04-13T05:16:41.602040+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:16:41.602040+0000","last_undegraded":"2024-04-13T05:16:41.602040+0000","last_fullsized":"2024-04-13T05:16:41.602040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207400+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:16:14.207400+0000","last_peered":"2024-04-13T05:16:14.207400+0000","last_clean":"2024-04-13T05:16:14.207400+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:16:14.207400+0000","last_undegraded":"2024-04-13T05:16:14.207400+0000","last_fullsized":"2024-04-13T05:16:14.207400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711063+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:16:14.711063+0000","last_peered":"2024-04-13T05:16:14.711063+0000","last_clean":"2024-04-13T05:16:14.711063+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:16:14.711063+0000","last_undegraded":"2024-04-13T05:16:14.711063+0000","last_fullsized":"2024-04-13T05:16:14.711063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.490158+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:16:14.490158+0000","last_peered":"2024-04-13T05:16:14.490158+0000","last_clean":"2024-04-13T05:16:14.490158+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:16:14.490158+0000","last_undegraded":"2024-04-13T05:16:14.490158+0000","last_fullsized":"2024-04-13T05:16:14.490158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574537+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:16:41.574537+0000","last_peered":"2024-04-13T05:16:41.574537+0000","last_clean":"2024-04-13T05:16:41.574537+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:16:41.574537+0000","last_undegraded":"2024-04-13T05:16:41.574537+0000","last_fullsized":"2024-04-13T05:16:41.574537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711427+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:16:14.711427+0000","last_peered":"2024-04-13T05:16:14.711427+0000","last_clean":"2024-04-13T05:16:14.711427+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:16:14.711427+0000","last_undegraded":"2024-04-13T05:16:14.711427+0000","last_fullsized":"2024-04-13T05:16:14.711427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214710+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:16:14.214710+0000","last_peered":"2024-04-13T05:16:14.214710+0000","last_clean":"2024-04-13T05:16:14.214710+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:16:14.214710+0000","last_undegraded":"2024-04-13T05:16:14.214710+0000","last_fullsized":"2024-04-13T05:16:14.214710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711158+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:16:14.711158+0000","last_peered":"2024-04-13T05:16:14.711158+0000","last_clean":"2024-04-13T05:16:14.711158+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:16:14.711158+0000","last_undegraded":"2024-04-13T05:16:14.711158+0000","last_fullsized":"2024-04-13T05:16:14.711158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574744+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:16:41.574744+0000","last_peered":"2024-04-13T05:16:41.574744+0000","last_clean":"2024-04-13T05:16:41.574744+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:16:41.574744+0000","last_undegraded":"2024-04-13T05:16:41.574744+0000","last_fullsized":"2024-04-13T05:16:41.574744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205197+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:16:14.205197+0000","last_peered":"2024-04-13T05:16:14.205197+0000","last_clean":"2024-04-13T05:16:14.205197+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:16:14.205197+0000","last_undegraded":"2024-04-13T05:16:14.205197+0000","last_fullsized":"2024-04-13T05:16:14.205197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:05.240813+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:17:05.240813+0000","last_peered":"2024-04-13T05:17:05.240813+0000","last_clean":"2024-04-13T05:17:05.240813+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:17:05.240813+0000","last_undegraded":"2024-04-13T05:17:05.240813+0000","last_fullsized":"2024-04-13T05:17:05.240813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595268+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:16:41.595268+0000","last_peered":"2024-04-13T05:16:41.595268+0000","last_clean":"2024-04-13T05:16:41.595268+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:16:41.595268+0000","last_undegraded":"2024-04-13T05:16:41.595268+0000","last_fullsized":"2024-04-13T05:16:41.595268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207010+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:16:14.207010+0000","last_peered":"2024-04-13T05:16:14.207010+0000","last_clean":"2024-04-13T05:16:14.207010+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:16:14.207010+0000","last_undegraded":"2024-04-13T05:16:14.207010+0000","last_fullsized":"2024-04-13T05:16:14.207010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.209094+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:16:14.209094+0000","last_peered":"2024-04-13T05:16:14.209094+0000","last_clean":"2024-04-13T05:16:14.209094+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:16:14.209094+0000","last_undegraded":"2024-04-13T05:16:14.209094+0000","last_fullsized":"2024-04-13T05:16:14.209094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596065+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:16:41.596065+0000","last_peered":"2024-04-13T05:16:41.596065+0000","last_clean":"2024-04-13T05:16:41.596065+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:16:41.596065+0000","last_undegraded":"2024-04-13T05:16:41.596065+0000","last_fullsized":"2024-04-13T05:16:41.596065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208632+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:16:14.208632+0000","last_peered":"2024-04-13T05:16:14.208632+0000","last_clean":"2024-04-13T05:16:14.208632+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:16:14.208632+0000","last_undegraded":"2024-04-13T05:16:14.208632+0000","last_fullsized":"2024-04-13T05:16:14.208632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709476+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:16:14.709476+0000","last_peered":"2024-04-13T05:16:14.709476+0000","last_clean":"2024-04-13T05:16:14.709476+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:16:14.709476+0000","last_undegraded":"2024-04-13T05:16:14.709476+0000","last_fullsized":"2024-04-13T05:16:14.709476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586590+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:16:41.586590+0000","last_peered":"2024-04-13T05:16:41.586590+0000","last_clean":"2024-04-13T05:16:41.586590+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:16:41.586590+0000","last_undegraded":"2024-04-13T05:16:41.586590+0000","last_fullsized":"2024-04-13T05:16:41.586590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208596+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:16:14.208596+0000","last_peered":"2024-04-13T05:16:14.208596+0000","last_clean":"2024-04-13T05:16:14.208596+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:16:14.208596+0000","last_undegraded":"2024-04-13T05:16:14.208596+0000","last_fullsized":"2024-04-13T05:16:14.208596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205469+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:16:14.205469+0000","last_peered":"2024-04-13T05:16:14.205469+0000","last_clean":"2024-04-13T05:16:14.205469+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:16:14.205469+0000","last_undegraded":"2024-04-13T05:16:14.205469+0000","last_fullsized":"2024-04-13T05:16:14.205469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591578+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:16:41.591578+0000","last_peered":"2024-04-13T05:16:41.591578+0000","last_clean":"2024-04-13T05:16:41.591578+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:16:41.591578+0000","last_undegraded":"2024-04-13T05:16:41.591578+0000","last_fullsized":"2024-04-13T05:16:41.591578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207219+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:16:14.207219+0000","last_peered":"2024-04-13T05:16:14.207219+0000","last_clean":"2024-04-13T05:16:14.207219+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:16:14.207219+0000","last_undegraded":"2024-04-13T05:16:14.207219+0000","last_fullsized":"2024-04-13T05:16:14.207219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:25.449178+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:17:25.449178+0000","last_peered":"2024-04-13T05:17:25.449178+0000","last_clean":"2024-04-13T05:17:25.449178+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:17:25.449178+0000","last_undegraded":"2024-04-13T05:17:25.449178+0000","last_fullsized":"2024-04-13T05:17:25.449178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597776+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:16:41.597776+0000","last_peered":"2024-04-13T05:16:41.597776+0000","last_clean":"2024-04-13T05:16:41.597776+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:16:41.597776+0000","last_undegraded":"2024-04-13T05:16:41.597776+0000","last_fullsized":"2024-04-13T05:16:41.597776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208228+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:16:14.208228+0000","last_peered":"2024-04-13T05:16:14.208228+0000","last_clean":"2024-04-13T05:16:14.208228+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:16:14.208228+0000","last_undegraded":"2024-04-13T05:16:14.208228+0000","last_fullsized":"2024-04-13T05:16:14.208228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711142+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:16:14.711142+0000","last_peered":"2024-04-13T05:16:14.711142+0000","last_clean":"2024-04-13T05:16:14.711142+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:16:14.711142+0000","last_undegraded":"2024-04-13T05:16:14.711142+0000","last_fullsized":"2024-04-13T05:16:14.711142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.602016+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:16:41.602016+0000","last_peered":"2024-04-13T05:16:41.602016+0000","last_clean":"2024-04-13T05:16:41.602016+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:16:41.602016+0000","last_undegraded":"2024-04-13T05:16:41.602016+0000","last_fullsized":"2024-04-13T05:16:41.602016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709227+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:16:14.709227+0000","last_peered":"2024-04-13T05:16:14.709227+0000","last_clean":"2024-04-13T05:16:14.709227+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:16:14.709227+0000","last_undegraded":"2024-04-13T05:16:14.709227+0000","last_fullsized":"2024-04-13T05:16:14.709227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208151+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:16:14.208151+0000","last_peered":"2024-04-13T05:16:14.208151+0000","last_clean":"2024-04-13T05:16:14.208151+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:16:14.208151+0000","last_undegraded":"2024-04-13T05:16:14.208151+0000","last_fullsized":"2024-04-13T05:16:14.208151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601798+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:16:41.601798+0000","last_peered":"2024-04-13T05:16:41.601798+0000","last_clean":"2024-04-13T05:16:41.601798+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:16:41.601798+0000","last_undegraded":"2024-04-13T05:16:41.601798+0000","last_fullsized":"2024-04-13T05:16:41.601798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548391+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:16:14.548391+0000","last_peered":"2024-04-13T05:16:14.548391+0000","last_clean":"2024-04-13T05:16:14.548391+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:16:14.548391+0000","last_undegraded":"2024-04-13T05:16:14.548391+0000","last_fullsized":"2024-04-13T05:16:14.548391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489672+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:16:14.489672+0000","last_peered":"2024-04-13T05:16:14.489672+0000","last_clean":"2024-04-13T05:16:14.489672+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:16:14.489672+0000","last_undegraded":"2024-04-13T05:16:14.489672+0000","last_fullsized":"2024-04-13T05:16:14.489672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585990+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:16:41.585990+0000","last_peered":"2024-04-13T05:16:41.585990+0000","last_clean":"2024-04-13T05:16:41.585990+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:16:41.585990+0000","last_undegraded":"2024-04-13T05:16:41.585990+0000","last_fullsized":"2024-04-13T05:16:41.585990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.487926+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:16:14.487926+0000","last_peered":"2024-04-13T05:16:14.487926+0000","last_clean":"2024-04-13T05:16:14.487926+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:16:14.487926+0000","last_undegraded":"2024-04-13T05:16:14.487926+0000","last_fullsized":"2024-04-13T05:16:14.487926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501896+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:16:14.501896+0000","last_peered":"2024-04-13T05:16:14.501896+0000","last_clean":"2024-04-13T05:16:14.501896+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:16:14.501896+0000","last_undegraded":"2024-04-13T05:16:14.501896+0000","last_fullsized":"2024-04-13T05:16:14.501896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594768+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:16:41.594768+0000","last_peered":"2024-04-13T05:16:41.594768+0000","last_clean":"2024-04-13T05:16:41.594768+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:16:41.594768+0000","last_undegraded":"2024-04-13T05:16:41.594768+0000","last_fullsized":"2024-04-13T05:16:41.594768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584585+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:16:41.584585+0000","last_peered":"2024-04-13T05:16:41.584585+0000","last_clean":"2024-04-13T05:16:41.584585+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:16:41.584585+0000","last_undegraded":"2024-04-13T05:16:41.584585+0000","last_fullsized":"2024-04-13T05:16:41.584585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587156+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:16:41.587156+0000","last_peered":"2024-04-13T05:16:41.587156+0000","last_clean":"2024-04-13T05:16:41.587156+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:16:41.587156+0000","last_undegraded":"2024-04-13T05:16:41.587156+0000","last_fullsized":"2024-04-13T05:16:41.587156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.592009+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:16:41.592009+0000","last_peered":"2024-04-13T05:16:41.592009+0000","last_clean":"2024-04-13T05:16:41.592009+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:16:41.592009+0000","last_undegraded":"2024-04-13T05:16:41.592009+0000","last_fullsized":"2024-04-13T05:16:41.592009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591014+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:16:41.591014+0000","last_peered":"2024-04-13T05:16:41.591014+0000","last_clean":"2024-04-13T05:16:41.591014+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:16:41.591014+0000","last_undegraded":"2024-04-13T05:16:41.591014+0000","last_fullsized":"2024-04-13T05:16:41.591014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586928+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:16:41.586928+0000","last_peered":"2024-04-13T05:16:41.586928+0000","last_clean":"2024-04-13T05:16:41.586928+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:16:41.586928+0000","last_undegraded":"2024-04-13T05:16:41.586928+0000","last_fullsized":"2024-04-13T05:16:41.586928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571520+0000","last_change":"2024-04-13T05:16:14.714804+0000","last_active":"2024-04-13T05:16:41.571520+0000","last_peered":"2024-04-13T05:16:41.571520+0000","last_clean":"2024-04-13T05:16:41.571520+0000","last_became_active":"2024-04-13T05:16:14.713542+0000","last_became_peered":"2024-04-13T05:16:14.713542+0000","last_unstale":"2024-04-13T05:16:41.571520+0000","last_undegraded":"2024-04-13T05:16:41.571520+0000","last_fullsized":"2024-04-13T05:16:41.571520+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:21:57.719452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600113+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:16:41.600113+0000","last_peered":"2024-04-13T05:16:41.600113+0000","last_clean":"2024-04-13T05:16:41.600113+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:16:41.600113+0000","last_undegraded":"2024-04-13T05:16:41.600113+0000","last_fullsized":"2024-04-13T05:16:41.600113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.570115+0000","last_change":"2024-04-13T05:16:14.713320+0000","last_active":"2024-04-13T05:16:41.570115+0000","last_peered":"2024-04-13T05:16:41.570115+0000","last_clean":"2024-04-13T05:16:41.570115+0000","last_became_active":"2024-04-13T05:16:14.712507+0000","last_became_peered":"2024-04-13T05:16:14.712507+0000","last_unstale":"2024-04-13T05:16:41.570115+0000","last_undegraded":"2024-04-13T05:16:41.570115+0000","last_fullsized":"2024-04-13T05:16:41.570115+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:07.206767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.598327+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:16:41.598327+0000","last_peered":"2024-04-13T05:16:41.598327+0000","last_clean":"2024-04-13T05:16:41.598327+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:16:41.598327+0000","last_undegraded":"2024-04-13T05:16:41.598327+0000","last_fullsized":"2024-04-13T05:16:41.598327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596446+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:16:41.596446+0000","last_peered":"2024-04-13T05:16:41.596446+0000","last_clean":"2024-04-13T05:16:41.596446+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:16:41.596446+0000","last_undegraded":"2024-04-13T05:16:41.596446+0000","last_fullsized":"2024-04-13T05:16:41.596446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600578+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:16:41.600578+0000","last_peered":"2024-04-13T05:16:41.600578+0000","last_clean":"2024-04-13T05:16:41.600578+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:16:41.600578+0000","last_undegraded":"2024-04-13T05:16:41.600578+0000","last_fullsized":"2024-04-13T05:16:41.600578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593910+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:16:41.593910+0000","last_peered":"2024-04-13T05:16:41.593910+0000","last_clean":"2024-04-13T05:16:41.593910+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:16:41.593910+0000","last_undegraded":"2024-04-13T05:16:41.593910+0000","last_fullsized":"2024-04-13T05:16:41.593910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571621+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:16:41.571621+0000","last_peered":"2024-04-13T05:16:41.571621+0000","last_clean":"2024-04-13T05:16:41.571621+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:16:41.571621+0000","last_undegraded":"2024-04-13T05:16:41.571621+0000","last_fullsized":"2024-04-13T05:16:41.571621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72809,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":944,"num_read_kb":552,"num_write":180,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32900,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712252,"statfs":{"total":95995035648,"available":95961346048,"internally_reserved":0,"allocated":1765376,"data_stored":1182287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808898,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":530645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":525887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1123505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":589428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":523113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1155072,"data_stored":591047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1118068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:17:56.545 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-13T05:17:56.545 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-13T05:17:56.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:56 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/476964192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:17:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 -- 172.21.15.57:0/229712621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94104040 msgr2=0x7f4f94104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 --2- 172.21.15.57:0/229712621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94104040 0x7f4f94104460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009900 tx=0x7f4f8402bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 -- 172.21.15.57:0/229712621 shutdown_connections 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 --2- 172.21.15.57:0/229712621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94105230 0x7f4f941056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 --2- 172.21.15.57:0/229712621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f94105bf0 0x7f4f9410c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 --2- 172.21.15.57:0/229712621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94104040 0x7f4f94104460 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 -- 172.21.15.57:0/229712621 >> 172.21.15.57:0/229712621 conn(0x7f4f940ff5d0 msgr2=0x7f4f941019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 -- 172.21.15.57:0/229712621 shutdown_connections 2024-04-13T05:17:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.804+0000 7f4f99600640 1 -- 172.21.15.57:0/229712621 wait complete. 2024-04-13T05:17:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.805+0000 7f4f99600640 1 Processor -- start 2024-04-13T05:17:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.805+0000 7f4f99600640 1 -- start start 2024-04-13T05:17:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f99600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94104040 0x7f4f9419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f99600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 0x7f4f9419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f99600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f94105bf0 0x7f4f941a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f99600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9410fbd0 con 0x7f4f94105230 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f99600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9410fa50 con 0x7f4f94104040 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f99600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9410fd50 con 0x7f4f94105bf0 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 0x7f4f9419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 0x7f4f9419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33920/0 (socket says 172.21.15.57:33920) 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 -- 172.21.15.57:0/2582397753 learned_addr learned my addr 172.21.15.57:0/2582397753 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f98c00640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f94105bf0 0x7f4f941a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93e00640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94104040 0x7f4f9419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 -- 172.21.15.57:0/2582397753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f94105bf0 msgr2=0x7f4f941a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f94105bf0 0x7f4f941a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 -- 172.21.15.57:0/2582397753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94104040 msgr2=0x7f4f9419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:56.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94104040 0x7f4f9419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93400640 1 -- 172.21.15.57:0/2582397753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f941a1e20 con 0x7f4f94105230 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.806+0000 7f4f93e00640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94104040 0x7f4f9419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.807+0000 7f4f93400640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 0x7f4f9419a740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c920 tx=0x7f4f8000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.807+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80007c70 con 0x7f4f94105230 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.807+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f80007e10 con 0x7f4f94105230 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.807+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80018820 con 0x7f4f94105230 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.807+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f941a2110 con 0x7f4f94105230 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.807+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941a2610 con 0x7f4f94105230 2024-04-13T05:17:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.808+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9410b360 con 0x7f4f94105230 2024-04-13T05:17:56.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.811+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f800189c0 con 0x7f4f94105230 2024-04-13T05:17:56.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.812+0000 7f4f90c00640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4f780a3c30 0x7f4f780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:56.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.813+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4f800d09a0 con 0x7f4f94105230 2024-04-13T05:17:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.813+0000 7f4f93e00640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4f780a3c30 0x7f4f780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.814+0000 7f4f93e00640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4f780a3c30 0x7f4f780a60f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4f84036040 tx=0x7f4f84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:56.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:56.817+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f80014030 con 0x7f4f94105230 2024-04-13T05:17:57.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.021+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 --> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f94062400 con 0x7f4f780a3c30 2024-04-13T05:17:57.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.052+0000 7f4f90c00640 1 -- 172.21.15.57:0/2582397753 <== mgr.14152 v2:172.21.15.57:6800/3239836246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340546 (secure 0 0 0) 0x7f4f94062400 con 0x7f4f780a3c30 2024-04-13T05:17:57.057 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:57.061 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.059+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4f780a3c30 msgr2=0x7f4f780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4f780a3c30 0x7f4f780a60f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4f84036040 tx=0x7f4f84005fd0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 msgr2=0x7f4f9419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 0x7f4f9419a740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c920 tx=0x7f4f8000cde0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 shutdown_connections 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4f780a3c30 0x7f4f780a60f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f94105bf0 0x7f4f941a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f94105230 0x7f4f9419a740 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 --2- 172.21.15.57:0/2582397753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f94104040 0x7f4f9419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 >> 172.21.15.57:0/2582397753 conn(0x7f4f940ff5d0 msgr2=0x7f4f94108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 shutdown_connections 2024-04-13T05:17:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.060+0000 7f4f99600640 1 -- 172.21.15.57:0/2582397753 wait complete. 2024-04-13T05:17:57.084 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-13T05:17:55.674518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663411,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1070,"num_read_kb":661,"num_write":363,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":794,"ondisk_log_size":794,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367948,"kb_used_data":14636,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573876,"statfs":{"total":1151940427776,"available":1151563649024,"internally_reserved":0,"allocated":14987264,"data_stored":8281919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5793,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":53,"num_write":11,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004497"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600757+0000","last_change":"2024-04-13T05:14:53.604946+0000","last_active":"2024-04-13T05:16:41.600757+0000","last_peered":"2024-04-13T05:16:41.600757+0000","last_clean":"2024-04-13T05:16:41.600757+0000","last_became_active":"2024-04-13T05:14:53.604564+0000","last_became_peered":"2024-04-13T05:14:53.604564+0000","last_unstale":"2024-04-13T05:16:41.600757+0000","last_undegraded":"2024-04-13T05:16:41.600757+0000","last_fullsized":"2024-04-13T05:16:41.600757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:00:53.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595560+0000","last_change":"2024-04-13T05:14:53.609579+0000","last_active":"2024-04-13T05:16:41.595560+0000","last_peered":"2024-04-13T05:16:41.595560+0000","last_clean":"2024-04-13T05:16:41.595560+0000","last_became_active":"2024-04-13T05:14:53.606738+0000","last_became_peered":"2024-04-13T05:14:53.606738+0000","last_unstale":"2024-04-13T05:16:41.595560+0000","last_undegraded":"2024-04-13T05:16:41.595560+0000","last_fullsized":"2024-04-13T05:16:41.595560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.421306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601071+0000","last_change":"2024-04-13T05:14:53.604703+0000","last_active":"2024-04-13T05:16:41.601071+0000","last_peered":"2024-04-13T05:16:41.601071+0000","last_clean":"2024-04-13T05:16:41.601071+0000","last_became_active":"2024-04-13T05:14:53.604358+0000","last_became_peered":"2024-04-13T05:14:53.604358+0000","last_unstale":"2024-04-13T05:16:41.601071+0000","last_undegraded":"2024-04-13T05:16:41.601071+0000","last_fullsized":"2024-04-13T05:16:41.601071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:37:00.952648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.573856+0000","last_change":"2024-04-13T05:14:53.616976+0000","last_active":"2024-04-13T05:16:41.573856+0000","last_peered":"2024-04-13T05:16:41.573856+0000","last_clean":"2024-04-13T05:16:41.573856+0000","last_became_active":"2024-04-13T05:14:53.615575+0000","last_became_peered":"2024-04-13T05:14:53.615575+0000","last_unstale":"2024-04-13T05:16:41.573856+0000","last_undegraded":"2024-04-13T05:16:41.573856+0000","last_fullsized":"2024-04-13T05:16:41.573856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:06:07.952458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.580277+0000","last_change":"2024-04-13T05:14:53.603109+0000","last_active":"2024-04-13T05:16:41.580277+0000","last_peered":"2024-04-13T05:16:41.580277+0000","last_clean":"2024-04-13T05:16:41.580277+0000","last_became_active":"2024-04-13T05:14:53.602779+0000","last_became_peered":"2024-04-13T05:14:53.602779+0000","last_unstale":"2024-04-13T05:16:41.580277+0000","last_undegraded":"2024-04-13T05:16:41.580277+0000","last_fullsized":"2024-04-13T05:16:41.580277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:33:08.289560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591873+0000","last_change":"2024-04-13T05:14:53.602942+0000","last_active":"2024-04-13T05:16:41.591873+0000","last_peered":"2024-04-13T05:16:41.591873+0000","last_clean":"2024-04-13T05:16:41.591873+0000","last_became_active":"2024-04-13T05:14:53.601546+0000","last_became_peered":"2024-04-13T05:14:53.601546+0000","last_unstale":"2024-04-13T05:16:41.591873+0000","last_undegraded":"2024-04-13T05:16:41.591873+0000","last_fullsized":"2024-04-13T05:16:41.591873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:49:47.521735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593324+0000","last_change":"2024-04-13T05:14:53.592959+0000","last_active":"2024-04-13T05:16:41.593324+0000","last_peered":"2024-04-13T05:16:41.593324+0000","last_clean":"2024-04-13T05:16:41.593324+0000","last_became_active":"2024-04-13T05:14:53.592542+0000","last_became_peered":"2024-04-13T05:14:53.592542+0000","last_unstale":"2024-04-13T05:16:41.593324+0000","last_undegraded":"2024-04-13T05:16:41.593324+0000","last_fullsized":"2024-04-13T05:16:41.593324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:54:46.928614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597594+0000","last_change":"2024-04-13T05:14:53.608877+0000","last_active":"2024-04-13T05:16:41.597594+0000","last_peered":"2024-04-13T05:16:41.597594+0000","last_clean":"2024-04-13T05:16:41.597594+0000","last_became_active":"2024-04-13T05:14:53.608001+0000","last_became_peered":"2024-04-13T05:14:53.608001+0000","last_unstale":"2024-04-13T05:16:41.597594+0000","last_undegraded":"2024-04-13T05:16:41.597594+0000","last_fullsized":"2024-04-13T05:16:41.597594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:34.586020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600436+0000","last_change":"2024-04-13T05:14:53.603678+0000","last_active":"2024-04-13T05:16:41.600436+0000","last_peered":"2024-04-13T05:16:41.600436+0000","last_clean":"2024-04-13T05:16:41.600436+0000","last_became_active":"2024-04-13T05:14:53.601366+0000","last_became_peered":"2024-04-13T05:14:53.601366+0000","last_unstale":"2024-04-13T05:16:41.600436+0000","last_undegraded":"2024-04-13T05:16:41.600436+0000","last_fullsized":"2024-04-13T05:16:41.600436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:57:54.852400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596758+0000","last_change":"2024-04-13T05:16:14.217489+0000","last_active":"2024-04-13T05:16:41.596758+0000","last_peered":"2024-04-13T05:16:41.596758+0000","last_clean":"2024-04-13T05:16:41.596758+0000","last_became_active":"2024-04-13T05:16:14.216651+0000","last_became_peered":"2024-04-13T05:16:14.216651+0000","last_unstale":"2024-04-13T05:16:41.596758+0000","last_undegraded":"2024-04-13T05:16:41.596758+0000","last_fullsized":"2024-04-13T05:16:41.596758+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:55:10.808460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601929+0000","last_change":"2024-04-13T05:14:53.594647+0000","last_active":"2024-04-13T05:16:41.601929+0000","last_peered":"2024-04-13T05:16:41.601929+0000","last_clean":"2024-04-13T05:16:41.601929+0000","last_became_active":"2024-04-13T05:14:53.593439+0000","last_became_peered":"2024-04-13T05:14:53.593439+0000","last_unstale":"2024-04-13T05:16:41.601929+0000","last_undegraded":"2024-04-13T05:16:41.601929+0000","last_fullsized":"2024-04-13T05:16:41.601929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:06:35.359694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587123+0000","last_change":"2024-04-13T05:14:53.614923+0000","last_active":"2024-04-13T05:16:41.587123+0000","last_peered":"2024-04-13T05:16:41.587123+0000","last_clean":"2024-04-13T05:16:41.587123+0000","last_became_active":"2024-04-13T05:14:53.614483+0000","last_became_peered":"2024-04-13T05:14:53.614483+0000","last_unstale":"2024-04-13T05:16:41.587123+0000","last_undegraded":"2024-04-13T05:16:41.587123+0000","last_fullsized":"2024-04-13T05:16:41.587123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:16:14.839547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590389+0000","last_change":"2024-04-13T05:14:53.596438+0000","last_active":"2024-04-13T05:16:41.590389+0000","last_peered":"2024-04-13T05:16:41.590389+0000","last_clean":"2024-04-13T05:16:41.590389+0000","last_became_active":"2024-04-13T05:14:53.591615+0000","last_became_peered":"2024-04-13T05:14:53.591615+0000","last_unstale":"2024-04-13T05:16:41.590389+0000","last_undegraded":"2024-04-13T05:16:41.590389+0000","last_fullsized":"2024-04-13T05:16:41.590389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:10:30.097980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597106+0000","last_change":"2024-04-13T05:14:53.616834+0000","last_active":"2024-04-13T05:16:41.597106+0000","last_peered":"2024-04-13T05:16:41.597106+0000","last_clean":"2024-04-13T05:16:41.597106+0000","last_became_active":"2024-04-13T05:14:53.615895+0000","last_became_peered":"2024-04-13T05:14:53.615895+0000","last_unstale":"2024-04-13T05:16:41.597106+0000","last_undegraded":"2024-04-13T05:16:41.597106+0000","last_fullsized":"2024-04-13T05:16:41.597106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:59.413729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591785+0000","last_change":"2024-04-13T05:14:53.605471+0000","last_active":"2024-04-13T05:16:41.591785+0000","last_peered":"2024-04-13T05:16:41.591785+0000","last_clean":"2024-04-13T05:16:41.591785+0000","last_became_active":"2024-04-13T05:14:53.605075+0000","last_became_peered":"2024-04-13T05:14:53.605075+0000","last_unstale":"2024-04-13T05:16:41.591785+0000","last_undegraded":"2024-04-13T05:16:41.591785+0000","last_fullsized":"2024-04-13T05:16:41.591785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:47.928785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601120+0000","last_change":"2024-04-13T05:14:53.602298+0000","last_active":"2024-04-13T05:16:41.601120+0000","last_peered":"2024-04-13T05:16:41.601120+0000","last_clean":"2024-04-13T05:16:41.601120+0000","last_became_active":"2024-04-13T05:14:53.599684+0000","last_became_peered":"2024-04-13T05:14:53.599684+0000","last_unstale":"2024-04-13T05:16:41.601120+0000","last_undegraded":"2024-04-13T05:16:41.601120+0000","last_fullsized":"2024-04-13T05:16:41.601120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:26.288885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600809+0000","last_change":"2024-04-13T05:14:53.602966+0000","last_active":"2024-04-13T05:16:41.600809+0000","last_peered":"2024-04-13T05:16:41.600809+0000","last_clean":"2024-04-13T05:16:41.600809+0000","last_became_active":"2024-04-13T05:14:53.602476+0000","last_became_peered":"2024-04-13T05:14:53.602476+0000","last_unstale":"2024-04-13T05:16:41.600809+0000","last_undegraded":"2024-04-13T05:16:41.600809+0000","last_fullsized":"2024-04-13T05:16:41.600809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:54.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593257+0000","last_change":"2024-04-13T05:14:53.599481+0000","last_active":"2024-04-13T05:16:41.593257+0000","last_peered":"2024-04-13T05:16:41.593257+0000","last_clean":"2024-04-13T05:16:41.593257+0000","last_became_active":"2024-04-13T05:14:53.599060+0000","last_became_peered":"2024-04-13T05:14:53.599060+0000","last_unstale":"2024-04-13T05:16:41.593257+0000","last_undegraded":"2024-04-13T05:16:41.593257+0000","last_fullsized":"2024-04-13T05:16:41.593257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:17.723039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574834+0000","last_change":"2024-04-13T05:14:53.591147+0000","last_active":"2024-04-13T05:16:41.574834+0000","last_peered":"2024-04-13T05:16:41.574834+0000","last_clean":"2024-04-13T05:16:41.574834+0000","last_became_active":"2024-04-13T05:14:53.590544+0000","last_became_peered":"2024-04-13T05:14:53.590544+0000","last_unstale":"2024-04-13T05:16:41.574834+0000","last_undegraded":"2024-04-13T05:16:41.574834+0000","last_fullsized":"2024-04-13T05:16:41.574834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:52.959777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207323+0000","last_change":"2024-04-13T05:15:13.179323+0000","last_active":"2024-04-13T05:16:14.207323+0000","last_peered":"2024-04-13T05:16:14.207323+0000","last_clean":"2024-04-13T05:16:14.207323+0000","last_became_active":"2024-04-13T05:15:13.178782+0000","last_became_peered":"2024-04-13T05:15:13.178782+0000","last_unstale":"2024-04-13T05:16:14.207323+0000","last_undegraded":"2024-04-13T05:16:14.207323+0000","last_fullsized":"2024-04-13T05:16:14.207323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:49:34.448888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208557+0000","last_change":"2024-04-13T05:15:13.187277+0000","last_active":"2024-04-13T05:16:14.208557+0000","last_peered":"2024-04-13T05:16:14.208557+0000","last_clean":"2024-04-13T05:16:14.208557+0000","last_became_active":"2024-04-13T05:15:13.171486+0000","last_became_peered":"2024-04-13T05:15:13.171486+0000","last_unstale":"2024-04-13T05:16:14.208557+0000","last_undegraded":"2024-04-13T05:16:14.208557+0000","last_fullsized":"2024-04-13T05:16:14.208557+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:12:25.473933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596482+0000","last_change":"2024-04-13T05:14:53.603427+0000","last_active":"2024-04-13T05:16:41.596482+0000","last_peered":"2024-04-13T05:16:41.596482+0000","last_clean":"2024-04-13T05:16:41.596482+0000","last_became_active":"2024-04-13T05:14:53.603073+0000","last_became_peered":"2024-04-13T05:14:53.603073+0000","last_unstale":"2024-04-13T05:16:41.596482+0000","last_undegraded":"2024-04-13T05:16:41.596482+0000","last_fullsized":"2024-04-13T05:16:41.596482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:16.542496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207004+0000","last_change":"2024-04-13T05:14:58.103580+0000","last_active":"2024-04-13T05:16:14.207004+0000","last_peered":"2024-04-13T05:16:14.207004+0000","last_clean":"2024-04-13T05:16:14.207004+0000","last_became_active":"2024-04-13T05:14:58.103105+0000","last_became_peered":"2024-04-13T05:14:58.103105+0000","last_unstale":"2024-04-13T05:16:14.207004+0000","last_undegraded":"2024-04-13T05:16:14.207004+0000","last_fullsized":"2024-04-13T05:16:14.207004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:20:41.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.206186+0000","last_change":"2024-04-13T05:14:52.604746+0000","last_active":"2024-04-13T05:16:14.206186+0000","last_peered":"2024-04-13T05:16:14.206186+0000","last_clean":"2024-04-13T05:16:14.206186+0000","last_became_active":"2024-04-13T05:14:52.604429+0000","last_became_peered":"2024-04-13T05:14:52.604429+0000","last_unstale":"2024-04-13T05:16:14.206186+0000","last_undegraded":"2024-04-13T05:16:14.206186+0000","last_fullsized":"2024-04-13T05:16:14.206186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:28:14.796434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600939+0000","last_change":"2024-04-13T05:14:53.599477+0000","last_active":"2024-04-13T05:16:41.600939+0000","last_peered":"2024-04-13T05:16:41.600939+0000","last_clean":"2024-04-13T05:16:41.600939+0000","last_became_active":"2024-04-13T05:14:53.598923+0000","last_became_peered":"2024-04-13T05:14:53.598923+0000","last_unstale":"2024-04-13T05:16:41.600939+0000","last_undegraded":"2024-04-13T05:16:41.600939+0000","last_fullsized":"2024-04-13T05:16:41.600939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:07:31.322716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207399+0000","last_change":"2024-04-13T05:15:13.197015+0000","last_active":"2024-04-13T05:16:14.207399+0000","last_peered":"2024-04-13T05:16:14.207399+0000","last_clean":"2024-04-13T05:16:14.207399+0000","last_became_active":"2024-04-13T05:15:13.195675+0000","last_became_peered":"2024-04-13T05:15:13.195675+0000","last_unstale":"2024-04-13T05:16:14.207399+0000","last_undegraded":"2024-04-13T05:16:14.207399+0000","last_fullsized":"2024-04-13T05:16:14.207399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:46:36.572387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,10],"acting":[4,3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215166+0000","last_change":"2024-04-13T05:15:13.193015+0000","last_active":"2024-04-13T05:16:14.215166+0000","last_peered":"2024-04-13T05:16:14.215166+0000","last_clean":"2024-04-13T05:16:14.215166+0000","last_became_active":"2024-04-13T05:15:13.182624+0000","last_became_peered":"2024-04-13T05:15:13.182624+0000","last_unstale":"2024-04-13T05:16:14.215166+0000","last_undegraded":"2024-04-13T05:16:14.215166+0000","last_fullsized":"2024-04-13T05:16:14.215166+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:45:17.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600165+0000","last_change":"2024-04-13T05:14:53.611837+0000","last_active":"2024-04-13T05:16:41.600165+0000","last_peered":"2024-04-13T05:16:41.600165+0000","last_clean":"2024-04-13T05:16:41.600165+0000","last_became_active":"2024-04-13T05:14:53.609842+0000","last_became_peered":"2024-04-13T05:14:53.609842+0000","last_unstale":"2024-04-13T05:16:41.600165+0000","last_undegraded":"2024-04-13T05:16:41.600165+0000","last_fullsized":"2024-04-13T05:16:41.600165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:00:01.574795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207127+0000","last_change":"2024-04-13T05:14:58.103107+0000","last_active":"2024-04-13T05:16:14.207127+0000","last_peered":"2024-04-13T05:16:14.207127+0000","last_clean":"2024-04-13T05:16:14.207127+0000","last_became_active":"2024-04-13T05:14:58.102437+0000","last_became_peered":"2024-04-13T05:14:58.102437+0000","last_unstale":"2024-04-13T05:16:14.207127+0000","last_undegraded":"2024-04-13T05:16:14.207127+0000","last_fullsized":"2024-04-13T05:16:14.207127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:29:02.068376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711648+0000","last_change":"2024-04-13T05:14:52.607033+0000","last_active":"2024-04-13T05:16:14.711648+0000","last_peered":"2024-04-13T05:16:14.711648+0000","last_clean":"2024-04-13T05:16:14.711648+0000","last_became_active":"2024-04-13T05:14:52.606216+0000","last_became_peered":"2024-04-13T05:14:52.606216+0000","last_unstale":"2024-04-13T05:16:14.711648+0000","last_undegraded":"2024-04-13T05:16:14.711648+0000","last_fullsized":"2024-04-13T05:16:14.711648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:10:06.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601248+0000","last_change":"2024-04-13T05:14:53.616391+0000","last_active":"2024-04-13T05:16:41.601248+0000","last_peered":"2024-04-13T05:16:41.601248+0000","last_clean":"2024-04-13T05:16:41.601248+0000","last_became_active":"2024-04-13T05:14:53.615783+0000","last_became_peered":"2024-04-13T05:14:53.615783+0000","last_unstale":"2024-04-13T05:16:41.601248+0000","last_undegraded":"2024-04-13T05:16:41.601248+0000","last_fullsized":"2024-04-13T05:16:41.601248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:35:23.093839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208068+0000","last_change":"2024-04-13T05:14:58.108969+0000","last_active":"2024-04-13T05:16:14.208068+0000","last_peered":"2024-04-13T05:16:14.208068+0000","last_clean":"2024-04-13T05:16:14.208068+0000","last_became_active":"2024-04-13T05:14:58.108481+0000","last_became_peered":"2024-04-13T05:14:58.108481+0000","last_unstale":"2024-04-13T05:16:14.208068+0000","last_undegraded":"2024-04-13T05:16:14.208068+0000","last_fullsized":"2024-04-13T05:16:14.208068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:37:40.812277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:51.796889+0000","last_change":"2024-04-13T05:14:52.607760+0000","last_active":"2024-04-13T05:17:51.796889+0000","last_peered":"2024-04-13T05:17:51.796889+0000","last_clean":"2024-04-13T05:17:51.796889+0000","last_became_active":"2024-04-13T05:14:52.607345+0000","last_became_peered":"2024-04-13T05:14:52.607345+0000","last_unstale":"2024-04-13T05:17:51.796889+0000","last_undegraded":"2024-04-13T05:17:51.796889+0000","last_fullsized":"2024-04-13T05:17:51.796889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:10.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590755+0000","last_change":"2024-04-13T05:14:53.594358+0000","last_active":"2024-04-13T05:16:41.590755+0000","last_peered":"2024-04-13T05:16:41.590755+0000","last_clean":"2024-04-13T05:16:41.590755+0000","last_became_active":"2024-04-13T05:14:53.594039+0000","last_became_peered":"2024-04-13T05:14:53.594039+0000","last_unstale":"2024-04-13T05:16:41.590755+0000","last_undegraded":"2024-04-13T05:16:41.590755+0000","last_fullsized":"2024-04-13T05:16:41.590755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:01:21.235876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207679+0000","last_change":"2024-04-13T05:15:13.189675+0000","last_active":"2024-04-13T05:16:14.207679+0000","last_peered":"2024-04-13T05:16:14.207679+0000","last_clean":"2024-04-13T05:16:14.207679+0000","last_became_active":"2024-04-13T05:15:13.189081+0000","last_became_peered":"2024-04-13T05:15:13.189081+0000","last_unstale":"2024-04-13T05:16:14.207679+0000","last_undegraded":"2024-04-13T05:16:14.207679+0000","last_fullsized":"2024-04-13T05:16:14.207679+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:20:49.943320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.212785+0000","last_change":"2024-04-13T05:14:52.607198+0000","last_active":"2024-04-13T05:16:14.212785+0000","last_peered":"2024-04-13T05:16:14.212785+0000","last_clean":"2024-04-13T05:16:14.212785+0000","last_became_active":"2024-04-13T05:14:52.606757+0000","last_became_peered":"2024-04-13T05:14:52.606757+0000","last_unstale":"2024-04-13T05:16:14.212785+0000","last_undegraded":"2024-04-13T05:16:14.212785+0000","last_fullsized":"2024-04-13T05:16:14.212785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:04:05.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597193+0000","last_change":"2024-04-13T05:14:53.606921+0000","last_active":"2024-04-13T05:16:41.597193+0000","last_peered":"2024-04-13T05:16:41.597193+0000","last_clean":"2024-04-13T05:16:41.597193+0000","last_became_active":"2024-04-13T05:14:53.606536+0000","last_became_peered":"2024-04-13T05:14:53.606536+0000","last_unstale":"2024-04-13T05:16:41.597193+0000","last_undegraded":"2024-04-13T05:16:41.597193+0000","last_fullsized":"2024-04-13T05:16:41.597193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:08:48.206424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.215483+0000","last_change":"2024-04-13T05:14:58.115530+0000","last_active":"2024-04-13T05:16:14.215483+0000","last_peered":"2024-04-13T05:16:14.215483+0000","last_clean":"2024-04-13T05:16:14.215483+0000","last_became_active":"2024-04-13T05:14:58.115039+0000","last_became_peered":"2024-04-13T05:14:58.115039+0000","last_unstale":"2024-04-13T05:16:14.215483+0000","last_undegraded":"2024-04-13T05:16:14.215483+0000","last_fullsized":"2024-04-13T05:16:14.215483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:36:20.180858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.500836+0000","last_change":"2024-04-13T05:14:52.596576+0000","last_active":"2024-04-13T05:16:14.500836+0000","last_peered":"2024-04-13T05:16:14.500836+0000","last_clean":"2024-04-13T05:16:14.500836+0000","last_became_active":"2024-04-13T05:14:52.596079+0000","last_became_peered":"2024-04-13T05:14:52.596079+0000","last_unstale":"2024-04-13T05:16:14.500836+0000","last_undegraded":"2024-04-13T05:16:14.500836+0000","last_fullsized":"2024-04-13T05:16:14.500836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:05:03.201048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595783+0000","last_change":"2024-04-13T05:14:53.604500+0000","last_active":"2024-04-13T05:16:41.595783+0000","last_peered":"2024-04-13T05:16:41.595783+0000","last_clean":"2024-04-13T05:16:41.595783+0000","last_became_active":"2024-04-13T05:14:53.603504+0000","last_became_peered":"2024-04-13T05:14:53.603504+0000","last_unstale":"2024-04-13T05:16:41.595783+0000","last_undegraded":"2024-04-13T05:16:41.595783+0000","last_fullsized":"2024-04-13T05:16:41.595783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:25:10.689862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548990+0000","last_change":"2024-04-13T05:14:58.104215+0000","last_active":"2024-04-13T05:16:14.548990+0000","last_peered":"2024-04-13T05:16:14.548990+0000","last_clean":"2024-04-13T05:16:14.548990+0000","last_became_active":"2024-04-13T05:14:58.102653+0000","last_became_peered":"2024-04-13T05:14:58.102653+0000","last_unstale":"2024-04-13T05:16:14.548990+0000","last_undegraded":"2024-04-13T05:16:14.548990+0000","last_fullsized":"2024-04-13T05:16:14.548990+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:28:55.450071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488274+0000","last_change":"2024-04-13T05:14:52.605155+0000","last_active":"2024-04-13T05:16:14.488274+0000","last_peered":"2024-04-13T05:16:14.488274+0000","last_clean":"2024-04-13T05:16:14.488274+0000","last_became_active":"2024-04-13T05:14:52.604505+0000","last_became_peered":"2024-04-13T05:14:52.604505+0000","last_unstale":"2024-04-13T05:16:14.488274+0000","last_undegraded":"2024-04-13T05:16:14.488274+0000","last_fullsized":"2024-04-13T05:16:14.488274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:17:28.814306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595439+0000","last_change":"2024-04-13T05:14:53.606690+0000","last_active":"2024-04-13T05:16:41.595439+0000","last_peered":"2024-04-13T05:16:41.595439+0000","last_clean":"2024-04-13T05:16:41.595439+0000","last_became_active":"2024-04-13T05:14:53.602111+0000","last_became_peered":"2024-04-13T05:14:53.602111+0000","last_unstale":"2024-04-13T05:16:41.595439+0000","last_undegraded":"2024-04-13T05:16:41.595439+0000","last_fullsized":"2024-04-13T05:16:41.595439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:44:27.790146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205352+0000","last_change":"2024-04-13T05:14:58.109420+0000","last_active":"2024-04-13T05:16:14.205352+0000","last_peered":"2024-04-13T05:16:14.205352+0000","last_clean":"2024-04-13T05:16:14.205352+0000","last_became_active":"2024-04-13T05:14:58.106194+0000","last_became_peered":"2024-04-13T05:14:58.106194+0000","last_unstale":"2024-04-13T05:16:14.205352+0000","last_undegraded":"2024-04-13T05:16:14.205352+0000","last_fullsized":"2024-04-13T05:16:14.205352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:15:02.188847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":188,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:52.798933+0000","last_change":"2024-04-13T05:14:52.607757+0000","last_active":"2024-04-13T05:17:52.798933+0000","last_peered":"2024-04-13T05:17:52.798933+0000","last_clean":"2024-04-13T05:17:52.798933+0000","last_became_active":"2024-04-13T05:14:52.605310+0000","last_became_peered":"2024-04-13T05:14:52.605310+0000","last_unstale":"2024-04-13T05:17:52.798933+0000","last_undegraded":"2024-04-13T05:17:52.798933+0000","last_fullsized":"2024-04-13T05:17:52.798933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:40:17.648385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.599498+0000","last_change":"2024-04-13T05:14:53.598778+0000","last_active":"2024-04-13T05:16:41.599498+0000","last_peered":"2024-04-13T05:16:41.599498+0000","last_clean":"2024-04-13T05:16:41.599498+0000","last_became_active":"2024-04-13T05:14:53.598331+0000","last_became_peered":"2024-04-13T05:14:53.598331+0000","last_unstale":"2024-04-13T05:16:41.599498+0000","last_undegraded":"2024-04-13T05:16:41.599498+0000","last_fullsized":"2024-04-13T05:16:41.599498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:34:44.829382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207859+0000","last_change":"2024-04-13T05:14:58.101438+0000","last_active":"2024-04-13T05:16:14.207859+0000","last_peered":"2024-04-13T05:16:14.207859+0000","last_clean":"2024-04-13T05:16:14.207859+0000","last_became_active":"2024-04-13T05:14:58.100955+0000","last_became_peered":"2024-04-13T05:14:58.100955+0000","last_unstale":"2024-04-13T05:16:14.207859+0000","last_undegraded":"2024-04-13T05:16:14.207859+0000","last_fullsized":"2024-04-13T05:16:14.207859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712372+0000","last_change":"2024-04-13T05:14:52.608487+0000","last_active":"2024-04-13T05:16:14.712372+0000","last_peered":"2024-04-13T05:16:14.712372+0000","last_clean":"2024-04-13T05:16:14.712372+0000","last_became_active":"2024-04-13T05:14:52.606434+0000","last_became_peered":"2024-04-13T05:14:52.606434+0000","last_unstale":"2024-04-13T05:16:14.712372+0000","last_undegraded":"2024-04-13T05:16:14.712372+0000","last_fullsized":"2024-04-13T05:16:14.712372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:12:55.172221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.590058+0000","last_change":"2024-04-13T05:14:53.610567+0000","last_active":"2024-04-13T05:16:41.590058+0000","last_peered":"2024-04-13T05:16:41.590058+0000","last_clean":"2024-04-13T05:16:41.590058+0000","last_became_active":"2024-04-13T05:14:53.607018+0000","last_became_peered":"2024-04-13T05:14:53.607018+0000","last_unstale":"2024-04-13T05:16:41.590058+0000","last_undegraded":"2024-04-13T05:16:41.590058+0000","last_fullsized":"2024-04-13T05:16:41.590058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:50:07.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207958+0000","last_change":"2024-04-13T05:14:58.106402+0000","last_active":"2024-04-13T05:16:14.207958+0000","last_peered":"2024-04-13T05:16:14.207958+0000","last_clean":"2024-04-13T05:16:14.207958+0000","last_became_active":"2024-04-13T05:14:58.106033+0000","last_became_peered":"2024-04-13T05:14:58.106033+0000","last_unstale":"2024-04-13T05:16:14.207958+0000","last_undegraded":"2024-04-13T05:16:14.207958+0000","last_fullsized":"2024-04-13T05:16:14.207958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:09:17.787398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489335+0000","last_change":"2024-04-13T05:14:52.597259+0000","last_active":"2024-04-13T05:16:14.489335+0000","last_peered":"2024-04-13T05:16:14.489335+0000","last_clean":"2024-04-13T05:16:14.489335+0000","last_became_active":"2024-04-13T05:14:52.596655+0000","last_became_peered":"2024-04-13T05:14:52.596655+0000","last_unstale":"2024-04-13T05:16:14.489335+0000","last_undegraded":"2024-04-13T05:16:14.489335+0000","last_fullsized":"2024-04-13T05:16:14.489335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:19:24.388158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601469+0000","last_change":"2024-04-13T05:14:53.603216+0000","last_active":"2024-04-13T05:16:41.601469+0000","last_peered":"2024-04-13T05:16:41.601469+0000","last_clean":"2024-04-13T05:16:41.601469+0000","last_became_active":"2024-04-13T05:14:53.600362+0000","last_became_peered":"2024-04-13T05:14:53.600362+0000","last_unstale":"2024-04-13T05:16:41.601469+0000","last_undegraded":"2024-04-13T05:16:41.601469+0000","last_fullsized":"2024-04-13T05:16:41.601469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:29:47.364422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208081+0000","last_change":"2024-04-13T05:14:58.111165+0000","last_active":"2024-04-13T05:16:14.208081+0000","last_peered":"2024-04-13T05:16:14.208081+0000","last_clean":"2024-04-13T05:16:14.208081+0000","last_became_active":"2024-04-13T05:14:58.110357+0000","last_became_peered":"2024-04-13T05:14:58.110357+0000","last_unstale":"2024-04-13T05:16:14.208081+0000","last_undegraded":"2024-04-13T05:16:14.208081+0000","last_fullsized":"2024-04-13T05:16:14.208081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:29:47.147625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'38","reported_seq":386,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:53.802110+0000","last_change":"2024-04-13T05:14:52.605988+0000","last_active":"2024-04-13T05:17:53.802110+0000","last_peered":"2024-04-13T05:17:53.802110+0000","last_clean":"2024-04-13T05:17:53.802110+0000","last_became_active":"2024-04-13T05:14:52.603609+0000","last_became_peered":"2024-04-13T05:14:52.603609+0000","last_unstale":"2024-04-13T05:17:53.802110+0000","last_undegraded":"2024-04-13T05:17:53.802110+0000","last_fullsized":"2024-04-13T05:17:53.802110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:22:56.251208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67681,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":226,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595726+0000","last_change":"2024-04-13T05:14:53.597951+0000","last_active":"2024-04-13T05:16:41.595726+0000","last_peered":"2024-04-13T05:16:41.595726+0000","last_clean":"2024-04-13T05:16:41.595726+0000","last_became_active":"2024-04-13T05:14:53.597601+0000","last_became_peered":"2024-04-13T05:14:53.597601+0000","last_unstale":"2024-04-13T05:16:41.595726+0000","last_undegraded":"2024-04-13T05:16:41.595726+0000","last_fullsized":"2024-04-13T05:16:41.595726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:42:01.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.490155+0000","last_change":"2024-04-13T05:14:58.106824+0000","last_active":"2024-04-13T05:16:14.490155+0000","last_peered":"2024-04-13T05:16:14.490155+0000","last_clean":"2024-04-13T05:16:14.490155+0000","last_became_active":"2024-04-13T05:14:58.106089+0000","last_became_peered":"2024-04-13T05:14:58.106089+0000","last_unstale":"2024-04-13T05:16:14.490155+0000","last_undegraded":"2024-04-13T05:16:14.490155+0000","last_fullsized":"2024-04-13T05:16:14.490155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:19:36.230676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207832+0000","last_change":"2024-04-13T05:14:52.601467+0000","last_active":"2024-04-13T05:16:14.207832+0000","last_peered":"2024-04-13T05:16:14.207832+0000","last_clean":"2024-04-13T05:16:14.207832+0000","last_became_active":"2024-04-13T05:14:52.600660+0000","last_became_peered":"2024-04-13T05:14:52.600660+0000","last_unstale":"2024-04-13T05:16:14.207832+0000","last_undegraded":"2024-04-13T05:16:14.207832+0000","last_fullsized":"2024-04-13T05:16:14.207832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:32:00.157010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591125+0000","last_change":"2024-04-13T05:14:53.593968+0000","last_active":"2024-04-13T05:16:41.591125+0000","last_peered":"2024-04-13T05:16:41.591125+0000","last_clean":"2024-04-13T05:16:41.591125+0000","last_became_active":"2024-04-13T05:14:53.593613+0000","last_became_peered":"2024-04-13T05:14:53.593613+0000","last_unstale":"2024-04-13T05:16:41.591125+0000","last_undegraded":"2024-04-13T05:16:41.591125+0000","last_fullsized":"2024-04-13T05:16:41.591125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:12:06.998048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214510+0000","last_change":"2024-04-13T05:14:58.115642+0000","last_active":"2024-04-13T05:16:14.214510+0000","last_peered":"2024-04-13T05:16:14.214510+0000","last_clean":"2024-04-13T05:16:14.214510+0000","last_became_active":"2024-04-13T05:14:58.115118+0000","last_became_peered":"2024-04-13T05:14:58.115118+0000","last_unstale":"2024-04-13T05:16:14.214510+0000","last_undegraded":"2024-04-13T05:16:14.214510+0000","last_fullsized":"2024-04-13T05:16:14.214510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:20:32.181740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488900+0000","last_change":"2024-04-13T05:14:52.606503+0000","last_active":"2024-04-13T05:16:14.488900+0000","last_peered":"2024-04-13T05:16:14.488900+0000","last_clean":"2024-04-13T05:16:14.488900+0000","last_became_active":"2024-04-13T05:14:52.606131+0000","last_became_peered":"2024-04-13T05:14:52.606131+0000","last_unstale":"2024-04-13T05:16:14.488900+0000","last_undegraded":"2024-04-13T05:16:14.488900+0000","last_fullsized":"2024-04-13T05:16:14.488900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:20.338051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596593+0000","last_change":"2024-04-13T05:14:53.614798+0000","last_active":"2024-04-13T05:16:41.596593+0000","last_peered":"2024-04-13T05:16:41.596593+0000","last_clean":"2024-04-13T05:16:41.596593+0000","last_became_active":"2024-04-13T05:14:53.614447+0000","last_became_peered":"2024-04-13T05:14:53.614447+0000","last_unstale":"2024-04-13T05:16:41.596593+0000","last_undegraded":"2024-04-13T05:16:41.596593+0000","last_fullsized":"2024-04-13T05:16:41.596593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:12:49.145424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208701+0000","last_change":"2024-04-13T05:14:58.113293+0000","last_active":"2024-04-13T05:16:14.208701+0000","last_peered":"2024-04-13T05:16:14.208701+0000","last_clean":"2024-04-13T05:16:14.208701+0000","last_became_active":"2024-04-13T05:14:58.105047+0000","last_became_peered":"2024-04-13T05:14:58.105047+0000","last_unstale":"2024-04-13T05:16:14.208701+0000","last_undegraded":"2024-04-13T05:16:14.208701+0000","last_fullsized":"2024-04-13T05:16:14.208701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:59:25.161399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:51.804836+0000","last_change":"2024-04-13T05:14:52.606500+0000","last_active":"2024-04-13T05:17:51.804836+0000","last_peered":"2024-04-13T05:17:51.804836+0000","last_clean":"2024-04-13T05:17:51.804836+0000","last_became_active":"2024-04-13T05:14:52.606125+0000","last_became_peered":"2024-04-13T05:14:52.606125+0000","last_unstale":"2024-04-13T05:17:51.804836+0000","last_undegraded":"2024-04-13T05:17:51.804836+0000","last_fullsized":"2024-04-13T05:17:51.804836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:08.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593652+0000","last_change":"2024-04-13T05:14:53.596213+0000","last_active":"2024-04-13T05:16:41.593652+0000","last_peered":"2024-04-13T05:16:41.593652+0000","last_clean":"2024-04-13T05:16:41.593652+0000","last_became_active":"2024-04-13T05:14:53.595896+0000","last_became_peered":"2024-04-13T05:14:53.595896+0000","last_unstale":"2024-04-13T05:16:41.593652+0000","last_undegraded":"2024-04-13T05:16:41.593652+0000","last_fullsized":"2024-04-13T05:16:41.593652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:44:56.059911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.502070+0000","last_change":"2024-04-13T05:14:58.101493+0000","last_active":"2024-04-13T05:16:14.502070+0000","last_peered":"2024-04-13T05:16:14.502070+0000","last_clean":"2024-04-13T05:16:14.502070+0000","last_became_active":"2024-04-13T05:14:58.100208+0000","last_became_peered":"2024-04-13T05:14:58.100208+0000","last_unstale":"2024-04-13T05:16:14.502070+0000","last_undegraded":"2024-04-13T05:16:14.502070+0000","last_fullsized":"2024-04-13T05:16:14.502070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:30:26.713096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709866+0000","last_change":"2024-04-13T05:15:13.207505+0000","last_active":"2024-04-13T05:16:14.709866+0000","last_peered":"2024-04-13T05:16:14.709866+0000","last_clean":"2024-04-13T05:16:14.709866+0000","last_became_active":"2024-04-13T05:15:13.192737+0000","last_became_peered":"2024-04-13T05:15:13.192737+0000","last_unstale":"2024-04-13T05:16:14.709866+0000","last_undegraded":"2024-04-13T05:16:14.709866+0000","last_fullsized":"2024-04-13T05:16:14.709866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:11:38.215810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600587+0000","last_change":"2024-04-13T05:14:53.606683+0000","last_active":"2024-04-13T05:16:41.600587+0000","last_peered":"2024-04-13T05:16:41.600587+0000","last_clean":"2024-04-13T05:16:41.600587+0000","last_became_active":"2024-04-13T05:14:53.603067+0000","last_became_peered":"2024-04-13T05:14:53.603067+0000","last_unstale":"2024-04-13T05:16:41.600587+0000","last_undegraded":"2024-04-13T05:16:41.600587+0000","last_fullsized":"2024-04-13T05:16:41.600587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:25:52.059729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.488458+0000","last_change":"2024-04-13T05:14:58.110785+0000","last_active":"2024-04-13T05:16:14.488458+0000","last_peered":"2024-04-13T05:16:14.488458+0000","last_clean":"2024-04-13T05:16:14.488458+0000","last_became_active":"2024-04-13T05:14:58.109755+0000","last_became_peered":"2024-04-13T05:14:58.109755+0000","last_unstale":"2024-04-13T05:16:14.488458+0000","last_undegraded":"2024-04-13T05:16:14.488458+0000","last_fullsized":"2024-04-13T05:16:14.488458+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:11.164938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.710704+0000","last_change":"2024-04-13T05:14:52.603525+0000","last_active":"2024-04-13T05:16:14.710704+0000","last_peered":"2024-04-13T05:16:14.710704+0000","last_clean":"2024-04-13T05:16:14.710704+0000","last_became_active":"2024-04-13T05:14:52.603138+0000","last_became_peered":"2024-04-13T05:14:52.603138+0000","last_unstale":"2024-04-13T05:16:14.710704+0000","last_undegraded":"2024-04-13T05:16:14.710704+0000","last_fullsized":"2024-04-13T05:16:14.710704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T17:04:27.199002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584084+0000","last_change":"2024-04-13T05:15:13.190714+0000","last_active":"2024-04-13T05:16:41.584084+0000","last_peered":"2024-04-13T05:16:41.584084+0000","last_clean":"2024-04-13T05:16:41.584084+0000","last_became_active":"2024-04-13T05:15:13.190276+0000","last_became_peered":"2024-04-13T05:15:13.190276+0000","last_unstale":"2024-04-13T05:16:41.584084+0000","last_undegraded":"2024-04-13T05:16:41.584084+0000","last_fullsized":"2024-04-13T05:16:41.584084+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:10:18.564302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711265+0000","last_change":"2024-04-13T05:14:58.099574+0000","last_active":"2024-04-13T05:16:14.711265+0000","last_peered":"2024-04-13T05:16:14.711265+0000","last_clean":"2024-04-13T05:16:14.711265+0000","last_became_active":"2024-04-13T05:14:58.099075+0000","last_became_peered":"2024-04-13T05:14:58.099075+0000","last_unstale":"2024-04-13T05:16:14.711265+0000","last_undegraded":"2024-04-13T05:16:14.711265+0000","last_fullsized":"2024-04-13T05:16:14.711265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:54:49.398957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208277+0000","last_change":"2024-04-13T05:14:52.606871+0000","last_active":"2024-04-13T05:16:14.208277+0000","last_peered":"2024-04-13T05:16:14.208277+0000","last_clean":"2024-04-13T05:16:14.208277+0000","last_became_active":"2024-04-13T05:14:52.606478+0000","last_became_peered":"2024-04-13T05:14:52.606478+0000","last_unstale":"2024-04-13T05:16:14.208277+0000","last_undegraded":"2024-04-13T05:16:14.208277+0000","last_fullsized":"2024-04-13T05:16:14.208277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:16.122986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597318+0000","last_change":"2024-04-13T05:14:53.589177+0000","last_active":"2024-04-13T05:16:41.597318+0000","last_peered":"2024-04-13T05:16:41.597318+0000","last_clean":"2024-04-13T05:16:41.597318+0000","last_became_active":"2024-04-13T05:14:53.588690+0000","last_became_peered":"2024-04-13T05:14:53.588690+0000","last_unstale":"2024-04-13T05:16:41.597318+0000","last_undegraded":"2024-04-13T05:16:41.597318+0000","last_fullsized":"2024-04-13T05:16:41.597318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:19:47.193931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501299+0000","last_change":"2024-04-13T05:14:58.096395+0000","last_active":"2024-04-13T05:16:14.501299+0000","last_peered":"2024-04-13T05:16:14.501299+0000","last_clean":"2024-04-13T05:16:14.501299+0000","last_became_active":"2024-04-13T05:14:58.095946+0000","last_became_peered":"2024-04-13T05:14:58.095946+0000","last_unstale":"2024-04-13T05:16:14.501299+0000","last_undegraded":"2024-04-13T05:16:14.501299+0000","last_fullsized":"2024-04-13T05:16:14.501299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:13:09.506809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711835+0000","last_change":"2024-04-13T05:14:52.609661+0000","last_active":"2024-04-13T05:16:14.711835+0000","last_peered":"2024-04-13T05:16:14.711835+0000","last_clean":"2024-04-13T05:16:14.711835+0000","last_became_active":"2024-04-13T05:14:52.605888+0000","last_became_peered":"2024-04-13T05:14:52.605888+0000","last_unstale":"2024-04-13T05:16:14.711835+0000","last_undegraded":"2024-04-13T05:16:14.711835+0000","last_fullsized":"2024-04-13T05:16:14.711835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:07:42.058198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591498+0000","last_change":"2024-04-13T05:14:53.602120+0000","last_active":"2024-04-13T05:16:41.591498+0000","last_peered":"2024-04-13T05:16:41.591498+0000","last_clean":"2024-04-13T05:16:41.591498+0000","last_became_active":"2024-04-13T05:14:53.601786+0000","last_became_peered":"2024-04-13T05:14:53.601786+0000","last_unstale":"2024-04-13T05:16:41.591498+0000","last_undegraded":"2024-04-13T05:16:41.591498+0000","last_fullsized":"2024-04-13T05:16:41.591498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:37:24.040893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501784+0000","last_change":"2024-04-13T05:14:58.101507+0000","last_active":"2024-04-13T05:16:14.501784+0000","last_peered":"2024-04-13T05:16:14.501784+0000","last_clean":"2024-04-13T05:16:14.501784+0000","last_became_active":"2024-04-13T05:14:58.100115+0000","last_became_peered":"2024-04-13T05:14:58.100115+0000","last_unstale":"2024-04-13T05:16:14.501784+0000","last_undegraded":"2024-04-13T05:16:14.501784+0000","last_fullsized":"2024-04-13T05:16:14.501784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:17:47.001861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587661+0000","last_change":"2024-04-13T05:14:53.602310+0000","last_active":"2024-04-13T05:16:41.587661+0000","last_peered":"2024-04-13T05:16:41.587661+0000","last_clean":"2024-04-13T05:16:41.587661+0000","last_became_active":"2024-04-13T05:14:53.601874+0000","last_became_peered":"2024-04-13T05:14:53.601874+0000","last_unstale":"2024-04-13T05:16:41.587661+0000","last_undegraded":"2024-04-13T05:16:41.587661+0000","last_fullsized":"2024-04-13T05:16:41.587661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:39:29.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":190,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:52.806787+0000","last_change":"2024-04-13T05:14:52.596669+0000","last_active":"2024-04-13T05:17:52.806787+0000","last_peered":"2024-04-13T05:17:52.806787+0000","last_clean":"2024-04-13T05:17:52.806787+0000","last_became_active":"2024-04-13T05:14:52.595919+0000","last_became_peered":"2024-04-13T05:14:52.595919+0000","last_unstale":"2024-04-13T05:17:52.806787+0000","last_undegraded":"2024-04-13T05:17:52.806787+0000","last_fullsized":"2024-04-13T05:17:52.806787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:42:32.259849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.712596+0000","last_change":"2024-04-13T05:14:58.104569+0000","last_active":"2024-04-13T05:16:14.712596+0000","last_peered":"2024-04-13T05:16:14.712596+0000","last_clean":"2024-04-13T05:16:14.712596+0000","last_became_active":"2024-04-13T05:14:58.104087+0000","last_became_peered":"2024-04-13T05:14:58.104087+0000","last_unstale":"2024-04-13T05:16:14.712596+0000","last_undegraded":"2024-04-13T05:16:14.712596+0000","last_fullsized":"2024-04-13T05:16:14.712596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:29:21.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.602040+0000","last_change":"2024-04-13T05:14:53.602549+0000","last_active":"2024-04-13T05:16:41.602040+0000","last_peered":"2024-04-13T05:16:41.602040+0000","last_clean":"2024-04-13T05:16:41.602040+0000","last_became_active":"2024-04-13T05:14:53.602001+0000","last_became_peered":"2024-04-13T05:14:53.602001+0000","last_unstale":"2024-04-13T05:16:41.602040+0000","last_undegraded":"2024-04-13T05:16:41.602040+0000","last_fullsized":"2024-04-13T05:16:41.602040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:06:02.094913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207400+0000","last_change":"2024-04-13T05:14:52.601157+0000","last_active":"2024-04-13T05:16:14.207400+0000","last_peered":"2024-04-13T05:16:14.207400+0000","last_clean":"2024-04-13T05:16:14.207400+0000","last_became_active":"2024-04-13T05:14:52.600320+0000","last_became_peered":"2024-04-13T05:14:52.600320+0000","last_unstale":"2024-04-13T05:16:14.207400+0000","last_undegraded":"2024-04-13T05:16:14.207400+0000","last_fullsized":"2024-04-13T05:16:14.207400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:18:49.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711063+0000","last_change":"2024-04-13T05:13:13.135399+0000","last_active":"2024-04-13T05:16:14.711063+0000","last_peered":"2024-04-13T05:16:14.711063+0000","last_clean":"2024-04-13T05:16:14.711063+0000","last_became_active":"2024-04-13T05:13:13.134439+0000","last_became_peered":"2024-04-13T05:13:13.134439+0000","last_unstale":"2024-04-13T05:16:14.711063+0000","last_undegraded":"2024-04-13T05:16:14.711063+0000","last_fullsized":"2024-04-13T05:16:14.711063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:08:29.932305+0000","last_clean_scrub_stamp":"2024-04-13T05:08:29.932305+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:47:46.167849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.490158+0000","last_change":"2024-04-13T05:15:13.196100+0000","last_active":"2024-04-13T05:16:14.490158+0000","last_peered":"2024-04-13T05:16:14.490158+0000","last_clean":"2024-04-13T05:16:14.490158+0000","last_became_active":"2024-04-13T05:15:13.195473+0000","last_became_peered":"2024-04-13T05:15:13.195473+0000","last_unstale":"2024-04-13T05:16:14.490158+0000","last_undegraded":"2024-04-13T05:16:14.490158+0000","last_fullsized":"2024-04-13T05:16:14.490158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:49:31.610524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574537+0000","last_change":"2024-04-13T05:14:53.598204+0000","last_active":"2024-04-13T05:16:41.574537+0000","last_peered":"2024-04-13T05:16:41.574537+0000","last_clean":"2024-04-13T05:16:41.574537+0000","last_became_active":"2024-04-13T05:14:53.597652+0000","last_became_peered":"2024-04-13T05:14:53.597652+0000","last_unstale":"2024-04-13T05:16:41.574537+0000","last_undegraded":"2024-04-13T05:16:41.574537+0000","last_fullsized":"2024-04-13T05:16:41.574537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:46:41.311382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711427+0000","last_change":"2024-04-13T05:14:52.609425+0000","last_active":"2024-04-13T05:16:14.711427+0000","last_peered":"2024-04-13T05:16:14.711427+0000","last_clean":"2024-04-13T05:16:14.711427+0000","last_became_active":"2024-04-13T05:14:52.609097+0000","last_became_peered":"2024-04-13T05:14:52.609097+0000","last_unstale":"2024-04-13T05:16:14.711427+0000","last_undegraded":"2024-04-13T05:16:14.711427+0000","last_fullsized":"2024-04-13T05:16:14.711427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:25:49.322867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.214710+0000","last_change":"2024-04-13T05:14:58.103098+0000","last_active":"2024-04-13T05:16:14.214710+0000","last_peered":"2024-04-13T05:16:14.214710+0000","last_clean":"2024-04-13T05:16:14.214710+0000","last_became_active":"2024-04-13T05:14:58.102615+0000","last_became_peered":"2024-04-13T05:14:58.102615+0000","last_unstale":"2024-04-13T05:16:14.214710+0000","last_undegraded":"2024-04-13T05:16:14.214710+0000","last_fullsized":"2024-04-13T05:16:14.214710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:37:12.694415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711158+0000","last_change":"2024-04-13T05:14:52.609734+0000","last_active":"2024-04-13T05:16:14.711158+0000","last_peered":"2024-04-13T05:16:14.711158+0000","last_clean":"2024-04-13T05:16:14.711158+0000","last_became_active":"2024-04-13T05:14:52.608641+0000","last_became_peered":"2024-04-13T05:14:52.608641+0000","last_unstale":"2024-04-13T05:16:14.711158+0000","last_undegraded":"2024-04-13T05:16:14.711158+0000","last_fullsized":"2024-04-13T05:16:14.711158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T11:43:21.106108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.574744+0000","last_change":"2024-04-13T05:14:53.591792+0000","last_active":"2024-04-13T05:16:41.574744+0000","last_peered":"2024-04-13T05:16:41.574744+0000","last_clean":"2024-04-13T05:16:41.574744+0000","last_became_active":"2024-04-13T05:14:53.590256+0000","last_became_peered":"2024-04-13T05:14:53.590256+0000","last_unstale":"2024-04-13T05:16:41.574744+0000","last_undegraded":"2024-04-13T05:16:41.574744+0000","last_fullsized":"2024-04-13T05:16:41.574744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:00:16.525270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205197+0000","last_change":"2024-04-13T05:14:58.107002+0000","last_active":"2024-04-13T05:16:14.205197+0000","last_peered":"2024-04-13T05:16:14.205197+0000","last_clean":"2024-04-13T05:16:14.205197+0000","last_became_active":"2024-04-13T05:14:58.106494+0000","last_became_peered":"2024-04-13T05:14:58.106494+0000","last_unstale":"2024-04-13T05:16:14.205197+0000","last_undegraded":"2024-04-13T05:16:14.205197+0000","last_fullsized":"2024-04-13T05:16:14.205197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:52:39.404732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:05.240813+0000","last_change":"2024-04-13T05:14:52.596772+0000","last_active":"2024-04-13T05:17:05.240813+0000","last_peered":"2024-04-13T05:17:05.240813+0000","last_clean":"2024-04-13T05:17:05.240813+0000","last_became_active":"2024-04-13T05:14:52.594161+0000","last_became_peered":"2024-04-13T05:14:52.594161+0000","last_unstale":"2024-04-13T05:17:05.240813+0000","last_undegraded":"2024-04-13T05:17:05.240813+0000","last_fullsized":"2024-04-13T05:17:05.240813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:31:23.842355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.595268+0000","last_change":"2024-04-13T05:14:53.602205+0000","last_active":"2024-04-13T05:16:41.595268+0000","last_peered":"2024-04-13T05:16:41.595268+0000","last_clean":"2024-04-13T05:16:41.595268+0000","last_became_active":"2024-04-13T05:14:53.601813+0000","last_became_peered":"2024-04-13T05:14:53.601813+0000","last_unstale":"2024-04-13T05:16:41.595268+0000","last_undegraded":"2024-04-13T05:16:41.595268+0000","last_fullsized":"2024-04-13T05:16:41.595268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:10:03.172369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207010+0000","last_change":"2024-04-13T05:14:58.105204+0000","last_active":"2024-04-13T05:16:14.207010+0000","last_peered":"2024-04-13T05:16:14.207010+0000","last_clean":"2024-04-13T05:16:14.207010+0000","last_became_active":"2024-04-13T05:14:58.104794+0000","last_became_peered":"2024-04-13T05:14:58.104794+0000","last_unstale":"2024-04-13T05:16:14.207010+0000","last_undegraded":"2024-04-13T05:16:14.207010+0000","last_fullsized":"2024-04-13T05:16:14.207010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:22:35.764940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.209094+0000","last_change":"2024-04-13T05:14:52.608644+0000","last_active":"2024-04-13T05:16:14.209094+0000","last_peered":"2024-04-13T05:16:14.209094+0000","last_clean":"2024-04-13T05:16:14.209094+0000","last_became_active":"2024-04-13T05:14:52.608227+0000","last_became_peered":"2024-04-13T05:14:52.608227+0000","last_unstale":"2024-04-13T05:16:14.209094+0000","last_undegraded":"2024-04-13T05:16:14.209094+0000","last_fullsized":"2024-04-13T05:16:14.209094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:41:31.491346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596065+0000","last_change":"2024-04-13T05:14:53.605721+0000","last_active":"2024-04-13T05:16:41.596065+0000","last_peered":"2024-04-13T05:16:41.596065+0000","last_clean":"2024-04-13T05:16:41.596065+0000","last_became_active":"2024-04-13T05:14:53.605356+0000","last_became_peered":"2024-04-13T05:14:53.605356+0000","last_unstale":"2024-04-13T05:16:41.596065+0000","last_undegraded":"2024-04-13T05:16:41.596065+0000","last_fullsized":"2024-04-13T05:16:41.596065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:05:24.603806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208632+0000","last_change":"2024-04-13T05:14:58.101704+0000","last_active":"2024-04-13T05:16:14.208632+0000","last_peered":"2024-04-13T05:16:14.208632+0000","last_clean":"2024-04-13T05:16:14.208632+0000","last_became_active":"2024-04-13T05:14:58.100739+0000","last_became_peered":"2024-04-13T05:14:58.100739+0000","last_unstale":"2024-04-13T05:16:14.208632+0000","last_undegraded":"2024-04-13T05:16:14.208632+0000","last_fullsized":"2024-04-13T05:16:14.208632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:17:05.180668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709476+0000","last_change":"2024-04-13T05:14:52.604569+0000","last_active":"2024-04-13T05:16:14.709476+0000","last_peered":"2024-04-13T05:16:14.709476+0000","last_clean":"2024-04-13T05:16:14.709476+0000","last_became_active":"2024-04-13T05:14:52.604157+0000","last_became_peered":"2024-04-13T05:14:52.604157+0000","last_unstale":"2024-04-13T05:16:14.709476+0000","last_undegraded":"2024-04-13T05:16:14.709476+0000","last_fullsized":"2024-04-13T05:16:14.709476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:57:01.373724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586590+0000","last_change":"2024-04-13T05:14:53.583821+0000","last_active":"2024-04-13T05:16:41.586590+0000","last_peered":"2024-04-13T05:16:41.586590+0000","last_clean":"2024-04-13T05:16:41.586590+0000","last_became_active":"2024-04-13T05:14:53.583065+0000","last_became_peered":"2024-04-13T05:14:53.583065+0000","last_unstale":"2024-04-13T05:16:41.586590+0000","last_undegraded":"2024-04-13T05:16:41.586590+0000","last_fullsized":"2024-04-13T05:16:41.586590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:10:02.064905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208596+0000","last_change":"2024-04-13T05:14:58.109122+0000","last_active":"2024-04-13T05:16:14.208596+0000","last_peered":"2024-04-13T05:16:14.208596+0000","last_clean":"2024-04-13T05:16:14.208596+0000","last_became_active":"2024-04-13T05:14:58.106341+0000","last_became_peered":"2024-04-13T05:14:58.106341+0000","last_unstale":"2024-04-13T05:16:14.208596+0000","last_undegraded":"2024-04-13T05:16:14.208596+0000","last_fullsized":"2024-04-13T05:16:14.208596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:29.011248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.205469+0000","last_change":"2024-04-13T05:14:52.600749+0000","last_active":"2024-04-13T05:16:14.205469+0000","last_peered":"2024-04-13T05:16:14.205469+0000","last_clean":"2024-04-13T05:16:14.205469+0000","last_became_active":"2024-04-13T05:14:52.595643+0000","last_became_peered":"2024-04-13T05:14:52.595643+0000","last_unstale":"2024-04-13T05:16:14.205469+0000","last_undegraded":"2024-04-13T05:16:14.205469+0000","last_fullsized":"2024-04-13T05:16:14.205469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:26:11.522203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591578+0000","last_change":"2024-04-13T05:14:53.603342+0000","last_active":"2024-04-13T05:16:41.591578+0000","last_peered":"2024-04-13T05:16:41.591578+0000","last_clean":"2024-04-13T05:16:41.591578+0000","last_became_active":"2024-04-13T05:14:53.602912+0000","last_became_peered":"2024-04-13T05:14:53.602912+0000","last_unstale":"2024-04-13T05:16:41.591578+0000","last_undegraded":"2024-04-13T05:16:41.591578+0000","last_fullsized":"2024-04-13T05:16:41.591578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:40:18.246850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.207219+0000","last_change":"2024-04-13T05:14:58.108039+0000","last_active":"2024-04-13T05:16:14.207219+0000","last_peered":"2024-04-13T05:16:14.207219+0000","last_clean":"2024-04-13T05:16:14.207219+0000","last_became_active":"2024-04-13T05:14:58.107598+0000","last_became_peered":"2024-04-13T05:14:58.107598+0000","last_unstale":"2024-04-13T05:16:14.207219+0000","last_undegraded":"2024-04-13T05:16:14.207219+0000","last_fullsized":"2024-04-13T05:16:14.207219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:50:19.006522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:17:25.449178+0000","last_change":"2024-04-13T05:15:13.178075+0000","last_active":"2024-04-13T05:17:25.449178+0000","last_peered":"2024-04-13T05:17:25.449178+0000","last_clean":"2024-04-13T05:17:25.449178+0000","last_became_active":"2024-04-13T05:15:13.177643+0000","last_became_peered":"2024-04-13T05:15:13.177643+0000","last_unstale":"2024-04-13T05:17:25.449178+0000","last_undegraded":"2024-04-13T05:17:25.449178+0000","last_fullsized":"2024-04-13T05:17:25.449178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:22:11.254785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.597776+0000","last_change":"2024-04-13T05:14:53.611020+0000","last_active":"2024-04-13T05:16:41.597776+0000","last_peered":"2024-04-13T05:16:41.597776+0000","last_clean":"2024-04-13T05:16:41.597776+0000","last_became_active":"2024-04-13T05:14:53.610630+0000","last_became_peered":"2024-04-13T05:14:53.610630+0000","last_unstale":"2024-04-13T05:16:41.597776+0000","last_undegraded":"2024-04-13T05:16:41.597776+0000","last_fullsized":"2024-04-13T05:16:41.597776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:33:52.807938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208228+0000","last_change":"2024-04-13T05:14:58.105149+0000","last_active":"2024-04-13T05:16:14.208228+0000","last_peered":"2024-04-13T05:16:14.208228+0000","last_clean":"2024-04-13T05:16:14.208228+0000","last_became_active":"2024-04-13T05:14:58.104618+0000","last_became_peered":"2024-04-13T05:14:58.104618+0000","last_unstale":"2024-04-13T05:16:14.208228+0000","last_undegraded":"2024-04-13T05:16:14.208228+0000","last_fullsized":"2024-04-13T05:16:14.208228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:00:55.279889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.711142+0000","last_change":"2024-04-13T05:14:52.609722+0000","last_active":"2024-04-13T05:16:14.711142+0000","last_peered":"2024-04-13T05:16:14.711142+0000","last_clean":"2024-04-13T05:16:14.711142+0000","last_became_active":"2024-04-13T05:14:52.609343+0000","last_became_peered":"2024-04-13T05:14:52.609343+0000","last_unstale":"2024-04-13T05:16:14.711142+0000","last_undegraded":"2024-04-13T05:16:14.711142+0000","last_fullsized":"2024-04-13T05:16:14.711142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:00:28.347603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.602016+0000","last_change":"2024-04-13T05:14:53.595747+0000","last_active":"2024-04-13T05:16:41.602016+0000","last_peered":"2024-04-13T05:16:41.602016+0000","last_clean":"2024-04-13T05:16:41.602016+0000","last_became_active":"2024-04-13T05:14:53.595206+0000","last_became_peered":"2024-04-13T05:14:53.595206+0000","last_unstale":"2024-04-13T05:16:41.602016+0000","last_undegraded":"2024-04-13T05:16:41.602016+0000","last_fullsized":"2024-04-13T05:16:41.602016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:29:28.962919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.709227+0000","last_change":"2024-04-13T05:14:58.101397+0000","last_active":"2024-04-13T05:16:14.709227+0000","last_peered":"2024-04-13T05:16:14.709227+0000","last_clean":"2024-04-13T05:16:14.709227+0000","last_became_active":"2024-04-13T05:14:58.100953+0000","last_became_peered":"2024-04-13T05:14:58.100953+0000","last_unstale":"2024-04-13T05:16:14.709227+0000","last_undegraded":"2024-04-13T05:16:14.709227+0000","last_fullsized":"2024-04-13T05:16:14.709227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:39:55.151356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.208151+0000","last_change":"2024-04-13T05:14:52.607829+0000","last_active":"2024-04-13T05:16:14.208151+0000","last_peered":"2024-04-13T05:16:14.208151+0000","last_clean":"2024-04-13T05:16:14.208151+0000","last_became_active":"2024-04-13T05:14:52.607469+0000","last_became_peered":"2024-04-13T05:14:52.607469+0000","last_unstale":"2024-04-13T05:16:14.208151+0000","last_undegraded":"2024-04-13T05:16:14.208151+0000","last_fullsized":"2024-04-13T05:16:14.208151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:58.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.601798+0000","last_change":"2024-04-13T05:14:53.602598+0000","last_active":"2024-04-13T05:16:41.601798+0000","last_peered":"2024-04-13T05:16:41.601798+0000","last_clean":"2024-04-13T05:16:41.601798+0000","last_became_active":"2024-04-13T05:14:53.602177+0000","last_became_peered":"2024-04-13T05:14:53.602177+0000","last_unstale":"2024-04-13T05:16:41.601798+0000","last_undegraded":"2024-04-13T05:16:41.601798+0000","last_fullsized":"2024-04-13T05:16:41.601798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:15:55.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.548391+0000","last_change":"2024-04-13T05:14:58.100315+0000","last_active":"2024-04-13T05:16:14.548391+0000","last_peered":"2024-04-13T05:16:14.548391+0000","last_clean":"2024-04-13T05:16:14.548391+0000","last_became_active":"2024-04-13T05:14:58.099841+0000","last_became_peered":"2024-04-13T05:14:58.099841+0000","last_unstale":"2024-04-13T05:16:14.548391+0000","last_undegraded":"2024-04-13T05:16:14.548391+0000","last_fullsized":"2024-04-13T05:16:14.548391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:24:24.694541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.489672+0000","last_change":"2024-04-13T05:14:52.595536+0000","last_active":"2024-04-13T05:16:14.489672+0000","last_peered":"2024-04-13T05:16:14.489672+0000","last_clean":"2024-04-13T05:16:14.489672+0000","last_became_active":"2024-04-13T05:14:52.595179+0000","last_became_peered":"2024-04-13T05:14:52.595179+0000","last_unstale":"2024-04-13T05:16:14.489672+0000","last_undegraded":"2024-04-13T05:16:14.489672+0000","last_fullsized":"2024-04-13T05:16:14.489672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:05:28.580361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.585990+0000","last_change":"2024-04-13T05:14:53.603172+0000","last_active":"2024-04-13T05:16:41.585990+0000","last_peered":"2024-04-13T05:16:41.585990+0000","last_clean":"2024-04-13T05:16:41.585990+0000","last_became_active":"2024-04-13T05:14:53.602797+0000","last_became_peered":"2024-04-13T05:14:53.602797+0000","last_unstale":"2024-04-13T05:16:41.585990+0000","last_undegraded":"2024-04-13T05:16:41.585990+0000","last_fullsized":"2024-04-13T05:16:41.585990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T16:24:51.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.487926+0000","last_change":"2024-04-13T05:14:58.104790+0000","last_active":"2024-04-13T05:16:14.487926+0000","last_peered":"2024-04-13T05:16:14.487926+0000","last_clean":"2024-04-13T05:16:14.487926+0000","last_became_active":"2024-04-13T05:14:58.104321+0000","last_became_peered":"2024-04-13T05:14:58.104321+0000","last_unstale":"2024-04-13T05:16:14.487926+0000","last_undegraded":"2024-04-13T05:16:14.487926+0000","last_fullsized":"2024-04-13T05:16:14.487926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:57.057317+0000","last_clean_scrub_stamp":"2024-04-13T05:14:57.057317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:38:43.487232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:14.501896+0000","last_change":"2024-04-13T05:14:52.600653+0000","last_active":"2024-04-13T05:16:14.501896+0000","last_peered":"2024-04-13T05:16:14.501896+0000","last_clean":"2024-04-13T05:16:14.501896+0000","last_became_active":"2024-04-13T05:14:52.599189+0000","last_became_peered":"2024-04-13T05:14:52.599189+0000","last_unstale":"2024-04-13T05:16:14.501896+0000","last_undegraded":"2024-04-13T05:16:14.501896+0000","last_fullsized":"2024-04-13T05:16:14.501896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:51.551017+0000","last_clean_scrub_stamp":"2024-04-13T05:14:51.551017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:32:07.382820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.594768+0000","last_change":"2024-04-13T05:14:53.605580+0000","last_active":"2024-04-13T05:16:41.594768+0000","last_peered":"2024-04-13T05:16:41.594768+0000","last_clean":"2024-04-13T05:16:41.594768+0000","last_became_active":"2024-04-13T05:14:53.601979+0000","last_became_peered":"2024-04-13T05:14:53.601979+0000","last_unstale":"2024-04-13T05:16:41.594768+0000","last_undegraded":"2024-04-13T05:16:41.594768+0000","last_fullsized":"2024-04-13T05:16:41.594768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T09:31:37.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.584585+0000","last_change":"2024-04-13T05:14:53.609142+0000","last_active":"2024-04-13T05:16:41.584585+0000","last_peered":"2024-04-13T05:16:41.584585+0000","last_clean":"2024-04-13T05:16:41.584585+0000","last_became_active":"2024-04-13T05:14:53.607884+0000","last_became_peered":"2024-04-13T05:14:53.607884+0000","last_unstale":"2024-04-13T05:16:41.584585+0000","last_undegraded":"2024-04-13T05:16:41.584585+0000","last_fullsized":"2024-04-13T05:16:41.584585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:36:45.903983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.587156+0000","last_change":"2024-04-13T05:14:53.603044+0000","last_active":"2024-04-13T05:16:41.587156+0000","last_peered":"2024-04-13T05:16:41.587156+0000","last_clean":"2024-04-13T05:16:41.587156+0000","last_became_active":"2024-04-13T05:14:53.602703+0000","last_became_peered":"2024-04-13T05:14:53.602703+0000","last_unstale":"2024-04-13T05:16:41.587156+0000","last_undegraded":"2024-04-13T05:16:41.587156+0000","last_fullsized":"2024-04-13T05:16:41.587156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T12:29:12.198401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.592009+0000","last_change":"2024-04-13T05:14:53.596222+0000","last_active":"2024-04-13T05:16:41.592009+0000","last_peered":"2024-04-13T05:16:41.592009+0000","last_clean":"2024-04-13T05:16:41.592009+0000","last_became_active":"2024-04-13T05:14:53.595812+0000","last_became_peered":"2024-04-13T05:14:53.595812+0000","last_unstale":"2024-04-13T05:16:41.592009+0000","last_undegraded":"2024-04-13T05:16:41.592009+0000","last_fullsized":"2024-04-13T05:16:41.592009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T13:41:51.289927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.591014+0000","last_change":"2024-04-13T05:14:53.610261+0000","last_active":"2024-04-13T05:16:41.591014+0000","last_peered":"2024-04-13T05:16:41.591014+0000","last_clean":"2024-04-13T05:16:41.591014+0000","last_became_active":"2024-04-13T05:14:53.609467+0000","last_became_peered":"2024-04-13T05:14:53.609467+0000","last_unstale":"2024-04-13T05:16:41.591014+0000","last_undegraded":"2024-04-13T05:16:41.591014+0000","last_fullsized":"2024-04-13T05:16:41.591014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:30:15.998826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.586928+0000","last_change":"2024-04-13T05:14:53.598654+0000","last_active":"2024-04-13T05:16:41.586928+0000","last_peered":"2024-04-13T05:16:41.586928+0000","last_clean":"2024-04-13T05:16:41.586928+0000","last_became_active":"2024-04-13T05:14:53.597594+0000","last_became_peered":"2024-04-13T05:14:53.597594+0000","last_unstale":"2024-04-13T05:16:41.586928+0000","last_undegraded":"2024-04-13T05:16:41.586928+0000","last_fullsized":"2024-04-13T05:16:41.586928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T15:30:18.331682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571520+0000","last_change":"2024-04-13T05:16:14.714804+0000","last_active":"2024-04-13T05:16:41.571520+0000","last_peered":"2024-04-13T05:16:41.571520+0000","last_clean":"2024-04-13T05:16:41.571520+0000","last_became_active":"2024-04-13T05:16:14.713542+0000","last_became_peered":"2024-04-13T05:16:14.713542+0000","last_unstale":"2024-04-13T05:16:41.571520+0000","last_undegraded":"2024-04-13T05:16:41.571520+0000","last_fullsized":"2024-04-13T05:16:41.571520+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T08:21:57.719452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600113+0000","last_change":"2024-04-13T05:14:53.592606+0000","last_active":"2024-04-13T05:16:41.600113+0000","last_peered":"2024-04-13T05:16:41.600113+0000","last_clean":"2024-04-13T05:16:41.600113+0000","last_became_active":"2024-04-13T05:14:53.592229+0000","last_became_peered":"2024-04-13T05:14:53.592229+0000","last_unstale":"2024-04-13T05:16:41.600113+0000","last_undegraded":"2024-04-13T05:16:41.600113+0000","last_fullsized":"2024-04-13T05:16:41.600113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:47:03.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.570115+0000","last_change":"2024-04-13T05:16:14.713320+0000","last_active":"2024-04-13T05:16:41.570115+0000","last_peered":"2024-04-13T05:16:41.570115+0000","last_clean":"2024-04-13T05:16:41.570115+0000","last_became_active":"2024-04-13T05:16:14.712507+0000","last_became_peered":"2024-04-13T05:16:14.712507+0000","last_unstale":"2024-04-13T05:16:41.570115+0000","last_undegraded":"2024-04-13T05:16:41.570115+0000","last_fullsized":"2024-04-13T05:16:41.570115+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:07.206767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.598327+0000","last_change":"2024-04-13T05:14:53.603234+0000","last_active":"2024-04-13T05:16:41.598327+0000","last_peered":"2024-04-13T05:16:41.598327+0000","last_clean":"2024-04-13T05:16:41.598327+0000","last_became_active":"2024-04-13T05:14:53.602896+0000","last_became_peered":"2024-04-13T05:14:53.602896+0000","last_unstale":"2024-04-13T05:16:41.598327+0000","last_undegraded":"2024-04-13T05:16:41.598327+0000","last_fullsized":"2024-04-13T05:16:41.598327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:35:29.047892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.596446+0000","last_change":"2024-04-13T05:14:53.605133+0000","last_active":"2024-04-13T05:16:41.596446+0000","last_peered":"2024-04-13T05:16:41.596446+0000","last_clean":"2024-04-13T05:16:41.596446+0000","last_became_active":"2024-04-13T05:14:53.604138+0000","last_became_peered":"2024-04-13T05:14:53.604138+0000","last_unstale":"2024-04-13T05:16:41.596446+0000","last_undegraded":"2024-04-13T05:16:41.596446+0000","last_fullsized":"2024-04-13T05:16:41.596446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:29:04.412667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.600578+0000","last_change":"2024-04-13T05:14:53.595227+0000","last_active":"2024-04-13T05:16:41.600578+0000","last_peered":"2024-04-13T05:16:41.600578+0000","last_clean":"2024-04-13T05:16:41.600578+0000","last_became_active":"2024-04-13T05:14:53.594003+0000","last_became_peered":"2024-04-13T05:14:53.594003+0000","last_unstale":"2024-04-13T05:16:41.600578+0000","last_undegraded":"2024-04-13T05:16:41.600578+0000","last_fullsized":"2024-04-13T05:16:41.600578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T14:57:16.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.593910+0000","last_change":"2024-04-13T05:14:53.597423+0000","last_active":"2024-04-13T05:16:41.593910+0000","last_peered":"2024-04-13T05:16:41.593910+0000","last_clean":"2024-04-13T05:16:41.593910+0000","last_became_active":"2024-04-13T05:14:53.596926+0000","last_became_peered":"2024-04-13T05:14:53.596926+0000","last_unstale":"2024-04-13T05:16:41.593910+0000","last_undegraded":"2024-04-13T05:16:41.593910+0000","last_fullsized":"2024-04-13T05:16:41.593910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:31:18.308301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-13T05:16:41.571621+0000","last_change":"2024-04-13T05:14:53.599862+0000","last_active":"2024-04-13T05:16:41.571621+0000","last_peered":"2024-04-13T05:16:41.571621+0000","last_clean":"2024-04-13T05:16:41.571621+0000","last_became_active":"2024-04-13T05:14:53.599058+0000","last_became_peered":"2024-04-13T05:14:53.599058+0000","last_unstale":"2024-04-13T05:16:41.571621+0000","last_undegraded":"2024-04-13T05:16:41.571621+0000","last_fullsized":"2024-04-13T05:16:41.571621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-13T05:14:52.559351+0000","last_clean_scrub_stamp":"2024-04-13T05:14:52.559351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T10:21:11.578057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72809,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":944,"num_read_kb":552,"num_write":180,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":526313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32900,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712252,"statfs":{"total":95995035648,"available":95961346048,"internally_reserved":0,"allocated":1765376,"data_stored":1182287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808898,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":530645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":525887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1123505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":521725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":589428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1089536,"data_stored":523113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1155072,"data_stored":591047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32824,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712328,"statfs":{"total":95995035648,"available":95961423872,"internally_reserved":0,"allocated":1687552,"data_stored":1118068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-13T05:17:57.089 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:17:57.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 -- 172.21.15.57:0/3608793377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 msgr2=0x7fc0d810c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 --2- 172.21.15.57:0/3608793377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d810c480 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009900 tx=0x7fc0d002bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:57.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 -- 172.21.15.57:0/3608793377 shutdown_connections 2024-04-13T05:17:57.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 --2- 172.21.15.57:0/3608793377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8105230 0x7fc0d81056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 --2- 172.21.15.57:0/3608793377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d810c480 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 --2- 172.21.15.57:0/3608793377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0d8104040 0x7fc0d8104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 -- 172.21.15.57:0/3608793377 >> 172.21.15.57:0/3608793377 conn(0x7fc0d80ff5d0 msgr2=0x7fc0d81019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:57.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 -- 172.21.15.57:0/3608793377 shutdown_connections 2024-04-13T05:17:57.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.355+0000 7fc0dee00640 1 -- 172.21.15.57:0/3608793377 wait complete. 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 Processor -- start 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 -- start start 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8104040 0x7fc0d819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0d8105230 0x7fc0d819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d810fbd0 con 0x7fc0d8105bf0 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d810fa50 con 0x7fc0d8104040 2024-04-13T05:17:57.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.356+0000 7fc0dee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d810fd50 con 0x7fc0d8105230 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d81a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33938/0 (socket says 172.21.15.57:33938) 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 -- 172.21.15.57:0/2783403276 learned_addr learned my addr 172.21.15.57:0/2783403276 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0dda00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8104040 0x7fc0d819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 -- 172.21.15.57:0/2783403276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0d8105230 msgr2=0x7fc0d819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0dd000640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0d8105230 0x7fc0d819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0d8105230 0x7fc0d819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 -- 172.21.15.57:0/2783403276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8104040 msgr2=0x7fc0d819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8104040 0x7fc0d819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0de400640 1 -- 172.21.15.57:0/2783403276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d81a1e60 con 0x7fc0d8105bf0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.357+0000 7fc0dda00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8104040 0x7fc0d819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.358+0000 7fc0de400640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d81a17f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0036040 tx=0x7fc0d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.358+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0043020 con 0x7fc0d8105bf0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.358+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0d0004490 con 0x7fc0d8105bf0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.358+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0003d10 con 0x7fc0d8105bf0 2024-04-13T05:17:57.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.358+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d81a20f0 con 0x7fc0d8105bf0 2024-04-13T05:17:57.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.358+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d81a25f0 con 0x7fc0d8105bf0 2024-04-13T05:17:57.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.359+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d810b360 con 0x7fc0d8105bf0 2024-04-13T05:17:57.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.361+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc0d0046020 con 0x7fc0d8105bf0 2024-04-13T05:17:57.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.362+0000 7fc0d6000640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0c00a3c30 0x7fc0c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.363+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc0d00f2c00 con 0x7fc0d8105bf0 2024-04-13T05:17:57.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.363+0000 7fc0dda00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0c00a3c30 0x7fc0c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.364+0000 7fc0dda00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0c00a3c30 0x7fc0c00a60f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8105090 tx=0x7fc0cc009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:57.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.367+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0d004d020 con 0x7fc0d8105bf0 2024-04-13T05:17:57.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:57 smithi189 ceph-mon[35531]: pgmap v466: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-13T05:17:57.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:57 smithi189 ceph-mon[35531]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:57 smithi083 ceph-mon[35133]: pgmap v466: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-13T05:17:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:57 smithi083 ceph-mon[35133]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:57.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:57 smithi057 ceph-mon[32331]: pgmap v466: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-13T05:17:57.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:57 smithi057 ceph-mon[32331]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:57.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.578+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0d8062c60 con 0x7fc0d8105bf0 2024-04-13T05:17:57.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.582+0000 7fc0d6000640 1 -- 172.21.15.57:0/2783403276 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7fc0d00bf1e0 con 0x7fc0d8105bf0 2024-04-13T05:17:57.585 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:57.585 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:17:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0c00a3c30 msgr2=0x7fc0c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0c00a3c30 0x7fc0c00a60f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8105090 tx=0x7fc0cc009400 comp rx=0 tx=0).stop 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 msgr2=0x7fc0d81a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d81a17f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0036040 tx=0x7fc0d0004270 comp rx=0 tx=0).stop 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 shutdown_connections 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0c00a3c30 0x7fc0c00a60f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0d8105bf0 0x7fc0d81a17f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0d8105230 0x7fc0d819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 --2- 172.21.15.57:0/2783403276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0d8104040 0x7fc0d819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.585+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 >> 172.21.15.57:0/2783403276 conn(0x7fc0d80ff5d0 msgr2=0x7fc0d8108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.586+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 shutdown_connections 2024-04-13T05:17:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.586+0000 7fc0dee00640 1 -- 172.21.15.57:0/2783403276 wait complete. 2024-04-13T05:17:57.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:17:57.608 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 -- 172.21.15.57:0/2317906883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810105410 msgr2=0x7f6810105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 --2- 172.21.15.57:0/2317906883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810105410 0x7f6810105890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009900 tx=0x7f67fc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 -- 172.21.15.57:0/2317906883 shutdown_connections 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 --2- 172.21.15.57:0/2317906883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6810105dd0 0x7f681010c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 --2- 172.21.15.57:0/2317906883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810105410 0x7f6810105890 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 --2- 172.21.15.57:0/2317906883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810104220 0x7f6810104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 -- 172.21.15.57:0/2317906883 >> 172.21.15.57:0/2317906883 conn(0x7f68100ff7b0 msgr2=0x7f6810101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.840+0000 7f6818600640 1 -- 172.21.15.57:0/2317906883 shutdown_connections 2024-04-13T05:17:57.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.841+0000 7f6818600640 1 -- 172.21.15.57:0/2317906883 wait complete. 2024-04-13T05:17:57.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 Processor -- start 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 -- start start 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 0x7f681019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810105410 0x7f681019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6810105dd0 0x7f68101a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f681010fd60 con 0x7f6810104220 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f681010fbe0 con 0x7f6810105410 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6818600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f681010fee0 con 0x7f6810105dd0 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6815c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 0x7f681019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6815c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 0x7f681019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33944/0 (socket says 172.21.15.57:33944) 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.842+0000 7f6815c00640 1 -- 172.21.15.57:0/1221543841 learned_addr learned my addr 172.21.15.57:0/1221543841 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815200640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810105410 0x7f681019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6816600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6810105dd0 0x7f68101a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815c00640 1 -- 172.21.15.57:0/1221543841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6810105dd0 msgr2=0x7f68101a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815c00640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6810105dd0 0x7f68101a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815c00640 1 -- 172.21.15.57:0/1221543841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810105410 msgr2=0x7f681019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815c00640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810105410 0x7f681019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815c00640 1 -- 172.21.15.57:0/1221543841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68101a2010 con 0x7f6810104220 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815200640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810105410 0x7f681019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6815c00640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 0x7f681019a3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f680000e9c0 tx=0x7f680000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680000cd80 con 0x7f6810104220 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68000035f0 con 0x7f6810104220 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800010690 con 0x7f6810104220 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.843+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68101a2300 con 0x7f6810104220 2024-04-13T05:17:57.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.844+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a2800 con 0x7f6810104220 2024-04-13T05:17:57.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.845+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681010b5c0 con 0x7f6810104220 2024-04-13T05:17:57.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.847+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f68000037a0 con 0x7f6810104220 2024-04-13T05:17:57.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.848+0000 7f680e000640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f67f40a3c30 0x7f67f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:57.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.849+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f68000d0c70 con 0x7f6810104220 2024-04-13T05:17:57.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.849+0000 7f6815200640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f67f40a3c30 0x7f67f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:57.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.850+0000 7f6815200640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f67f40a3c30 0x7f67f40a60f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0096c0 tx=0x7f67fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:57.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:57.853+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f680009d250 con 0x7f6810104220 2024-04-13T05:17:58.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.060+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6810062bf0 con 0x7f6810104220 2024-04-13T05:17:58.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.061+0000 7f680e000640 1 -- 172.21.15.57:0/1221543841 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f680009cb50 con 0x7f6810104220 2024-04-13T05:17:58.063 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f67f40a3c30 msgr2=0x7f67f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f67f40a3c30 0x7f67f40a60f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0096c0 tx=0x7f67fc000bc0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 msgr2=0x7f681019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 0x7f681019a3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f680000e9c0 tx=0x7f680000ee80 comp rx=0 tx=0).stop 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 shutdown_connections 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f67f40a3c30 0x7f67f40a60f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6810105410 0x7f681019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6810105dd0 0x7f68101a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 --2- 172.21.15.57:0/1221543841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6810104220 0x7f681019a3f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 >> 172.21.15.57:0/1221543841 conn(0x7f68100ff7b0 msgr2=0x7f6810108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:58.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 shutdown_connections 2024-04-13T05:17:58.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.064+0000 7f6818600640 1 -- 172.21.15.57:0/1221543841 wait complete. 2024-04-13T05:17:58.086 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:17:58.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 -- 172.21.15.57:0/2404246637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c101fc0 msgr2=0x7f8b0c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 --2- 172.21.15.57:0/2404246637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c101fc0 0x7f8b0c102440 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009900 tx=0x7f8af802bc40 comp rx=0 tx=0).stop 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 -- 172.21.15.57:0/2404246637 shutdown_connections 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 --2- 172.21.15.57:0/2404246637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b0c102980 0x7f8b0c109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 --2- 172.21.15.57:0/2404246637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c101fc0 0x7f8b0c102440 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 --2- 172.21.15.57:0/2404246637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 -- 172.21.15.57:0/2404246637 >> 172.21.15.57:0/2404246637 conn(0x7f8b0c0fc360 msgr2=0x7f8b0c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 -- 172.21.15.57:0/2404246637 shutdown_connections 2024-04-13T05:17:58.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.317+0000 7f8b12a00640 1 -- 172.21.15.57:0/2404246637 wait complete. 2024-04-13T05:17:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.318+0000 7f8b12a00640 1 Processor -- start 2024-04-13T05:17:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 -- start start 2024-04-13T05:17:58.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c104dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b0c101fc0 0x7f8b0c103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b0c102980 0x7f8b0c103950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c10c8e0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c10c760 con 0x7f8b0c101fc0 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b12a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c10ca60 con 0x7f8b0c102980 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b11600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c104dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b11600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c104dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33960/0 (socket says 172.21.15.57:33960) 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b11600640 1 -- 172.21.15.57:0/3215798446 learned_addr learned my addr 172.21.15.57:0/3215798446 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.319+0000 7f8b10c00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b0c101fc0 0x7f8b0c103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b12000640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b0c102980 0x7f8b0c103950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b11600640 1 -- 172.21.15.57:0/3215798446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b0c102980 msgr2=0x7f8b0c103950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b11600640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b0c102980 0x7f8b0c103950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b11600640 1 -- 172.21.15.57:0/3215798446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b0c101fc0 msgr2=0x7f8b0c103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b11600640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b0c101fc0 0x7f8b0c103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b11600640 1 -- 172.21.15.57:0/3215798446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c1041e0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b11600640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c104dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c970 tx=0x7f8afc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc007c10 con 0x7f8b0c100dd0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8afc007db0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c1a2130 con 0x7f8b0c100dd0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.320+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f870 con 0x7f8b0c100dd0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.321+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a26d0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.322+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0c1a29f0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.324+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8afc00fa10 con 0x7f8b0c100dd0 2024-04-13T05:17:58.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.326+0000 7f8b0a000640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8af00a3c30 0x7f8af00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.326+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8afc0d0bf0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.326+0000 7f8b10c00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8af00a3c30 0x7f8af00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.327+0000 7f8b10c00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8af00a3c30 0x7f8af00a60f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c104b40 tx=0x7f8af802c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.330+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8afc09d260 con 0x7f8b0c100dd0 2024-04-13T05:17:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:58 smithi083 ceph-mon[35133]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:58 smithi083 ceph-mon[35133]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2783403276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1221543841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:58 smithi057 ceph-mon[32331]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:58 smithi057 ceph-mon[32331]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2783403276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:58.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1221543841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:58.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.537+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b0c1a2ce0 con 0x7f8b0c100dd0 2024-04-13T05:17:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.538+0000 7f8b0a000640 1 -- 172.21.15.57:0/3215798446 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f8afc09cb60 con 0x7f8b0c100dd0 2024-04-13T05:17:58.540 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.540+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8af00a3c30 msgr2=0x7f8af00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.540+0000 7f8b12a00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8af00a3c30 0x7f8af00a60f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c104b40 tx=0x7f8af802c810 comp rx=0 tx=0).stop 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.540+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 msgr2=0x7f8b0c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.540+0000 7f8b12a00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c104dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c970 tx=0x7f8afc00ce30 comp rx=0 tx=0).stop 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 shutdown_connections 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8af00a3c30 0x7f8af00a60f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b0c101fc0 0x7f8b0c103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b0c102980 0x7f8b0c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 --2- 172.21.15.57:0/3215798446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100dd0 0x7f8b0c104dd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 >> 172.21.15.57:0/3215798446 conn(0x7f8b0c0fc360 msgr2=0x7f8b0c069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 shutdown_connections 2024-04-13T05:17:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.541+0000 7f8b12a00640 1 -- 172.21.15.57:0/3215798446 wait complete. 2024-04-13T05:17:58.563 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:17:58.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:58 smithi189 ceph-mon[35531]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:58 smithi189 ceph-mon[35531]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-13T05:17:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2783403276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:17:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1221543841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:17:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 -- 172.21.15.57:0/954452017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 msgr2=0x7f51c010c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 --2- 172.21.15.57:0/954452017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c010c660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009ca0 tx=0x7f51b802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 -- 172.21.15.57:0/954452017 shutdown_connections 2024-04-13T05:17:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 --2- 172.21.15.57:0/954452017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c010c660 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 --2- 172.21.15.57:0/954452017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 0x7f51c0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 --2- 172.21.15.57:0/954452017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 0x7f51c0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 -- 172.21.15.57:0/954452017 >> 172.21.15.57:0/954452017 conn(0x7f51c00ff7b0 msgr2=0x7f51c0101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 -- 172.21.15.57:0/954452017 shutdown_connections 2024-04-13T05:17:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.779+0000 7f51c8200640 1 -- 172.21.15.57:0/954452017 wait complete. 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 Processor -- start 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 -- start start 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 0x7f51c019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 0x7f51c019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c01a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f51c010fd60 con 0x7f51c0105dd0 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51c010fbe0 con 0x7f51c0104220 2024-04-13T05:17:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.780+0000 7f51c8200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51c010fee0 con 0x7f51c0105410 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c01a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c01a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33972/0 (socket says 172.21.15.57:33972) 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 -- 172.21.15.57:0/2173018095 learned_addr learned my addr 172.21.15.57:0/2173018095 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c5800640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 0x7f51c019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c4e00640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 0x7f51c019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 -- 172.21.15.57:0/2173018095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 msgr2=0x7f51c019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 0x7f51c019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 -- 172.21.15.57:0/2173018095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 msgr2=0x7f51c019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 0x7f51c019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 -- 172.21.15.57:0/2173018095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c01a2030 con 0x7f51c0105dd0 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c4e00640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 0x7f51c019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c5800640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 0x7f51c019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.781+0000 7f51c6200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c01a19c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f51b8032380 tx=0x7f51b8032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.782+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8041020 con 0x7f51c0105dd0 2024-04-13T05:17:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.782+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51b8003e20 con 0x7f51c0105dd0 2024-04-13T05:17:58.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.782+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8039040 con 0x7f51c0105dd0 2024-04-13T05:17:58.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.782+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c01a22c0 con 0x7f51c0105dd0 2024-04-13T05:17:58.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.782+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01a2760 con 0x7f51c0105dd0 2024-04-13T05:17:58.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.783+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c010b560 con 0x7f51c0105dd0 2024-04-13T05:17:58.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.785+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51b8044020 con 0x7f51c0105dd0 2024-04-13T05:17:58.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.788+0000 7f51be000640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51a80a3c30 0x7f51a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:58.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.788+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f51b80f29a0 con 0x7f51c0105dd0 2024-04-13T05:17:58.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.788+0000 7f51c5800640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51a80a3c30 0x7f51a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:58.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.789+0000 7f51c5800640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51a80a3c30 0x7f51a80a60f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f51c0105270 tx=0x7f51b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:58.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:58.792+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51b804b020 con 0x7f51c0105dd0 2024-04-13T05:17:59.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.002+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51c0062c60 con 0x7f51c0105dd0 2024-04-13T05:17:59.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.003+0000 7f51be000640 1 -- 172.21.15.57:0/2173018095 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f51b80bef80 con 0x7f51c0105dd0 2024-04-13T05:17:59.005 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51a80a3c30 msgr2=0x7f51a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51a80a3c30 0x7f51a80a60f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f51c0105270 tx=0x7f51b4005f90 comp rx=0 tx=0).stop 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 msgr2=0x7f51c01a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c01a19c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f51b8032380 tx=0x7f51b8032820 comp rx=0 tx=0).stop 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 shutdown_connections 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51a80a3c30 0x7f51a80a60f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51c0105dd0 0x7f51c01a19c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51c0105410 0x7f51c019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 --2- 172.21.15.57:0/2173018095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0104220 0x7f51c019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 >> 172.21.15.57:0/2173018095 conn(0x7f51c00ff7b0 msgr2=0x7f51c0108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:59.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 shutdown_connections 2024-04-13T05:17:59.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.006+0000 7f51c8200640 1 -- 172.21.15.57:0/2173018095 wait complete. 2024-04-13T05:17:59.027 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:17:59.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 -- 172.21.15.57:0/1695565537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c105dd0 msgr2=0x7fd48c10c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 --2- 172.21.15.57:0/1695565537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c105dd0 0x7fd48c10c660 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd484009ca0 tx=0x7fd48402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 -- 172.21.15.57:0/1695565537 shutdown_connections 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 --2- 172.21.15.57:0/1695565537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48c105410 0x7fd48c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 --2- 172.21.15.57:0/1695565537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c105dd0 0x7fd48c10c660 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 --2- 172.21.15.57:0/1695565537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd48c104220 0x7fd48c104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 -- 172.21.15.57:0/1695565537 >> 172.21.15.57:0/1695565537 conn(0x7fd48c0ff7b0 msgr2=0x7fd48c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 -- 172.21.15.57:0/1695565537 shutdown_connections 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.255+0000 7fd493400640 1 -- 172.21.15.57:0/1695565537 wait complete. 2024-04-13T05:17:59.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.256+0000 7fd493400640 1 Processor -- start 2024-04-13T05:17:59.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.256+0000 7fd493400640 1 -- start start 2024-04-13T05:17:59.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.256+0000 7fd493400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 0x7fd48c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.256+0000 7fd493400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48c105410 0x7fd48c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.256+0000 7fd493400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd48c105dd0 0x7fd48c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd493400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10fd60 con 0x7fd48c104220 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd493400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10fbe0 con 0x7fd48c105410 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd493400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10fee0 con 0x7fd48c105dd0 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 0x7fd48c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 0x7fd48c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34002/0 (socket says 172.21.15.57:34002) 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 -- 172.21.15.57:0/2382961506 learned_addr learned my addr 172.21.15.57:0/2382961506 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd491400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd48c105dd0 0x7fd48c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 -- 172.21.15.57:0/2382961506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd48c105dd0 msgr2=0x7fd48c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd48c105dd0 0x7fd48c1a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd48be00640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48c105410 0x7fd48c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 -- 172.21.15.57:0/2382961506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48c105410 msgr2=0x7fd48c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48c105410 0x7fd48c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.257+0000 7fd490a00640 1 -- 172.21.15.57:0/2382961506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a2010 con 0x7fd48c104220 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.258+0000 7fd490a00640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 0x7fd48c19a3f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd48000d950 tx=0x7fd48000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.258+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480017070 con 0x7fd48c104220 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.258+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd480009c30 con 0x7fd48c104220 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.258+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1a2300 con 0x7fd48c104220 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.258+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48000f620 con 0x7fd48c104220 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.258+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a2800 con 0x7fd48c104220 2024-04-13T05:17:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.259+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c10b540 con 0x7fd48c104220 2024-04-13T05:17:59.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.261+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd48001a070 con 0x7fd48c104220 2024-04-13T05:17:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.263+0000 7fd489600640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4740a3bb0 0x7fd4740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.263+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd4800d0320 con 0x7fd48c104220 2024-04-13T05:17:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.263+0000 7fd48be00640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4740a3bb0 0x7fd4740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.264+0000 7fd48be00640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4740a3bb0 0x7fd4740a6070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd478005fd0 tx=0x7fd478005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.267+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd480009de0 con 0x7fd48c104220 2024-04-13T05:17:59.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:59 smithi057 ceph-mon[32331]: pgmap v467: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-13T05:17:59.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3215798446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:59.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:17:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2173018095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:59.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.477+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd48c062bc0 con 0x7fd48c104220 2024-04-13T05:17:59.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.479+0000 7fd489600640 1 -- 172.21.15.57:0/2382961506 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fd48009c900 con 0x7fd48c104220 2024-04-13T05:17:59.481 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:17:59.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4740a3bb0 msgr2=0x7fd4740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4740a3bb0 0x7fd4740a6070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd478005fd0 tx=0x7fd478005850 comp rx=0 tx=0).stop 2024-04-13T05:17:59.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 msgr2=0x7fd48c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 0x7fd48c19a3f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd48000d950 tx=0x7fd48000de10 comp rx=0 tx=0).stop 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 shutdown_connections 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4740a3bb0 0x7fd4740a6070 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd48c105410 0x7fd48c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd48c105dd0 0x7fd48c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 --2- 172.21.15.57:0/2382961506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd48c104220 0x7fd48c19a3f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 >> 172.21.15.57:0/2382961506 conn(0x7fd48c0ff7b0 msgr2=0x7fd48c108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 shutdown_connections 2024-04-13T05:17:59.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.481+0000 7fd493400640 1 -- 172.21.15.57:0/2382961506 wait complete. 2024-04-13T05:17:59.502 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:17:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:59 smithi083 ceph-mon[35133]: pgmap v467: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-13T05:17:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3215798446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:17:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2173018095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:59.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:59 smithi189 ceph-mon[35531]: pgmap v467: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-13T05:17:59.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3215798446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:17:59.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:17:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2173018095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:17:59.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 -- 172.21.15.57:0/2248099276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598103940 msgr2=0x7f059810a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 --2- 172.21.15.57:0/2248099276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598103940 0x7f059810a260 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0590009ca0 tx=0x7f059002c100 comp rx=0 tx=0).stop 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 -- 172.21.15.57:0/2248099276 shutdown_connections 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 --2- 172.21.15.57:0/2248099276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598102f80 0x7f0598103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 --2- 172.21.15.57:0/2248099276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598103940 0x7f059810a260 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 --2- 172.21.15.57:0/2248099276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0598101e30 0x7f0598102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 -- 172.21.15.57:0/2248099276 >> 172.21.15.57:0/2248099276 conn(0x7f05980fd3a0 msgr2=0x7f05980ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 -- 172.21.15.57:0/2248099276 shutdown_connections 2024-04-13T05:17:59.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.717+0000 7f0597e00640 1 -- 172.21.15.57:0/2248099276 wait complete. 2024-04-13T05:17:59.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.718+0000 7f0597e00640 1 Processor -- start 2024-04-13T05:17:59.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 -- start start 2024-04-13T05:17:59.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 0x7f059819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0598102f80 0x7f059819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598103940 0x7f05981a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d960 con 0x7f0598101e30 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d7e0 con 0x7f0598103940 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0597e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f059810dae0 con 0x7f0598102f80 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0596a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 0x7f059819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0596a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 0x7f059819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34022/0 (socket says 172.21.15.57:34022) 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.719+0000 7f0596a00640 1 -- 172.21.15.57:0/552311940 learned_addr learned my addr 172.21.15.57:0/552311940 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596000640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0598102f80 0x7f059819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0597400640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598103940 0x7f05981a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596a00640 1 -- 172.21.15.57:0/552311940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0598102f80 msgr2=0x7f059819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596a00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0598102f80 0x7f059819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596a00640 1 -- 172.21.15.57:0/552311940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598103940 msgr2=0x7f05981a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596a00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598103940 0x7f05981a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596a00640 1 -- 172.21.15.57:0/552311940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981a1e40 con 0x7f0598101e30 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0597400640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598103940 0x7f05981a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0596a00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 0x7f059819a220 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f058800d950 tx=0x7f058800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:59.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588017070 con 0x7f0598101e30 2024-04-13T05:17:59.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0588009c30 con 0x7f0598101e30 2024-04-13T05:17:59.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.720+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981a2130 con 0x7f0598101e30 2024-04-13T05:17:59.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.721+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058800f620 con 0x7f0598101e30 2024-04-13T05:17:59.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.721+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a2680 con 0x7f0598101e30 2024-04-13T05:17:59.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.722+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05981a28e0 con 0x7f0598101e30 2024-04-13T05:17:59.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.724+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f05880109a0 con 0x7f0598101e30 2024-04-13T05:17:59.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.725+0000 7f058f400640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f057c0a3bb0 0x7f057c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:17:59.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.725+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f05880d04e0 con 0x7f0598101e30 2024-04-13T05:17:59.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.725+0000 7f0596000640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f057c0a3bb0 0x7f057c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:17:59.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.726+0000 7f0596000640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f057c0a3bb0 0x7f057c0a6070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0584005fd0 tx=0x7f0584005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:17:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.730+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0588009de0 con 0x7f0598101e30 2024-04-13T05:17:59.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.982+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05981a2ab0 con 0x7f0598101e30 2024-04-13T05:17:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.984+0000 7f058f400640 1 -- 172.21.15.57:0/552311940 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7f058809cac0 con 0x7f0598101e30 2024-04-13T05:17:59.987 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:17:59.987 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:17:49.577177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:17:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f057c0a3bb0 msgr2=0x7f057c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f057c0a3bb0 0x7f057c0a6070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0584005fd0 tx=0x7f0584005850 comp rx=0 tx=0).stop 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 msgr2=0x7f059819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 0x7f059819a220 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f058800d950 tx=0x7f058800de10 comp rx=0 tx=0).stop 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 shutdown_connections 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f057c0a3bb0 0x7f057c0a6070 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0598103940 0x7f05981a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0598102f80 0x7f059819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 --2- 172.21.15.57:0/552311940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0598101e30 0x7f059819a220 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:17:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 >> 172.21.15.57:0/552311940 conn(0x7f05980fd3a0 msgr2=0x7f05980741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:17:59.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 shutdown_connections 2024-04-13T05:17:59.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:17:59.987+0000 7f0597e00640 1 -- 172.21.15.57:0/552311940 wait complete. 2024-04-13T05:18:00.010 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:18:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 -- 172.21.15.57:0/2777653278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310104040 msgr2=0x7f8310104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 --2- 172.21.15.57:0/2777653278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310104040 0x7f8310104460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8300009900 tx=0x7f830002bc50 comp rx=0 tx=0).stop 2024-04-13T05:18:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 -- 172.21.15.57:0/2777653278 shutdown_connections 2024-04-13T05:18:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 --2- 172.21.15.57:0/2777653278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8310105bf0 0x7f831010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 --2- 172.21.15.57:0/2777653278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 0x7f83101056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 --2- 172.21.15.57:0/2777653278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310104040 0x7f8310104460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 -- 172.21.15.57:0/2777653278 >> 172.21.15.57:0/2777653278 conn(0x7f83100ff5d0 msgr2=0x7f83101019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 -- 172.21.15.57:0/2777653278 shutdown_connections 2024-04-13T05:18:00.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.236+0000 7f830f200640 1 -- 172.21.15.57:0/2777653278 wait complete. 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.237+0000 7f830f200640 1 Processor -- start 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.237+0000 7f830f200640 1 -- start start 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8310104040 0x7f831019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 0x7f831019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 0x7f83101a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f831010fbd0 con 0x7f8310105bf0 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f831010fa50 con 0x7f8310104040 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f831010fd50 con 0x7f8310105230 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 0x7f83101a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 0x7f83101a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34050/0 (socket says 172.21.15.57:34050) 2024-04-13T05:18:00.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830e800640 1 -- 172.21.15.57:0/1578830773 learned_addr learned my addr 172.21.15.57:0/1578830773 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830d400640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 0x7f831019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.238+0000 7f830de00640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8310104040 0x7f831019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830e800640 1 -- 172.21.15.57:0/1578830773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 msgr2=0x7f831019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830e800640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 0x7f831019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830e800640 1 -- 172.21.15.57:0/1578830773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8310104040 msgr2=0x7f831019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830e800640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8310104040 0x7f831019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830e800640 1 -- 172.21.15.57:0/1578830773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83101a1e20 con 0x7f8310105bf0 2024-04-13T05:18:00.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830d400640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 0x7f831019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830e800640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 0x7f83101a17b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f830800cad0 tx=0x7f830800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830800ae20 con 0x7f8310105bf0 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8308005020 con 0x7f8310105bf0 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83101a2110 con 0x7f8310105bf0 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.239+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830800f6b0 con 0x7f8310105bf0 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.240+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a2610 con 0x7f8310105bf0 2024-04-13T05:18:00.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.241+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831010b3e0 con 0x7f8310105bf0 2024-04-13T05:18:00.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.243+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f83080051d0 con 0x7f8310105bf0 2024-04-13T05:18:00.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.245+0000 7f8306a00640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f82f40a3bb0 0x7f82f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.245+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f83080d0930 con 0x7f8310105bf0 2024-04-13T05:18:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.245+0000 7f830de00640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f82f40a3bb0 0x7f82f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.246+0000 7f830de00640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f82f40a3bb0 0x7f82f40a6070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f830002c260 tx=0x7f8300000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.253+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8308014030 con 0x7f8310105bf0 2024-04-13T05:18:00.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2382961506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:18:00.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/552311940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:18:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.465+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8310062bc0 con 0x7f8310105bf0 2024-04-13T05:18:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.468+0000 7f8306a00640 1 -- 172.21.15.57:0/1578830773 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f830809cf10 con 0x7f8310105bf0 2024-04-13T05:18:00.471 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:18:00.471 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:18:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.471+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f82f40a3bb0 msgr2=0x7f82f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.471+0000 7f830f200640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f82f40a3bb0 0x7f82f40a6070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f830002c260 tx=0x7f8300000bc0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.471+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 msgr2=0x7f83101a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.471+0000 7f830f200640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 0x7f83101a17b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f830800cad0 tx=0x7f830800cf90 comp rx=0 tx=0).stop 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 shutdown_connections 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f82f40a3bb0 0x7f82f40a6070 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8310105bf0 0x7f83101a17b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8310105230 0x7f831019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 --2- 172.21.15.57:0/1578830773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8310104040 0x7f831019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 >> 172.21.15.57:0/1578830773 conn(0x7f83100ff5d0 msgr2=0x7f83101085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 shutdown_connections 2024-04-13T05:18:00.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.472+0000 7f830f200640 1 -- 172.21.15.57:0/1578830773 wait complete. 2024-04-13T05:18:00.497 DEBUG:tasks.fwd_scrub:Yielding 2024-04-13T05:18:00.497 INFO:teuthology.run_tasks:Running task workunit... 2024-04-13T05:18:00.513 INFO:tasks.workunit:Pulling workunits from ref 218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:18:00.514 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-13T05:18:00.514 INFO:tasks.workunit:timeout=3h 2024-04-13T05:18:00.514 INFO:tasks.workunit:cleanup=True 2024-04-13T05:18:00.514 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-13T05:18:00.514 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-13T05:18:00.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:18:00.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:18:00.515 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:18:00.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2382961506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:18:00.516 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/552311940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-13 05:15:23.390874339 +0000 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-13 05:15:23.390874339 +0000 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-13 05:17:49.079498110 +0000 2024-04-13T05:18:00.571 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2024-04-13T05:18:00.572 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-13T05:18:00.572 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-13T05:18:00.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2382961506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:18:00.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/552311940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:18:00.719 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 218a691417ef57c5d22a8d5cf3c5f41f7b542838 2024-04-13T05:18:00.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 -- 172.21.15.57:0/3555493814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105dd0 msgr2=0x7f483810c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 --2- 172.21.15.57:0/3555493814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105dd0 0x7f483810c660 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4830009ca0 tx=0x7f483002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 -- 172.21.15.57:0/3555493814 shutdown_connections 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 --2- 172.21.15.57:0/3555493814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4838105410 0x7f4838105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 --2- 172.21.15.57:0/3555493814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105dd0 0x7f483810c660 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 --2- 172.21.15.57:0/3555493814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4838104220 0x7f4838104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 -- 172.21.15.57:0/3555493814 >> 172.21.15.57:0/3555493814 conn(0x7f48380ff770 msgr2=0x7f4838101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 -- 172.21.15.57:0/3555493814 shutdown_connections 2024-04-13T05:18:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.739+0000 7f483f600640 1 -- 172.21.15.57:0/3555493814 wait complete. 2024-04-13T05:18:00.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.740+0000 7f483f600640 1 Processor -- start 2024-04-13T05:18:00.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.740+0000 7f483f600640 1 -- start start 2024-04-13T05:18:00.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483f600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4838104220 0x7f483819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483f600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 0x7f483819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483f600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4838105dd0 0x7f48381a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483f600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f483810fd60 con 0x7f4838105410 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483f600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f483810fbe0 con 0x7f4838104220 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483f600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f483810fee0 con 0x7f4838105dd0 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 0x7f483819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 0x7f483819a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34068/0 (socket says 172.21.15.57:34068) 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 -- 172.21.15.57:0/2277186123 learned_addr learned my addr 172.21.15.57:0/2277186123 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483d600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4838105dd0 0x7f48381a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f483cc00640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4838104220 0x7f483819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 -- 172.21.15.57:0/2277186123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4838105dd0 msgr2=0x7f48381a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4838105dd0 0x7f48381a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 -- 172.21.15.57:0/2277186123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4838104220 msgr2=0x7f483819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4838104220 0x7f483819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.741+0000 7f4837e00640 1 -- 172.21.15.57:0/2277186123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48381a2000 con 0x7f4838105410 2024-04-13T05:18:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.742+0000 7f4837e00640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 0x7f483819a920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f482400d950 tx=0x7f482400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.742+0000 7f4835600640 1 -- 172.21.15.57:0/2277186123 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824017070 con 0x7f4838105410 2024-04-13T05:18:00.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.742+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381a22f0 con 0x7f4838105410 2024-04-13T05:18:00.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.742+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381a2890 con 0x7f4838105410 2024-04-13T05:18:00.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.742+0000 7f4835600640 1 -- 172.21.15.57:0/2277186123 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4824009c30 con 0x7f4838105410 2024-04-13T05:18:00.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.742+0000 7f4835600640 1 -- 172.21.15.57:0/2277186123 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824020060 con 0x7f4838105410 2024-04-13T05:18:00.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 Processor -- start 2024-04-13T05:18:00.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 -- start start 2024-04-13T05:18:00.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4835600640 1 -- 172.21.15.57:0/2277186123 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4824010460 con 0x7f4838105410 2024-04-13T05:18:00.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1eeb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 0x55a0f1ef3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1e51bc0 con 0x55a0f1ef0de0 2024-04-13T05:18:00.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1df8660 con 0x55a0f1eebc70 2024-04-13T05:18:00.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.746+0000 7f4842122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1e4db80 con 0x55a0f1eeb280 2024-04-13T05:18:00.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.747+0000 7f4837e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 0x55a0f1ef3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.747+0000 7f483d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1eeb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.747+0000 7f4837e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 0x55a0f1ef3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34076/0 (socket says 172.21.15.57:34076) 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.747+0000 7f4837e00640 1 -- 172.21.15.57:0/190454065 learned_addr learned my addr 172.21.15.57:0/190454065 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.747+0000 7f483cc00640 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4837e00640 1 -- 172.21.15.57:0/190454065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 msgr2=0x55a0f1eeb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4837e00640 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1eeb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4837e00640 1 -- 172.21.15.57:0/190454065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1eebc70 msgr2=0x55a0f1ef0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4837e00640 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4837e00640 1 -- 172.21.15.57:0/190454065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f1e1b220 con 0x55a0f1ef0de0 2024-04-13T05:18:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4835600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48200a3bb0 0x7f48200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4837e00640 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 0x55a0f1ef3270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f48240d8290 tx=0x7f48240d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32995072147c4e27 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f4835600640 1 -- 172.21.15.57:0/2277186123 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f48240d0100 con 0x7f4838105410 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.748+0000 7f483d600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48200a3bb0 0x7f48200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.749+0000 7f4829c00640 1 -- 172.21.15.57:0/190454065 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824098280 con 0x55a0f1ef0de0 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.749+0000 7f4829c00640 1 -- 172.21.15.57:0/190454065 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f482409c6e0 con 0x55a0f1ef0de0 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.749+0000 7f4829c00640 1 -- 172.21.15.57:0/190454065 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482409b940 con 0x55a0f1ef0de0 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 -- 172.21.15.57:0/190454065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 msgr2=0x55a0f1ef3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 0x55a0f1ef3270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f48240d8290 tx=0x7f48240d8750 comp rx=0 tx=0).stop 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f483d600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48200a3bb0 0x7f48200a6070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4830006010 tx=0x7f4830005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 -- 172.21.15.57:0/190454065 shutdown_connections 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1ef0de0 0x55a0f1ef3270 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 --2- 172.21.15.57:0/190454065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1eeb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 -- 172.21.15.57:0/190454065 >> 172.21.15.57:0/190454065 conn(0x55a0f1ee67f0 msgr2=0x55a0f1ee6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 -- 172.21.15.57:0/190454065 shutdown_connections 2024-04-13T05:18:00.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 -- 172.21.15.57:0/190454065 wait complete. 2024-04-13T05:18:00.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.750+0000 7f4842122740 1 -- start start 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 Processor -- start 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 0x55a0f1ef5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ef0de0 0x55a0f1efccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1ef3d30 con 0x55a0f1eeb280 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1e0ba30 con 0x55a0f1ef0de0 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1ef3eb0 con 0x55a0f1eebc70 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f4842122740 10 client.0 _ll_register_callbacks cb 0x55a0f1b63c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f483cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 0x55a0f1ef5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f483cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 0x55a0f1ef5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34084/0 (socket says 172.21.15.57:34084) 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.751+0000 7f483cc00640 1 -- 172.21.15.57:0/2144119049 learned_addr learned my addr 172.21.15.57:0/2144119049 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:00.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483d600640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ef0de0 0x55a0f1efccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f4837e00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483cc00640 1 -- 172.21.15.57:0/2144119049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eebc70 msgr2=0x55a0f1ef5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483cc00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483cc00640 1 -- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ef0de0 msgr2=0x55a0f1efccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483cc00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ef0de0 0x55a0f1efccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483cc00640 1 -- 172.21.15.57:0/2144119049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f1c2c7e0 con 0x55a0f1eeb280 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f4837e00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f483cc00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 0x55a0f1ef5770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f482c009d40 tx=0x7f482c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36717cd1cfae45d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.752+0000 7f481e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f481e000640 1 -- 172.21.15.57:0/2144119049 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c012860 con 0x55a0f1eeb280 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f481e000640 1 -- 172.21.15.57:0/2144119049 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f482c012a00 con 0x55a0f1eeb280 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482409c6e0 con 0x55a0f1eeb280 2024-04-13T05:18:00.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f481e000640 1 -- 172.21.15.57:0/2144119049 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f482c025050 con 0x55a0f1eeb280 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f4842122740 10 client.15591 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0f1c8cf70 con 0x55a0f1eeb280 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.753+0000 7f481e000640 1 -- 172.21.15.57:0/2144119049 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f482409c6e0 con 0x55a0f1eeb280 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f481e000640 1 -- 172.21.15.57:0/2144119049 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f482c00f040 con 0x55a0f1eeb280 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4842122740 10 client.15591 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4842122740 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4842122740 20 client.15591 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4842122740 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1f01720 0x55a0f1f03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4842122740 4 client.15591 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:00.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0f1e5f5e0 con 0x55a0f1f01720 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.754+0000 7f4837e00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1f01720 0x55a0f1f03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.755+0000 7f4837e00640 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1f01720 0x55a0f1f03bc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e730d52cd150c88 server_cookie=f82222d031b2926e in_seq=0 out_seq=0 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.755+0000 7f481e000640 10 client.15591 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f481e000640 1 -- 172.21.15.57:0/2144119049 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f482407ae50 con 0x55a0f1f01720 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f481e000640 10 client.15591 handle_command_reply: tid=0 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 client.15591 shutdown 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 msgr2=0x55a0f1ef5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 0x55a0f1ef5770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f482c009d40 tx=0x7f482c0117b0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 shutdown_connections 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1f01720 0x55a0f1f03bc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ef0de0 0x55a0f1efccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 --2- 172.21.15.57:0/2144119049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eeb280 0x55a0f1ef5770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f481e000640 0 client.15591 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:00.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 >> 172.21.15.57:0/2144119049 conn(0x55a0f1ee67f0 msgr2=0x55a0f1ee59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 shutdown_connections 2024-04-13T05:18:00.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.757+0000 7f4842122740 1 -- 172.21.15.57:0/2144119049 wait complete. 2024-04-13T05:18:00.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.758+0000 7f4842122740 20 client.15591 trim_cache size 0 max 16384 2024-04-13T05:18:00.768 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-13T05:18:00.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.777+0000 7f4842122740 1 Processor -- start 2024-04-13T05:18:00.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 -- start start 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1ef4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4818012490 con 0x55a0f1eebc70 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f481800fd60 con 0x55a0f1ee67f0 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f4842122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f481800dbb0 con 0x55a0f1eeb280 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f483d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f483d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34092/0 (socket says 172.21.15.57:34092) 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.778+0000 7f483d600640 1 -- 172.21.15.57:0/3339156651 learned_addr learned my addr 172.21.15.57:0/3339156651 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:00.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f4837e00640 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483cc00640 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1ef4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483d600640 1 -- 172.21.15.57:0/3339156651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 msgr2=0x55a0f1efcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483d600640 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483d600640 1 -- 172.21.15.57:0/3339156651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 msgr2=0x55a0f1ef4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483d600640 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1ef4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483d600640 1 -- 172.21.15.57:0/3339156651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f1eed810 con 0x55a0f1eebc70 2024-04-13T05:18:00.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.779+0000 7f483cc00640 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1ef4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:00.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.780+0000 7f483d600640 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef52d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4830032420 tx=0x7f4830040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f62e160ebeb93f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.780+0000 7f4829200640 1 -- 172.21.15.57:0/3339156651 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830039040 con 0x55a0f1eebc70 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.780+0000 7f4829200640 1 -- 172.21.15.57:0/3339156651 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4830052070 con 0x55a0f1eebc70 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.780+0000 7f4829200640 1 -- 172.21.15.57:0/3339156651 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300406a0 con 0x55a0f1eebc70 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 -- 172.21.15.57:0/3339156651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 msgr2=0x55a0f1ef52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef52d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4830032420 tx=0x7f4830040000 comp rx=0 tx=0).stop 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 -- 172.21.15.57:0/3339156651 shutdown_connections 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1ef52d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 --2- 172.21.15.57:0/3339156651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1ef4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 -- 172.21.15.57:0/3339156651 >> 172.21.15.57:0/3339156651 conn(0x55a0f1ee1870 msgr2=0x55a0f1eea2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 -- 172.21.15.57:0/3339156651 shutdown_connections 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.781+0000 7f4842122740 1 -- 172.21.15.57:0/3339156651 wait complete. 2024-04-13T05:18:00.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 -- start start 2024-04-13T05:18:00.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 Processor -- start 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1efc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1efb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f481800e220 con 0x55a0f1eebc70 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f481800f690 con 0x55a0f1ee67f0 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f481800e8f0 con 0x55a0f1eeb280 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.782+0000 7f4842122740 10 client.0 _ll_register_callbacks cb 0x7f482c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1efb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1efb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34108/0 (socket says 172.21.15.57:34108) 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 -- 172.21.15.57:0/3923204758 learned_addr learned my addr 172.21.15.57:0/3923204758 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483cc00640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1efc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f4837e00640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 -- 172.21.15.57:0/3923204758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 msgr2=0x55a0f1efad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 -- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 msgr2=0x55a0f1efc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1efc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 -- 172.21.15.57:0/3923204758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f1ef4850 con 0x55a0f1eebc70 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f483d600640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1efb290 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4830005cc0 tx=0x7f4830003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f84007de86a6977 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.783+0000 7f481e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.784+0000 7f481e000640 1 -- 172.21.15.57:0/3923204758 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830004450 con 0x55a0f1eebc70 2024-04-13T05:18:00.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.784+0000 7f481e000640 1 -- 172.21.15.57:0/3923204758 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f48300045f0 con 0x55a0f1eebc70 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.784+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4830052070 con 0x55a0f1eebc70 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.784+0000 7f481e000640 1 -- 172.21.15.57:0/3923204758 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4830003e80 con 0x55a0f1eebc70 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.784+0000 7f4842122740 10 client.15597 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.784+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0f1b24170 con 0x55a0f1eebc70 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f481e000640 1 -- 172.21.15.57:0/3923204758 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4830052070 con 0x55a0f1eebc70 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f481e000640 1 -- 172.21.15.57:0/3923204758 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4830060590 con 0x55a0f1eebc70 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f4842122740 10 client.15597 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f4842122740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f4842122740 20 client.15597 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f4842122740 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1ef0de0 0x55a0f1f6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f4842122740 4 client.15597 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:18:00.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.785+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0f1e59450 con 0x55a0f1ef0de0 2024-04-13T05:18:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.786+0000 7f483cc00640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1ef0de0 0x55a0f1f6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.787+0000 7f483cc00640 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1ef0de0 0x55a0f1f6b0f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a08d03da91c58b1 server_cookie=af652df779c7b125 in_seq=0 out_seq=0 2024-04-13T05:18:00.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.787+0000 7f481e000640 10 client.15597 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.789+0000 7f481e000640 1 -- 172.21.15.57:0/3923204758 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0f1e59450 con 0x55a0f1ef0de0 2024-04-13T05:18:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.789+0000 7f481e000640 10 client.15597 handle_command_reply: tid=0 2024-04-13T05:18:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.789+0000 7f4842122740 1 client.15597 shutdown 2024-04-13T05:18:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 msgr2=0x55a0f1efb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1efb290 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4830005cc0 tx=0x7f4830003c90 comp rx=0 tx=0).stop 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 shutdown_connections 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0f1ef0de0 0x55a0f1f6b0f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0f1eebc70 0x55a0f1efb290 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f1eeb280 0x55a0f1efad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 --2- 172.21.15.57:0/3923204758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0f1ee67f0 0x55a0f1efc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 >> 172.21.15.57:0/3923204758 conn(0x55a0f1ee1870 msgr2=0x55a0f1ee5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f481e000640 0 client.15597 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 shutdown_connections 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 1 -- 172.21.15.57:0/3923204758 wait complete. 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.790+0000 7f4842122740 20 client.15597 trim_cache size 0 max 16384 2024-04-13T05:18:00.793 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:18:00.794 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:18:00.794 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "887acbe1-f535-492a-a287-7dc3677ff570", 2024-04-13T05:18:00.794 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:18:00.794 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:18:00.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48200a3bb0 msgr2=0x7f48200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48200a3bb0 0x7f48200a6070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4830006010 tx=0x7f4830005c30 comp rx=0 tx=0).stop 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 msgr2=0x7f483819a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 0x7f483819a920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f482400d950 tx=0x7f482400de10 comp rx=0 tx=0).stop 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 shutdown_connections 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f48200a3bb0 0x7f48200a6070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4838105dd0 0x7f48381a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4838105410 0x7f483819a920 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 --2- 172.21.15.57:0/2277186123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4838104220 0x7f483819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 >> 172.21.15.57:0/2277186123 conn(0x7f48380ff770 msgr2=0x7f4838108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 shutdown_connections 2024-04-13T05:18:00.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:00.791+0000 7f483f600640 1 -- 172.21.15.57:0/2277186123 wait complete. 2024-04-13T05:18:00.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:18:00.810 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.109+0000 7f93dca00640 1 -- 172.21.15.57:0/3491504791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 msgr2=0x7f93d8107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.109+0000 7f93dca00640 1 --2- 172.21.15.57:0/3491504791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d8107670 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009900 tx=0x7f93c402bc30 comp rx=0 tx=0).stop 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 -- 172.21.15.57:0/3491504791 shutdown_connections 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 --2- 172.21.15.57:0/3491504791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 0x7f93d810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 --2- 172.21.15.57:0/3491504791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d8107670 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 --2- 172.21.15.57:0/3491504791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 0x7f93d8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 -- 172.21.15.57:0/3491504791 >> 172.21.15.57:0/3491504791 conn(0x7f93d80fc360 msgr2=0x7f93d80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:01.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 -- 172.21.15.57:0/3491504791 shutdown_connections 2024-04-13T05:18:01.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.110+0000 7f93dca00640 1 -- 172.21.15.57:0/3491504791 wait complete. 2024-04-13T05:18:01.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.111+0000 7f93dca00640 1 Processor -- start 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.111+0000 7f93dca00640 1 -- start start 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.111+0000 7f93dca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 0x7f93d819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.111+0000 7f93dca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93dca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 0x7f93d81a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93dca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f93d810da40 con 0x7f93d8105280 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93dca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93d810d8c0 con 0x7f93d8102950 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93dca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93d810dbc0 con 0x7f93d8107bb0 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34134/0 (socket says 172.21.15.57:34134) 2024-04-13T05:18:01.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 -- 172.21.15.57:0/557958975 learned_addr learned my addr 172.21.15.57:0/557958975 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:01.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d6800640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 0x7f93d819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d7200640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 0x7f93d81a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 -- 172.21.15.57:0/557958975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 msgr2=0x7f93d81a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 0x7f93d81a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 -- 172.21.15.57:0/557958975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 msgr2=0x7f93d819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.112+0000 7f93d5e00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 0x7f93d819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.113+0000 7f93d5e00640 1 -- 172.21.15.57:0/557958975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d81a1f50 con 0x7f93d8105280 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.113+0000 7f93d6800640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 0x7f93d819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.113+0000 7f93d7200640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 0x7f93d81a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.113+0000 7f93d5e00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d819a870 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f93c4000c00 tx=0x7f93c4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.113+0000 7f93cf400640 1 -- 172.21.15.57:0/557958975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4043020 con 0x7f93d8105280 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.114+0000 7f93cf400640 1 -- 172.21.15.57:0/557958975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93c40043d0 con 0x7f93d8105280 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.114+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d81a21e0 con 0x7f93d8105280 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.114+0000 7f93cf400640 1 -- 172.21.15.57:0/557958975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4003bb0 con 0x7f93d8105280 2024-04-13T05:18:01.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.114+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d81a2780 con 0x7f93d8105280 2024-04-13T05:18:01.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.117+0000 7f93cf400640 1 -- 172.21.15.57:0/557958975 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93c4046020 con 0x7f93d8105280 2024-04-13T05:18:01.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.117+0000 7f93e0419740 1 Processor -- start 2024-04-13T05:18:01.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 -- start start 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b090fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 0x5595b0914be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b0915250 0x5595b09176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5595b0876020 con 0x5595b09100e0 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595b081c6d0 con 0x5595b090f6f0 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93e0419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5595b0871fe0 con 0x5595b0915250 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 0x5595b0914be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 0x5595b0914be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34150/0 (socket says 172.21.15.57:34150) 2024-04-13T05:18:01.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.118+0000 7f93d6800640 1 -- 172.21.15.57:0/2210253229 learned_addr learned my addr 172.21.15.57:0/2210253229 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:01.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d7200640 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b090fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d5e00640 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b0915250 0x5595b09176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d6800640 1 -- 172.21.15.57:0/2210253229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b0915250 msgr2=0x5595b09176e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d6800640 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b0915250 0x5595b09176e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d6800640 1 -- 172.21.15.57:0/2210253229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 msgr2=0x5595b090fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d6800640 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b090fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d6800640 1 -- 172.21.15.57:0/2210253229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b06527d0 con 0x5595b09100e0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d7200640 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b090fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93cf400640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93bc0a3c30 0x7f93bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.119+0000 7f93d6800640 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 0x5595b0914be0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009d50 tx=0x7f93c80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d8d892cf82e431 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.120+0000 7f93cf400640 1 -- 172.21.15.57:0/557958975 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f93c40f2a00 con 0x7f93d8105280 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.120+0000 7f93d7200640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93bc0a3c30 0x7f93bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.120+0000 7f93c3400640 1 -- 172.21.15.57:0/2210253229 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8012650 con 0x5595b09100e0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.120+0000 7f93c3400640 1 -- 172.21.15.57:0/2210253229 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93c8009e40 con 0x5595b09100e0 2024-04-13T05:18:01.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.120+0000 7f93c3400640 1 -- 172.21.15.57:0/2210253229 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c800fbf0 con 0x5595b09100e0 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 -- 172.21.15.57:0/2210253229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 msgr2=0x5595b0914be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93d7200640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93bc0a3c30 0x7f93bc0a60f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5595b0910940 tx=0x7f93d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 0x5595b0914be0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009d50 tx=0x7f93c80117b0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 -- 172.21.15.57:0/2210253229 shutdown_connections 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b0915250 0x5595b09176e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b09100e0 0x5595b0914be0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 --2- 172.21.15.57:0/2210253229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b090fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 -- 172.21.15.57:0/2210253229 >> 172.21.15.57:0/2210253229 conn(0x5595b090ac60 msgr2=0x5595b090b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 -- 172.21.15.57:0/2210253229 shutdown_connections 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 -- 172.21.15.57:0/2210253229 wait complete. 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.121+0000 7f93e0419740 1 -- start start 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 Processor -- start 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b09199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0919f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 0x5595b0920f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5595b0918200 con 0x5595b0915250 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595b082fa90 con 0x5595b09100e0 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5595b0918380 con 0x5595b090f6f0 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93e0419740 10 client.0 _ll_register_callbacks cb 0x5595b082f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:01.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.122+0000 7f93d7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 0x5595b0920f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 0x5595b0920f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34156/0 (socket says 172.21.15.57:34156) 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 -- 172.21.15.57:0/4170242105 learned_addr learned my addr 172.21.15.57:0/4170242105 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d5e00640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0919f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d6800640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b09199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 -- 172.21.15.57:0/4170242105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 msgr2=0x5595b09199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b09199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 -- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 msgr2=0x5595b0919f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0919f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 -- 172.21.15.57:0/4170242105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b05883f0 con 0x5595b0915250 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93d7200640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 0x5595b0920f70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f93d0013fd0 tx=0x7f93d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc0f4cd0d0c4b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.123+0000 7f93b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93b7e00640 1 -- 172.21.15.57:0/4170242105 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d002c060 con 0x5595b0915250 2024-04-13T05:18:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93b7e00640 1 -- 172.21.15.57:0/4170242105 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f93d001dd10 con 0x5595b0915250 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595b0815600 con 0x5595b0915250 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93b7e00640 1 -- 172.21.15.57:0/4170242105 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93d0033050 con 0x5595b0915250 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93e0419740 10 client.15609 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595b05480f0 con 0x5595b0915250 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.124+0000 7f93b7e00640 1 -- 172.21.15.57:0/4170242105 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5595b0815600 con 0x5595b0915250 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93b7e00640 1 -- 172.21.15.57:0/4170242105 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f93d000e4f0 con 0x5595b0915250 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93e0419740 10 client.15609 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93e0419740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93e0419740 20 client.15609 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93e0419740 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b09259a0 0x5595b0927df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93e0419740 4 client.15609 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595b0883a40 con 0x5595b09259a0 2024-04-13T05:18:01.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.125+0000 7f93d6800640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b09259a0 0x5595b0927df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.126+0000 7f93d6800640 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b09259a0 0x5595b0927df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447760e4e119016 server_cookie=1af9062d3a256ef9 in_seq=0 out_seq=0 2024-04-13T05:18:01.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.126+0000 7f93b7e00640 10 client.15609 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:01.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.128+0000 7f93b7e00640 1 -- 172.21.15.57:0/4170242105 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595b0883a40 con 0x5595b09259a0 2024-04-13T05:18:01.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.128+0000 7f93b7e00640 10 client.15609 handle_command_reply: tid=0 2024-04-13T05:18:01.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.128+0000 7f93e0419740 1 client.15609 shutdown 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 msgr2=0x5595b0920f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 0x5595b0920f70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f93d0013fd0 tx=0x7f93d0004a40 comp rx=0 tx=0).stop 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 shutdown_connections 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b09259a0 0x5595b0927df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0919f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b0915250 0x5595b0920f70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 --2- 172.21.15.57:0/4170242105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b09199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 >> 172.21.15.57:0/4170242105 conn(0x5595b090ac60 msgr2=0x5595b0909e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93b7e00640 0 client.15609 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:01.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 shutdown_connections 2024-04-13T05:18:01.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 1 -- 172.21.15.57:0/4170242105 wait complete. 2024-04-13T05:18:01.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.129+0000 7f93e0419740 20 client.15609 trim_cache size 0 max 16384 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.150+0000 7f93e0419740 1 Processor -- start 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.150+0000 7f93e0419740 1 -- start start 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93e0419740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b09199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93e0419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b0919f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93e0419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0920f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93e0419740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81af3e0 con 0x5595b090ac60 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93e0419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81accb0 con 0x5595b09100e0 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93e0419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81aab00 con 0x5595b090f6f0 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b09199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b09199f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34162/0 (socket says 172.21.15.57:34162) 2024-04-13T05:18:01.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d6800640 1 -- 172.21.15.57:0/3780257798 learned_addr learned my addr 172.21.15.57:0/3780257798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d5e00640 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b0919f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d7200640 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0920f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d6800640 1 -- 172.21.15.57:0/3780257798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 msgr2=0x5595b0919f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.151+0000 7f93d6800640 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b0919f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93d6800640 1 -- 172.21.15.57:0/3780257798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 msgr2=0x5595b0920f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93d6800640 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0920f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93d6800640 1 -- 172.21.15.57:0/3780257798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b0910970 con 0x5595b090ac60 2024-04-13T05:18:01.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93d7200640 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0920f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:01.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93d6800640 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b09199f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005940 tx=0x7f93c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beedd05de2612635 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:01.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93c2a00640 1 -- 172.21.15.57:0/3780257798 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8005130 con 0x5595b090ac60 2024-04-13T05:18:01.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93c2a00640 1 -- 172.21.15.57:0/3780257798 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93c802e070 con 0x5595b090ac60 2024-04-13T05:18:01.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.152+0000 7f93c2a00640 1 -- 172.21.15.57:0/3780257798 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8003850 con 0x5595b090ac60 2024-04-13T05:18:01.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 -- 172.21.15.57:0/3780257798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 msgr2=0x5595b09199f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b09199f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005940 tx=0x7f93c8005f90 comp rx=0 tx=0).stop 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 -- 172.21.15.57:0/3780257798 shutdown_connections 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b09100e0 0x5595b0920f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b090f6f0 0x5595b0919f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 --2- 172.21.15.57:0/3780257798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b09199f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.153+0000 7f93e0419740 1 -- 172.21.15.57:0/3780257798 >> 172.21.15.57:0/3780257798 conn(0x5595b0914d20 msgr2=0x5595b0905d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.154+0000 7f93e0419740 1 -- 172.21.15.57:0/3780257798 shutdown_connections 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.154+0000 7f93e0419740 1 -- 172.21.15.57:0/3780257798 wait complete. 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.154+0000 7f93e0419740 1 -- start start 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.154+0000 7f93e0419740 1 Processor -- start 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b0920f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b091d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b09100e0 0x5595b091df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81ab170 con 0x5595b090ac60 2024-04-13T05:18:01.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81ac5e0 con 0x5595b090f6f0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81ab840 con 0x5595b09100e0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93e0419740 10 client.0 _ll_register_callbacks cb 0x7f93d0010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.155+0000 7f93d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b0920f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b0920f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34178/0 (socket says 172.21.15.57:34178) 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 -- 172.21.15.57:0/3468617932 learned_addr learned my addr 172.21.15.57:0/3468617932 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d5e00640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b091d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d7200640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b09100e0 0x5595b091df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 -- 172.21.15.57:0/3468617932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b09100e0 msgr2=0x5595b091df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b09100e0 0x5595b091df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 -- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 msgr2=0x5595b091d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b091d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 -- 172.21.15.57:0/3468617932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b0989f20 con 0x5595b090ac60 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93d6800640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b0920f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f93c802b040 tx=0x7f93c80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31587fa18ce53f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.156+0000 7f93b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:01.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93b7e00640 1 -- 172.21.15.57:0/3468617932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8005130 con 0x5595b090ac60 2024-04-13T05:18:01.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93b7e00640 1 -- 172.21.15.57:0/3468617932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f93c80052d0 con 0x5595b090ac60 2024-04-13T05:18:01.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c802e070 con 0x5595b090ac60 2024-04-13T05:18:01.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93b7e00640 1 -- 172.21.15.57:0/3468617932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93c8045050 con 0x5595b090ac60 2024-04-13T05:18:01.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93e0419740 10 client.15615 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:01.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595b098b170 con 0x5595b090ac60 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93b7e00640 1 -- 172.21.15.57:0/3468617932 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93c802e070 con 0x5595b090ac60 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.157+0000 7f93b7e00640 1 -- 172.21.15.57:0/3468617932 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f93c800f580 con 0x5595b090ac60 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93e0419740 10 client.15615 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93e0419740 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93e0419740 20 client.15615 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93e0419740 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b0915250 0x5595b098d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93e0419740 4 client.15615 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595b087d8b0 con 0x5595b0915250 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.158+0000 7f93d5e00640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b0915250 0x5595b098d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.159+0000 7f93d5e00640 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b0915250 0x5595b098d9c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42cf53b467c49928 server_cookie=ac809e0eae11d9d3 in_seq=0 out_seq=0 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.159+0000 7f93b7e00640 10 client.15615 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93b7e00640 1 -- 172.21.15.57:0/3468617932 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5595b087d8b0 con 0x5595b0915250 2024-04-13T05:18:01.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93b7e00640 10 client.15615 handle_command_reply: tid=0 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 client.15615 shutdown 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 msgr2=0x5595b0920f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b0920f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f93c802b040 tx=0x7f93c80093f0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 shutdown_connections 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5595b0915250 0x5595b098d9c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595b090f6f0 0x5595b091d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595b09100e0 0x5595b091df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 --2- 172.21.15.57:0/3468617932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595b090ac60 0x5595b0920f00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 >> 172.21.15.57:0/3468617932 conn(0x5595b0914d20 msgr2=0x5595b09199f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93b7e00640 0 client.15615 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 shutdown_connections 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.160+0000 7f93e0419740 1 -- 172.21.15.57:0/3468617932 wait complete. 2024-04-13T05:18:01.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93e0419740 20 client.15615 trim_cache size 0 max 16384 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: "887acbe1-f535-492a-a287-7dc3677ff570": { 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "887acbe1-f535-492a-a287-7dc3677ff570", 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:18:01.165 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:18:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93bc0a3c30 msgr2=0x7f93bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93bc0a3c30 0x7f93bc0a60f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5595b0910940 tx=0x7f93d0009500 comp rx=0 tx=0).stop 2024-04-13T05:18:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 msgr2=0x7f93d819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d819a870 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f93c4000c00 tx=0x7f93c4005e60 comp rx=0 tx=0).stop 2024-04-13T05:18:01.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 shutdown_connections 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f93bc0a3c30 0x7f93bc0a60f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d8107bb0 0x7f93d81a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93d8105280 0x7f93d819a870 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 --2- 172.21.15.57:0/557958975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93d8102950 0x7f93d819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.161+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 >> 172.21.15.57:0/557958975 conn(0x7f93d80fc360 msgr2=0x7f93d81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.162+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 shutdown_connections 2024-04-13T05:18:01.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:01.162+0000 7f93dca00640 1 -- 172.21.15.57:0/557958975 wait complete. 2024-04-13T05:18:01.181 INFO:tasks.cephfs.filesystem:scrub status for tag:887acbe1-f535-492a-a287-7dc3677ff570 - {'path': '/', 'tag': '887acbe1-f535-492a-a287-7dc3677ff570', 'options': 'recursive,force'} 2024-04-13T05:18:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:01 smithi083 ceph-mon[35133]: pgmap v468: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-13T05:18:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1578830773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:18:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:01 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:18:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:01 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:01 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:18:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:01 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:01 smithi057 ceph-mon[32331]: pgmap v468: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-13T05:18:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1578830773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:18:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:01 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:18:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:01 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:01 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:18:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:01 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:01 smithi189 ceph-mon[35531]: pgmap v468: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-13T05:18:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1578830773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:18:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:01 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:18:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:01 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:01 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:18:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:01 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:03 smithi057 ceph-mon[32331]: pgmap v469: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-13T05:18:03.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:03 smithi189 ceph-mon[35531]: pgmap v469: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-13T05:18:03.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:03 smithi083 ceph-mon[35133]: pgmap v469: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-13T05:18:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:05 smithi057 ceph-mon[32331]: pgmap v470: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:18:05.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:05 smithi189 ceph-mon[35531]: pgmap v470: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:18:05.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:05 smithi083 ceph-mon[35133]: pgmap v470: 129 pgs: 129 active+clean; 648 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-13T05:18:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:07 smithi057 ceph-mon[32331]: pgmap v471: 129 pgs: 129 active+clean; 653 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-13T05:18:07.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:07 smithi189 ceph-mon[35531]: pgmap v471: 129 pgs: 129 active+clean; 653 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-13T05:18:07.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:07 smithi083 ceph-mon[35133]: pgmap v471: 129 pgs: 129 active+clean; 653 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-13T05:18:09.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:09 smithi057 ceph-mon[32331]: pgmap v472: 129 pgs: 129 active+clean; 653 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 5 op/s 2024-04-13T05:18:09.554 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:09 smithi083 ceph-mon[35133]: pgmap v472: 129 pgs: 129 active+clean; 653 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 5 op/s 2024-04-13T05:18:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:09 smithi189 ceph-mon[35531]: pgmap v472: 129 pgs: 129 active+clean; 653 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 5 op/s 2024-04-13T05:18:11.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:11 smithi189 ceph-mon[35531]: pgmap v473: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-13T05:18:11.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:11 smithi083 ceph-mon[35133]: pgmap v473: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-13T05:18:11.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:11 smithi057 ceph-mon[32331]: pgmap v473: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-13T05:18:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:12 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:18:12.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:12 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:18:12.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:12 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:18:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:13 smithi189 ceph-mon[35531]: pgmap v474: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-13T05:18:13.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:13 smithi083 ceph-mon[35133]: pgmap v474: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-13T05:18:13.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:13 smithi057 ceph-mon[32331]: pgmap v474: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-13T05:18:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:15 smithi189 ceph-mon[35531]: pgmap v475: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-13T05:18:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:15 smithi083 ceph-mon[35133]: pgmap v475: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-13T05:18:15.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:15 smithi057 ceph-mon[32331]: pgmap v475: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-13T05:18:17.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:17 smithi189 ceph-mon[35531]: pgmap v476: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-13T05:18:17.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:17 smithi083 ceph-mon[35133]: pgmap v476: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-13T05:18:17.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:17 smithi057 ceph-mon[32331]: pgmap v476: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-13T05:18:19.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:19 smithi189 ceph-mon[35531]: pgmap v477: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-13T05:18:19.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:19 smithi083 ceph-mon[35133]: pgmap v477: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-13T05:18:19.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:19 smithi057 ceph-mon[32331]: pgmap v477: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-13T05:18:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:21 smithi189 ceph-mon[35531]: pgmap v478: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-13T05:18:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:21 smithi083 ceph-mon[35133]: pgmap v478: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-13T05:18:21.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:21 smithi057 ceph-mon[32331]: pgmap v478: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-13T05:18:23.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:23 smithi057 ceph-mon[32331]: pgmap v479: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-13T05:18:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:23 smithi083 ceph-mon[35133]: pgmap v479: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-13T05:18:23.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:23 smithi189 ceph-mon[35531]: pgmap v479: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-13T05:18:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:25 smithi083 ceph-mon[35133]: pgmap v480: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:18:25.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:25 smithi057 ceph-mon[32331]: pgmap v480: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:18:25.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:25 smithi189 ceph-mon[35531]: pgmap v480: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:18:26.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:26 smithi083 ceph-mon[35133]: pgmap v481: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:18:26.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:26 smithi057 ceph-mon[32331]: pgmap v481: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:18:26.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:26 smithi189 ceph-mon[35531]: pgmap v481: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:18:29.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:28 smithi057 ceph-mon[32331]: pgmap v482: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:29.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:28 smithi189 ceph-mon[35531]: pgmap v482: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:29.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:28 smithi083 ceph-mon[35133]: pgmap v482: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:30 smithi057 ceph-mon[32331]: pgmap v483: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:18:31.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:30 smithi189 ceph-mon[35531]: pgmap v483: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:18:31.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:18:31.182 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:18:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:30 smithi083 ceph-mon[35133]: pgmap v483: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 -- 172.21.15.57:0/3748829911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73641096f0 msgr2=0x7f7364109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 --2- 172.21.15.57:0/3748829911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73641096f0 0x7f7364109b70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 -- 172.21.15.57:0/3748829911 shutdown_connections 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 --2- 172.21.15.57:0/3748829911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 0x7f7364110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 --2- 172.21.15.57:0/3748829911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73641096f0 0x7f7364109b70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 --2- 172.21.15.57:0/3748829911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7364108500 0x7f7364108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.426+0000 7f7363e00640 1 -- 172.21.15.57:0/3748829911 >> 172.21.15.57:0/3748829911 conn(0x7f7364077490 msgr2=0x7f73640798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.427+0000 7f7363e00640 1 -- 172.21.15.57:0/3748829911 shutdown_connections 2024-04-13T05:18:31.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.427+0000 7f7363e00640 1 -- 172.21.15.57:0/3748829911 wait complete. 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 Processor -- start 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 -- start start 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 0x7f736419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73641096f0 0x7f736419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 0x7f73641a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7364114080 con 0x7f7364108500 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7364113f00 con 0x7f73641096f0 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7363e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7364114200 con 0x7f736410a0b0 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 0x7f736419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 0x7f736419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36748/0 (socket says 172.21.15.57:36748) 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.428+0000 7f7362a00640 1 -- 172.21.15.57:0/3052546879 learned_addr learned my addr 172.21.15.57:0/3052546879 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:31.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f735a000640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73641096f0 0x7f736419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7362a00640 1 -- 172.21.15.57:0/3052546879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 msgr2=0x7f73641a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7363400640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 0x7f73641a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7362a00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 0x7f73641a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7362a00640 1 -- 172.21.15.57:0/3052546879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73641096f0 msgr2=0x7f736419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7362a00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73641096f0 0x7f736419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7362a00640 1 -- 172.21.15.57:0/3052546879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73641a63a0 con 0x7f7364108500 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7363400640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 0x7f73641a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.429+0000 7f7362a00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 0x7f736419e780 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f735400c970 tx=0x7f735400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.430+0000 7f7368a00640 1 -- 172.21.15.57:0/3052546879 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354004220 con 0x7f7364108500 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.430+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73641a6690 con 0x7f7364108500 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.430+0000 7f7368a00640 1 -- 172.21.15.57:0/3052546879 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73540043c0 con 0x7f7364108500 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.430+0000 7f7368a00640 1 -- 172.21.15.57:0/3052546879 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735400f450 con 0x7f7364108500 2024-04-13T05:18:31.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.430+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73641a6cb0 con 0x7f7364108500 2024-04-13T05:18:31.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 Processor -- start 2024-04-13T05:18:31.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 -- start start 2024-04-13T05:18:31.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf26f0 0x55b39adf2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39adf7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39adfa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b39ad59020 con 0x55b39adf30e0 2024-04-13T05:18:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.434+0000 7f736cdb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b39acff6d0 con 0x55b39adf8250 2024-04-13T05:18:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f736cdb5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b39ad54fe0 con 0x55b39adf26f0 2024-04-13T05:18:31.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f7368a00640 1 -- 172.21.15.57:0/3052546879 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73540079b0 con 0x7f7364108500 2024-04-13T05:18:31.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39adf7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f7363400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf26f0 0x55b39adf2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f735a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39adfa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f7363400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf26f0 0x55b39adf2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:49294/0 (socket says 172.21.15.57:49294) 2024-04-13T05:18:31.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39adf7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36760/0 (socket says 172.21.15.57:36760) 2024-04-13T05:18:31.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f735a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39adfa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:33108/0 (socket says 172.21.15.57:33108) 2024-04-13T05:18:31.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.435+0000 7f7363400640 1 -- 172.21.15.57:0/3011143818 learned_addr learned my addr 172.21.15.57:0/3011143818 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:31.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.436+0000 7f7362a00640 1 -- 172.21.15.57:0/3011143818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf26f0 msgr2=0x55b39adf2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.436+0000 7f7362a00640 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf26f0 0x55b39adf2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.436+0000 7f7362a00640 1 -- 172.21.15.57:0/3011143818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 msgr2=0x55b39adfa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.436+0000 7f7362a00640 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39adfa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.436+0000 7f7362a00640 1 -- 172.21.15.57:0/3011143818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39ab357d0 con 0x55b39adf30e0 2024-04-13T05:18:31.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.436+0000 7f7362a00640 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39adf7be0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73540cdc60 tx=0x7f73540d6b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc89d5444822551 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.437+0000 7f7353e00640 1 -- 172.21.15.57:0/3011143818 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540e3070 con 0x55b39adf30e0 2024-04-13T05:18:31.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.437+0000 7f7353e00640 1 -- 172.21.15.57:0/3011143818 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73540d0bb0 con 0x55b39adf30e0 2024-04-13T05:18:31.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.437+0000 7f7353e00640 1 -- 172.21.15.57:0/3011143818 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73540d89e0 con 0x55b39adf30e0 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.437+0000 7f7368a00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73480a3c30 0x7f73480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.437+0000 7f7368a00640 1 -- 172.21.15.57:0/3052546879 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f73540d05c0 con 0x7f7364108500 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.437+0000 7f7363400640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73480a3c30 0x7f73480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 -- 172.21.15.57:0/3011143818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 msgr2=0x55b39adf7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39adf7be0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f73540cdc60 tx=0x7f73540d6b20 comp rx=0 tx=0).stop 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 -- 172.21.15.57:0/3011143818 shutdown_connections 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39adfa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39adf7be0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 --2- 172.21.15.57:0/3011143818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf26f0 0x55b39adf2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 -- 172.21.15.57:0/3011143818 >> 172.21.15.57:0/3011143818 conn(0x55b39adedc60 msgr2=0x55b39adee060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 -- 172.21.15.57:0/3011143818 shutdown_connections 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f736cdb5740 1 -- 172.21.15.57:0/3011143818 wait complete. 2024-04-13T05:18:31.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.438+0000 7f7363400640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73480a3c30 0x7f73480a60f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b39adf3940 tx=0x7f735c007800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 -- start start 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 Processor -- start 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39adfc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf30e0 0x55b39adfcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39ae03f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b39adfb200 con 0x55b39adf26f0 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b39ad12a90 con 0x55b39adf30e0 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b39adfb380 con 0x55b39adf8250 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f736cdb5740 10 client.0 _ll_register_callbacks cb 0x55b39ad12480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.439+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39adfc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39adfc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36770/0 (socket says 172.21.15.57:36770) 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 -- 172.21.15.57:0/2714981647 learned_addr learned my addr 172.21.15.57:0/2714981647 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f735a000640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf30e0 0x55b39adfcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7363400640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39ae03f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 -- 172.21.15.57:0/2714981647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 msgr2=0x55b39ae03f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39ae03f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 -- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf30e0 msgr2=0x55b39adfcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf30e0 0x55b39adfcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.440+0000 7f7362a00640 1 -- 172.21.15.57:0/2714981647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39aa6b3f0 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f7362a00640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39adfc9e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f73540a4be0 tx=0x7f7354000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d54323e700ede14 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f7350c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f7350c00640 1 -- 172.21.15.57:0/2714981647 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735401d070 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f7350c00640 1 -- 172.21.15.57:0/2714981647 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f73540d6280 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73540d0bb0 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f7350c00640 1 -- 172.21.15.57:0/2714981647 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7354098740 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f736cdb5740 10 client.15627 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.441+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b39aa2b0f0 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f7350c00640 1 -- 172.21.15.57:0/2714981647 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73540d0bb0 con 0x55b39adf26f0 2024-04-13T05:18:31.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f7350c00640 1 -- 172.21.15.57:0/2714981647 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f735409c700 con 0x55b39adf26f0 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f736cdb5740 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f736cdb5740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f736cdb5740 20 client.15627 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae08450 0x55b39ae0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f736cdb5740 4 client.15627 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.442+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39acf8600 con 0x55b39ae08450 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.443+0000 7f735a000640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae08450 0x55b39ae0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.444+0000 7f735a000640 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae08450 0x55b39ae0a840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b50cac77d1f744f server_cookie=d4cbeb09dc3b79a8 in_seq=0 out_seq=0 2024-04-13T05:18:31.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.444+0000 7f7350c00640 10 client.15627 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:31.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f7350c00640 1 -- 172.21.15.57:0/2714981647 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b39acf8600 con 0x55b39ae08450 2024-04-13T05:18:31.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f7350c00640 10 client.15627 handle_command_reply: tid=0 2024-04-13T05:18:31.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 client.15627 shutdown 2024-04-13T05:18:31.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 msgr2=0x55b39adfc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39adfc9e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f73540a4be0 tx=0x7f7354000bc0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 shutdown_connections 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae08450 0x55b39ae0a840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf30e0 0x55b39adfcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39ae03f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f7350c00640 0 client.15627 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2714981647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39adfc9e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 >> 172.21.15.57:0/2714981647 conn(0x55b39adedc60 msgr2=0x55b39adece80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 shutdown_connections 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 1 -- 172.21.15.57:0/2714981647 wait complete. 2024-04-13T05:18:31.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.446+0000 7f736cdb5740 20 client.15627 trim_cache size 0 max 16384 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 Processor -- start 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 -- start start 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf26f0 0x55b39adfce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39ae03f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39adf15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f73641b3ae0 con 0x55b39adf30e0 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73641b13b0 con 0x55b39adf26f0 2024-04-13T05:18:31.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f736cdb5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73641af200 con 0x55b39adf8250 2024-04-13T05:18:31.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f735a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39ae03f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f735a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39ae03f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36774/0 (socket says 172.21.15.57:36774) 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.466+0000 7f735a000640 1 -- 172.21.15.57:0/211805244 learned_addr learned my addr 172.21.15.57:0/211805244 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f7363400640 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39adf15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f7362a00640 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf26f0 0x55b39adfce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f735a000640 1 -- 172.21.15.57:0/211805244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 msgr2=0x55b39adf15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f735a000640 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39adf15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f735a000640 1 -- 172.21.15.57:0/211805244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf26f0 msgr2=0x55b39adfce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f735a000640 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf26f0 0x55b39adfce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f735a000640 1 -- 172.21.15.57:0/211805244 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39adf7680 con 0x55b39adf30e0 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f7362a00640 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf26f0 0x55b39adfce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.467+0000 7f735a000640 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39ae03f20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e92ad94e540b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f7353400640 1 -- 172.21.15.57:0/211805244 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c047050 con 0x55b39adf30e0 2024-04-13T05:18:31.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f7353400640 1 -- 172.21.15.57:0/211805244 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f734c049bd0 con 0x55b39adf30e0 2024-04-13T05:18:31.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f7353400640 1 -- 172.21.15.57:0/211805244 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c005460 con 0x55b39adf30e0 2024-04-13T05:18:31.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 -- 172.21.15.57:0/211805244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 msgr2=0x55b39ae03f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39ae03f20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c02c7e0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 -- 172.21.15.57:0/211805244 shutdown_connections 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf8250 0x55b39adf15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf30e0 0x55b39ae03f20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 --2- 172.21.15.57:0/211805244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf26f0 0x55b39adfce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 -- 172.21.15.57:0/211805244 >> 172.21.15.57:0/211805244 conn(0x55b39adedc60 msgr2=0x55b39adf0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 -- 172.21.15.57:0/211805244 shutdown_connections 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.468+0000 7f736cdb5740 1 -- 172.21.15.57:0/211805244 wait complete. 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 -- start start 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 Processor -- start 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39ae01800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf30e0 0x55b39ae01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39ae02d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f73641af870 con 0x55b39adf26f0 2024-04-13T05:18:31.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73641b0ce0 con 0x55b39adf8250 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73641aff40 con 0x55b39adf30e0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.469+0000 7f736cdb5740 10 client.0 _ll_register_callbacks cb 0x7f735c0076a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39ae01800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39ae01800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36786/0 (socket says 172.21.15.57:36786) 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 -- 172.21.15.57:0/2269338567 learned_addr learned my addr 172.21.15.57:0/2269338567 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f735a000640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf30e0 0x55b39ae01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7363400640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39ae02d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 -- 172.21.15.57:0/2269338567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf30e0 msgr2=0x55b39ae01d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf30e0 0x55b39ae01d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 -- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 msgr2=0x55b39ae02d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39ae02d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 -- 172.21.15.57:0/2269338567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39ae03340 con 0x55b39adf26f0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f735a000640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf30e0 0x55b39ae01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7362a00640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39ae01800 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f73540a5110 tx=0x7f73540d6b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1117409fd7061e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.470+0000 7f7350c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f7350c00640 1 -- 172.21.15.57:0/2269338567 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735401d070 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f7350c00640 1 -- 172.21.15.57:0/2269338567 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7354007ca0 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c049bd0 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f7350c00640 1 -- 172.21.15.57:0/2269338567 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f735409be00 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 10 client.15633 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b39ae6e830 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f7350c00640 1 -- 172.21.15.57:0/2269338567 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f734c049bd0 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f7350c00640 1 -- 172.21.15.57:0/2269338567 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f73540a4c80 con 0x55b39adf26f0 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 10 client.15633 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 20 client.15633 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.471+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae6ee50 0x55b39ae712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.472+0000 7f736cdb5740 4 client.15633 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:18:31.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.472+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b39ad66a40 con 0x55b39ae6ee50 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.472+0000 7f735a000640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae6ee50 0x55b39ae712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.473+0000 7f735a000640 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae6ee50 0x55b39ae712b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d9d93f8127b5ad server_cookie=6b4b9ee5362cd37f in_seq=0 out_seq=0 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.473+0000 7f7350c00640 10 client.15633 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f7350c00640 1 -- 172.21.15.57:0/2269338567 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b39ad66a40 con 0x55b39ae6ee50 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f7350c00640 10 client.15633 handle_command_reply: tid=0 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 client.15633 shutdown 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 msgr2=0x55b39ae01800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39ae01800 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f73540a5110 tx=0x7f73540d6b20 comp rx=0 tx=0).stop 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 shutdown_connections 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b39ae6ee50 0x55b39ae712b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b39adf8250 0x55b39ae02d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b39adf30e0 0x55b39ae01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f7350c00640 0 client.15633 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 --2- 172.21.15.57:0/2269338567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b39adf26f0 0x55b39ae01800 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 >> 172.21.15.57:0/2269338567 conn(0x55b39adedc60 msgr2=0x55b39adf1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 shutdown_connections 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.474+0000 7f736cdb5740 1 -- 172.21.15.57:0/2269338567 wait complete. 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f736cdb5740 20 client.15633 trim_cache size 0 max 16384 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:18:31.479 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:18:31.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73480a3c30 msgr2=0x7f73480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73480a3c30 0x7f73480a60f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b39adf3940 tx=0x7f735c007800 comp rx=0 tx=0).stop 2024-04-13T05:18:31.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 msgr2=0x7f736419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 0x7f736419e780 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f735400c970 tx=0x7f735400ce30 comp rx=0 tx=0).stop 2024-04-13T05:18:31.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 shutdown_connections 2024-04-13T05:18:31.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f73480a3c30 0x7f73480a60f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73641096f0 0x7f736419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f736410a0b0 0x7f73641a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 --2- 172.21.15.57:0/3052546879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7364108500 0x7f736419e780 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.475+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 >> 172.21.15.57:0/3052546879 conn(0x7f7364077490 msgr2=0x7f736410ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.476+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 shutdown_connections 2024-04-13T05:18:31.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.476+0000 7f7363e00640 1 -- 172.21.15.57:0/3052546879 wait complete. 2024-04-13T05:18:31.497 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:18:31.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.721+0000 7f270a200640 1 -- 172.21.15.57:0/3183043734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 msgr2=0x7f2704109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.721+0000 7f270a200640 1 --2- 172.21.15.57:0/3183043734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f2704109c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009900 tx=0x7f26ec02bc30 comp rx=0 tx=0).stop 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 -- 172.21.15.57:0/3183043734 shutdown_connections 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 --2- 172.21.15.57:0/3183043734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f270410a1d0 0x7f2704110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 --2- 172.21.15.57:0/3183043734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f2704109c90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 --2- 172.21.15.57:0/3183043734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2704108620 0x7f2704108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 -- 172.21.15.57:0/3183043734 >> 172.21.15.57:0/3183043734 conn(0x7f27040775e0 msgr2=0x7f2704079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 -- 172.21.15.57:0/3183043734 shutdown_connections 2024-04-13T05:18:31.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.722+0000 7f270a200640 1 -- 172.21.15.57:0/3183043734 wait complete. 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 Processor -- start 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 -- start start 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2704108620 0x7f270419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f270419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270410a1d0 0x7f27041a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2704114160 con 0x7f2704109810 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2704113fe0 con 0x7f2704108620 2024-04-13T05:18:31.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f270a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27041142e0 con 0x7f270410a1d0 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f2702a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f270419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.723+0000 7f2702a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f270419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36794/0 (socket says 172.21.15.57:36794) 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 -- 172.21.15.57:0/1934269422 learned_addr learned my addr 172.21.15.57:0/1934269422 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2703400640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2704108620 0x7f270419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2703e00640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270410a1d0 0x7f27041a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 -- 172.21.15.57:0/1934269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270410a1d0 msgr2=0x7f27041a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270410a1d0 0x7f27041a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 -- 172.21.15.57:0/1934269422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2704108620 msgr2=0x7f270419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2704108620 0x7f270419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 -- 172.21.15.57:0/1934269422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27041a64b0 con 0x7f2704109810 2024-04-13T05:18:31.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.724+0000 7f2702a00640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f270419edd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f26ec006010 tx=0x7f26ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.725+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec043020 con 0x7f2704109810 2024-04-13T05:18:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.725+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26ec004aa0 con 0x7f2704109810 2024-04-13T05:18:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.725+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27041a6740 con 0x7f2704109810 2024-04-13T05:18:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.725+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec003c60 con 0x7f2704109810 2024-04-13T05:18:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.725+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041a6ce0 con 0x7f2704109810 2024-04-13T05:18:31.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.726+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270410f9b0 con 0x7f2704109810 2024-04-13T05:18:31.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.729+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26ec046020 con 0x7f2704109810 2024-04-13T05:18:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.730+0000 7f26fbe00640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e80a3c30 0x7f26e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.730+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f26ec0f3290 con 0x7f2704109810 2024-04-13T05:18:31.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.730+0000 7f2703400640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e80a3c30 0x7f26e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:31.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.731+0000 7f2703400640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e80a3c30 0x7f26e80a60f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2704109670 tx=0x7f26f4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:31.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.733+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26ec04d020 con 0x7f2704109810 2024-04-13T05:18:31.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.988+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f270402d7a0 con 0x7f2704109810 2024-04-13T05:18:31.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.989+0000 7f26fbe00640 1 -- 172.21.15.57:0/1934269422 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7f26ec0bf870 con 0x7f2704109810 2024-04-13T05:18:31.992 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:18:31.992 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:17:49.577177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:18:31.992 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-04-13T05:18:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e80a3c30 msgr2=0x7f26e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e80a3c30 0x7f26e80a60f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2704109670 tx=0x7f26f4009870 comp rx=0 tx=0).stop 2024-04-13T05:18:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 msgr2=0x7f270419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f270419edd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f26ec006010 tx=0x7f26ec0042f0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 shutdown_connections 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26e80a3c30 0x7f26e80a60f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270410a1d0 0x7f27041a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2704109810 0x7f270419edd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 --2- 172.21.15.57:0/1934269422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2704108620 0x7f270419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 >> 172.21.15.57:0/1934269422 conn(0x7f27040775e0 msgr2=0x7f270410cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 shutdown_connections 2024-04-13T05:18:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:31.992+0000 7f270a200640 1 -- 172.21.15.57:0/1934269422 wait complete. 2024-04-13T05:18:32.016 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 -- 172.21.15.57:0/3438732813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac10a0b0 msgr2=0x7f74ac110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 --2- 172.21.15.57:0/3438732813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac10a0b0 0x7f74ac110940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009ca0 tx=0x7f74a402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 -- 172.21.15.57:0/3438732813 shutdown_connections 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 --2- 172.21.15.57:0/3438732813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74ac1096f0 0x7f74ac109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 --2- 172.21.15.57:0/3438732813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac10a0b0 0x7f74ac110940 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 --2- 172.21.15.57:0/3438732813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74ac108500 0x7f74ac108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 -- 172.21.15.57:0/3438732813 >> 172.21.15.57:0/3438732813 conn(0x7f74ac077490 msgr2=0x7f74ac0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 -- 172.21.15.57:0/3438732813 shutdown_connections 2024-04-13T05:18:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.242+0000 7f74b1a00640 1 -- 172.21.15.57:0/3438732813 wait complete. 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.243+0000 7f74b1a00640 1 Processor -- start 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 -- start start 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74ac108500 0x7f74ac19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 0x7f74ac19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74ac10a0b0 0x7f74ac1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac114010 con 0x7f74ac1096f0 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac113e90 con 0x7f74ac10a0b0 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74b1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac114190 con 0x7f74ac108500 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 0x7f74ac19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 0x7f74ac19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36810/0 (socket says 172.21.15.57:36810) 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74a3e00640 1 -- 172.21.15.57:0/90697001 learned_addr learned my addr 172.21.15.57:0/90697001 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.244+0000 7f74abe00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74ac108500 0x7f74ac19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74b1000640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74ac10a0b0 0x7f74ac1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a3e00640 1 -- 172.21.15.57:0/90697001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74ac108500 msgr2=0x7f74ac19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a3e00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74ac108500 0x7f74ac19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a3e00640 1 -- 172.21.15.57:0/90697001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74ac10a0b0 msgr2=0x7f74ac1a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a3e00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74ac10a0b0 0x7f74ac1a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a3e00640 1 -- 172.21.15.57:0/90697001 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac1a62a0 con 0x7f74ac1096f0 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a3e00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 0x7f74ac19ebc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f749400d950 tx=0x7f749400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a9600640 1 -- 172.21.15.57:0/90697001 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494017070 con 0x7f74ac1096f0 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a9600640 1 -- 172.21.15.57:0/90697001 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7494009c30 con 0x7f74ac1096f0 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74a9600640 1 -- 172.21.15.57:0/90697001 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749400f620 con 0x7f74ac1096f0 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac1a6590 con 0x7f74ac1096f0 2024-04-13T05:18:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.245+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1a6a90 con 0x7f74ac1096f0 2024-04-13T05:18:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 Processor -- start 2024-04-13T05:18:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 -- start start 2024-04-13T05:18:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74a9600640 1 -- 172.21.15.57:0/90697001 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f74940109a0 con 0x7f74ac1096f0 2024-04-13T05:18:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 0x55fe791ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55fe79118020 con 0x55fe791b7fa0 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe790be950 con 0x55fe791b2440 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74b5f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe79113fe0 con 0x55fe791b2e30 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.249+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 0x55fe791ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 0x55fe791ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36816/0 (socket says 172.21.15.57:36816) 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 -- 172.21.15.57:0/2252699521 learned_addr learned my addr 172.21.15.57:0/2252699521 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74b1000640 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74abe00640 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 -- 172.21.15.57:0/2252699521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 msgr2=0x55fe791b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 -- 172.21.15.57:0/2252699521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 msgr2=0x55fe791b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74a3e00640 1 -- 172.21.15.57:0/2252699521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe78e2a3f0 con 0x55fe791b7fa0 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.250+0000 7f74abe00640 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74a3e00640 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 0x55fe791ba430 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f74940d6580 tx=0x7f74940d6a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5297d4b391bcb1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74a9600640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74900a3c30 0x7f74900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74b1000640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74900a3c30 0x7f74900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74a9600640 1 -- 172.21.15.57:0/90697001 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f74940d0400 con 0x7f74ac1096f0 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74a1600640 1 -- 172.21.15.57:0/2252699521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940e3070 con 0x55fe791b7fa0 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74a1600640 1 -- 172.21.15.57:0/2252699521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7494009de0 con 0x55fe791b7fa0 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.251+0000 7f74a1600640 1 -- 172.21.15.57:0/2252699521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749409bca0 con 0x55fe791b7fa0 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 -- 172.21.15.57:0/2252699521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 msgr2=0x55fe791ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 0x55fe791ba430 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f74940d6580 tx=0x7f74940d6a40 comp rx=0 tx=0).stop 2024-04-13T05:18:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b1000640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74900a3c30 0x7f74900a60f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f74a4006010 tx=0x7f74a4005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 -- 172.21.15.57:0/2252699521 shutdown_connections 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b7fa0 0x55fe791ba430 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2252699521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 -- 172.21.15.57:0/2252699521 >> 172.21.15.57:0/2252699521 conn(0x55fe791ad9b0 msgr2=0x55fe791addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 -- 172.21.15.57:0/2252699521 shutdown_connections 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 -- 172.21.15.57:0/2252699521 wait complete. 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.252+0000 7f74b5f70740 1 -- start start 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 Processor -- start 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 0x55fe791bc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b7fa0 0x55fe791c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55fe791baf50 con 0x55fe791b2440 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe790d1c80 con 0x55fe791b7fa0 2024-04-13T05:18:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe791bb0d0 con 0x55fe791b2e30 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.253+0000 7f74b5f70740 10 client.0 _ll_register_callbacks cb 0x55fe790d1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 0x55fe791bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 0x55fe791bc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36826/0 (socket says 172.21.15.57:36826) 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 -- 172.21.15.57:0/229750809 learned_addr learned my addr 172.21.15.57:0/229750809 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74b1000640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b7fa0 0x55fe791c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74a3e00640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 -- 172.21.15.57:0/229750809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 msgr2=0x55fe791bcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791bcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 -- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b7fa0 msgr2=0x55fe791c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b7fa0 0x55fe791c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 -- 172.21.15.57:0/229750809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe78ef47d0 con 0x55fe791b2440 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f74abe00640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 0x55fe791bc980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f749c00be10 tx=0x7f749c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8292690aaa4d78b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.254+0000 7f749a000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f749a000640 1 -- 172.21.15.57:0/229750809 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c012650 con 0x55fe791b2440 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f749a000640 1 -- 172.21.15.57:0/229750809 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f749c0127f0 con 0x55fe791b2440 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7494009de0 con 0x55fe791b2440 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f749a000640 1 -- 172.21.15.57:0/229750809 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f749c0129f0 con 0x55fe791b2440 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f74b5f70740 10 client.15651 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:32.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fe791c7340 con 0x55fe791b2440 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.255+0000 7f749a000640 1 -- 172.21.15.57:0/229750809 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7494009de0 con 0x55fe791b2440 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f749a000640 1 -- 172.21.15.57:0/229750809 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f749c01bc80 con 0x55fe791b2440 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74b5f70740 10 client.15651 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74b5f70740 10 client.15651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74b5f70740 20 client.15651 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74b5f70740 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe791c8a90 0x55fe791caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74b5f70740 4 client.15651 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe790b7750 con 0x55fe791c8a90 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.256+0000 7f74a3e00640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe791c8a90 0x55fe791caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.257+0000 7f74a3e00640 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe791c8a90 0x55fe791caf50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1c423f7211c7d85 server_cookie=870aa87afd4dfa9e in_seq=0 out_seq=0 2024-04-13T05:18:32.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.257+0000 7f749a000640 10 client.15651 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:18:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f749a000640 1 -- 172.21.15.57:0/229750809 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe790b7750 con 0x55fe791c8a90 2024-04-13T05:18:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f749a000640 10 client.15651 handle_command_reply: tid=0 2024-04-13T05:18:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 client.15651 shutdown 2024-04-13T05:18:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 msgr2=0x55fe791bc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 0x55fe791bc980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f749c00be10 tx=0x7f749c0117b0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 shutdown_connections 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe791c8a90 0x55fe791caf50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b7fa0 0x55fe791c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b2e30 0x55fe791bcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 --2- 172.21.15.57:0/229750809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2440 0x55fe791bc980 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f749a000640 0 client.15651 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.259+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 >> 172.21.15.57:0/229750809 conn(0x55fe791ad9b0 msgr2=0x55fe791accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.260+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 shutdown_connections 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.260+0000 7f74b5f70740 1 -- 172.21.15.57:0/229750809 wait complete. 2024-04-13T05:18:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.260+0000 7f74b5f70740 20 client.15651 trim_cache size 0 max 16384 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 Processor -- start 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 -- start start 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b38d0 con 0x55fe791b2e30 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b11a0 con 0x55fe791b2440 2024-04-13T05:18:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74b5f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1aeff0 con 0x55fe791b7fa0 2024-04-13T05:18:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36840/0 (socket says 172.21.15.57:36840) 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.275+0000 7f74a3e00640 1 -- 172.21.15.57:0/1080101206 learned_addr learned my addr 172.21.15.57:0/1080101206 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74abe00640 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74b1000640 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a3e00640 1 -- 172.21.15.57:0/1080101206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 msgr2=0x55fe791cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a3e00640 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a3e00640 1 -- 172.21.15.57:0/1080101206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 msgr2=0x55fe791cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a3e00640 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a3e00640 1 -- 172.21.15.57:0/1080101206 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe791b3210 con 0x55fe791b2e30 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74abe00640 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a3e00640 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c3ec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f74940d6f40 tx=0x7f7494005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172d6ba6f137ecb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.276+0000 7f74a0c00640 1 -- 172.21.15.57:0/1080101206 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494020040 con 0x55fe791b2e30 2024-04-13T05:18:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74a0c00640 1 -- 172.21.15.57:0/1080101206 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74940d61b0 con 0x55fe791b2e30 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74a0c00640 1 -- 172.21.15.57:0/1080101206 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940a06f0 con 0x55fe791b2e30 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 -- 172.21.15.57:0/1080101206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 msgr2=0x55fe791c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c3ec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f74940d6f40 tx=0x7f7494005f70 comp rx=0 tx=0).stop 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 -- 172.21.15.57:0/1080101206 shutdown_connections 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c3ec0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 --2- 172.21.15.57:0/1080101206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 -- 172.21.15.57:0/1080101206 >> 172.21.15.57:0/1080101206 conn(0x55fe791ad9b0 msgr2=0x55fe791b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 -- 172.21.15.57:0/1080101206 shutdown_connections 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.277+0000 7f74b5f70740 1 -- 172.21.15.57:0/1080101206 wait complete. 2024-04-13T05:18:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 -- start start 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 Processor -- start 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1af660 con 0x55fe791b2e30 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b0ad0 con 0x55fe791b2440 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1afd30 con 0x55fe791b7fa0 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.278+0000 7f74b5f70740 10 client.0 _ll_register_callbacks cb 0x7f749c00b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36852/0 (socket says 172.21.15.57:36852) 2024-04-13T05:18:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 -- 172.21.15.57:0/2163092930 learned_addr learned my addr 172.21.15.57:0/2163092930 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74b1000640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74abe00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 -- 172.21.15.57:0/2163092930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 msgr2=0x55fe791c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 -- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 msgr2=0x55fe791c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74a3e00640 1 -- 172.21.15.57:0/2163092930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe791c31b0 con 0x55fe791b2e30 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.279+0000 7f74abe00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f74a3e00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c1bb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7494098d10 tx=0x7f7494010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a628eb21f8291575 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f749a000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f749a000640 1 -- 172.21.15.57:0/2163092930 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494020040 con 0x55fe791b2e30 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f749a000640 1 -- 172.21.15.57:0/2163092930 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f74940cdbb0 con 0x55fe791b2e30 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe79125a40 con 0x55fe791b2e30 2024-04-13T05:18:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f749a000640 1 -- 172.21.15.57:0/2163092930 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f749409be80 con 0x55fe791b2e30 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f74b5f70740 10 client.15657 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.280+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fe7922f090 con 0x55fe791b2e30 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f749a000640 1 -- 172.21.15.57:0/2163092930 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55fe79125a40 con 0x55fe791b2e30 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f749a000640 1 -- 172.21.15.57:0/2163092930 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74940989b0 con 0x55fe791b2e30 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74b5f70740 10 client.15657 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74b5f70740 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74b5f70740 20 client.15657 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe7922f4e0 0x55fe792318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74b5f70740 4 client.15657 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe7911f8b0 con 0x55fe7922f4e0 2024-04-13T05:18:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.281+0000 7f74abe00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe7922f4e0 0x55fe792318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.282+0000 7f74abe00640 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe7922f4e0 0x55fe792318d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5747e68905ecb06 server_cookie=a9b5172d39377546 in_seq=0 out_seq=0 2024-04-13T05:18:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.282+0000 7f749a000640 10 client.15657 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:18:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f749a000640 1 -- 172.21.15.57:0/2163092930 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe7911f8b0 con 0x55fe7922f4e0 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f749a000640 10 client.15657 handle_command_reply: tid=0 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 client.15657 shutdown 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 msgr2=0x55fe791c1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c1bb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7494098d10 tx=0x7f7494010f70 comp rx=0 tx=0).stop 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 shutdown_connections 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55fe7922f4e0 0x55fe792318d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe791b7fa0 0x55fe791c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe791b2e30 0x55fe791c1bb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 --2- 172.21.15.57:0/2163092930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe791b2440 0x55fe791c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f749a000640 0 client.15657 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 >> 172.21.15.57:0/2163092930 conn(0x55fe791ad9b0 msgr2=0x55fe791ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 shutdown_connections 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.283+0000 7f74b5f70740 1 -- 172.21.15.57:0/2163092930 wait complete. 2024-04-13T05:18:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b5f70740 20 client.15657 trim_cache size 0 max 16384 2024-04-13T05:18:32.289 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:18:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74900a3c30 msgr2=0x7f74900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74900a3c30 0x7f74900a60f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f74a4006010 tx=0x7f74a4005c30 comp rx=0 tx=0).stop 2024-04-13T05:18:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 msgr2=0x7f74ac19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 0x7f74ac19ebc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f749400d950 tx=0x7f749400de10 comp rx=0 tx=0).stop 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 shutdown_connections 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74900a3c30 0x7f74900a60f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74ac10a0b0 0x7f74ac1a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac1096f0 0x7f74ac19ebc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 --2- 172.21.15.57:0/90697001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74ac108500 0x7f74ac19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.284+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 >> 172.21.15.57:0/90697001 conn(0x7f74ac077490 msgr2=0x7f74ac10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.285+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 shutdown_connections 2024-04-13T05:18:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.285+0000 7f74b1a00640 1 -- 172.21.15.57:0/90697001 wait complete. 2024-04-13T05:18:32.306 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:18:32.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 -- 172.21.15.57:0/2970455572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 msgr2=0x7f253c078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 --2- 172.21.15.57:0/2970455572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c078020 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2528009ca0 tx=0x7f252802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 -- 172.21.15.57:0/2970455572 shutdown_connections 2024-04-13T05:18:32.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 --2- 172.21.15.57:0/2970455572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f253c078560 0x7f253c078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 --2- 172.21.15.57:0/2970455572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c078020 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 --2- 172.21.15.57:0/2970455572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c076980 0x7f253c076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 -- 172.21.15.57:0/2970455572 >> 172.21.15.57:0/2970455572 conn(0x7f253c0ff670 msgr2=0x7f253c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 -- 172.21.15.57:0/2970455572 shutdown_connections 2024-04-13T05:18:32.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.523+0000 7f253be00640 1 -- 172.21.15.57:0/2970455572 wait complete. 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.524+0000 7f253be00640 1 Processor -- start 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.524+0000 7f253be00640 1 -- start start 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f253c076980 0x7f253c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c078560 0x7f253c1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f253c114040 con 0x7f253c077bc0 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f253c113ec0 con 0x7f253c078560 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1141c0 con 0x7f253c076980 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36870/0 (socket says 172.21.15.57:36870) 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253a000640 1 -- 172.21.15.57:0/1420029184 learned_addr learned my addr 172.21.15.57:0/1420029184 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253aa00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f253c076980 0x7f253c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.525+0000 7f253b400640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c078560 0x7f253c1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253a000640 1 -- 172.21.15.57:0/1420029184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f253c076980 msgr2=0x7f253c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253a000640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f253c076980 0x7f253c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253a000640 1 -- 172.21.15.57:0/1420029184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c078560 msgr2=0x7f253c1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253a000640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c078560 0x7f253c1a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253a000640 1 -- 172.21.15.57:0/1420029184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c1a6340 con 0x7f253c077bc0 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253b400640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c078560 0x7f253c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253a000640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c19ec60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f252802c700 tx=0x7f25280328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f2533400640 1 -- 172.21.15.57:0/1420029184 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528041020 con 0x7f253c077bc0 2024-04-13T05:18:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f2533400640 1 -- 172.21.15.57:0/1420029184 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2528003e20 con 0x7f253c077bc0 2024-04-13T05:18:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c1a65d0 con 0x7f253c077bc0 2024-04-13T05:18:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.526+0000 7f2533400640 1 -- 172.21.15.57:0/1420029184 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528039040 con 0x7f253c077bc0 2024-04-13T05:18:32.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.527+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c1a6b10 con 0x7f253c077bc0 2024-04-13T05:18:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 Processor -- start 2024-04-13T05:18:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 -- start start 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 0x56382acfc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382acff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56382ac5d020 con 0x56382acf7e30 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56382ac03950 con 0x56382acfcfa0 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f25445dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56382ac58fe0 con 0x56382acf7440 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.530+0000 7f2533400640 1 -- 172.21.15.57:0/1420029184 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2528044020 con 0x7f253c077bc0 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 0x56382acfc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 0x56382acfc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36874/0 (socket says 172.21.15.57:36874) 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 -- 172.21.15.57:0/4004627138 learned_addr learned my addr 172.21.15.57:0/4004627138 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253b400640 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253a000640 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382acff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 -- 172.21.15.57:0/4004627138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 msgr2=0x56382acf7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 -- 172.21.15.57:0/4004627138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 msgr2=0x56382acff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382acff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253aa00640 1 -- 172.21.15.57:0/4004627138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382a96f3f0 con 0x56382acf7e30 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.531+0000 7f253a000640 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382acff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.532+0000 7f253aa00640 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 0x56382acfc930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f252c00bcb0 tx=0x7f252c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee28933b9b1544e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.532+0000 7f2527400640 1 -- 172.21.15.57:0/4004627138 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c015070 con 0x56382acf7e30 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.532+0000 7f2527400640 1 -- 172.21.15.57:0/4004627138 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f252c0118f0 con 0x56382acf7e30 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.532+0000 7f2527400640 1 -- 172.21.15.57:0/4004627138 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c010890 con 0x56382acf7e30 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.532+0000 7f2533400640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f25200a3c30 0x7f25200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f2533400640 1 -- 172.21.15.57:0/1420029184 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f25280f2ae0 con 0x7f253c077bc0 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f253b400640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f25200a3c30 0x7f25200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 -- 172.21.15.57:0/4004627138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 msgr2=0x56382acfc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 0x56382acfc930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f252c00bcb0 tx=0x7f252c00f730 comp rx=0 tx=0).stop 2024-04-13T05:18:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 -- 172.21.15.57:0/4004627138 shutdown_connections 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382acff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7e30 0x56382acfc930 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 --2- 172.21.15.57:0/4004627138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 -- 172.21.15.57:0/4004627138 >> 172.21.15.57:0/4004627138 conn(0x56382acf29b0 msgr2=0x56382acf2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 -- 172.21.15.57:0/4004627138 shutdown_connections 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.533+0000 7f25445dd740 1 -- 172.21.15.57:0/4004627138 wait complete. 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f253b400640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f25200a3c30 0x7f25200a60f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2534005fd0 tx=0x7f2534009420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f25445dd740 1 -- start start 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f25445dd740 1 Processor -- start 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f25445dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 0x56382ad019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f25445dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad01ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f25445dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382ad08f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.534+0000 7f25445dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56382acfff50 con 0x56382acf7440 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f25445dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56382ac16c80 con 0x56382acfcfa0 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f25445dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56382ad000d0 con 0x56382acf7e30 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f25445dd740 10 client.0 _ll_register_callbacks cb 0x56382ac16670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 0x56382ad019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 0x56382ad019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36880/0 (socket says 172.21.15.57:36880) 2024-04-13T05:18:32.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 -- 172.21.15.57:0/3414985481 learned_addr learned my addr 172.21.15.57:0/3414985481 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253a000640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad01ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253b400640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382ad08f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 -- 172.21.15.57:0/3414985481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 msgr2=0x56382ad01ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad01ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 -- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 msgr2=0x56382ad08f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382ad08f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.535+0000 7f253aa00640 1 -- 172.21.15.57:0/3414985481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382aa397d0 con 0x56382acf7440 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f253aa00640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 0x56382ad019a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f252c000c00 tx=0x7f252c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44bd2ad6abc74c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f251be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f251be00640 1 -- 172.21.15.57:0/3414985481 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c029060 con 0x56382acf7440 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f251be00640 1 -- 172.21.15.57:0/3414985481 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f252c011aa0 con 0x56382acf7440 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252c0118f0 con 0x56382acf7440 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f251be00640 1 -- 172.21.15.57:0/3414985481 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f252c030050 con 0x56382acf7440 2024-04-13T05:18:32.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f25445dd740 10 client.15669 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.536+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56382ad0c2e0 con 0x56382acf7440 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f251be00640 1 -- 172.21.15.57:0/3414985481 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f252c0118f0 con 0x56382acf7440 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f251be00640 1 -- 172.21.15.57:0/3414985481 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f252c010980 con 0x56382acf7440 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f25445dd740 10 client.15669 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f25445dd740 10 client.15669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f25445dd740 20 client.15669 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f25445dd740 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad0d6e0 0x56382ad0fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f25445dd740 4 client.15669 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.537+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56382abfc750 con 0x56382ad0d6e0 2024-04-13T05:18:32.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.538+0000 7f253a000640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad0d6e0 0x56382ad0fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.539+0000 7f253a000640 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad0d6e0 0x56382ad0fad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c8e9f0efca06f58 server_cookie=f113d46ad571f15a in_seq=0 out_seq=0 2024-04-13T05:18:32.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.539+0000 7f251be00640 10 client.15669 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f251be00640 1 -- 172.21.15.57:0/3414985481 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56382abfc750 con 0x56382ad0d6e0 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f251be00640 10 client.15669 handle_command_reply: tid=0 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 client.15669 shutdown 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 msgr2=0x56382ad019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 0x56382ad019a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f252c000c00 tx=0x7f252c00b9f0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 shutdown_connections 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad0d6e0 0x56382ad0fad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acfcfa0 0x56382ad08f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad01ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 --2- 172.21.15.57:0/3414985481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acf7440 0x56382ad019a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f251be00640 0 client.15669 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:18:32.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 >> 172.21.15.57:0/3414985481 conn(0x56382acf29b0 msgr2=0x56382acf1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 shutdown_connections 2024-04-13T05:18:32.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.541+0000 7f25445dd740 1 -- 172.21.15.57:0/3414985481 wait complete. 2024-04-13T05:18:32.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.542+0000 7f25445dd740 20 client.15669 trim_cache size 0 max 16384 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.556+0000 7f25445dd740 1 Processor -- start 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 -- start start 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7e30 0x56382ad08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1b3870 con 0x56382acfcfa0 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1b1140 con 0x56382acf7e30 2024-04-13T05:18:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f25445dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1aef90 con 0x56382acf7440 2024-04-13T05:18:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f253b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f253b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad15330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36886/0 (socket says 172.21.15.57:36886) 2024-04-13T05:18:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f253b400640 1 -- 172.21.15.57:0/965301262 learned_addr learned my addr 172.21.15.57:0/965301262 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f253aa00640 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.557+0000 7f253a000640 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7e30 0x56382ad08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.558+0000 7f253b400640 1 -- 172.21.15.57:0/965301262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 msgr2=0x56382acf66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.558+0000 7f253b400640 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.558+0000 7f253b400640 1 -- 172.21.15.57:0/965301262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7e30 msgr2=0x56382ad08ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.558+0000 7f253b400640 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7e30 0x56382ad08ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.558+0000 7f253b400640 1 -- 172.21.15.57:0/965301262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382ad10100 con 0x56382acfcfa0 2024-04-13T05:18:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.558+0000 7f253b400640 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad15330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2534009fd0 tx=0x7f2534014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e813fdf5543067 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f2526a00640 1 -- 172.21.15.57:0/965301262 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534028670 con 0x56382acfcfa0 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f2526a00640 1 -- 172.21.15.57:0/965301262 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2534023070 con 0x56382acfcfa0 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f2526a00640 1 -- 172.21.15.57:0/965301262 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253401fba0 con 0x56382acfcfa0 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 -- 172.21.15.57:0/965301262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 msgr2=0x56382ad15330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad15330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2534009fd0 tx=0x7f2534014000 comp rx=0 tx=0).stop 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 -- 172.21.15.57:0/965301262 shutdown_connections 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7e30 0x56382ad08ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad15330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 --2- 172.21.15.57:0/965301262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7440 0x56382acf66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.559+0000 7f25445dd740 1 -- 172.21.15.57:0/965301262 >> 172.21.15.57:0/965301262 conn(0x56382acf29b0 msgr2=0x56382acf1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.560+0000 7f25445dd740 1 -- 172.21.15.57:0/965301262 shutdown_connections 2024-04-13T05:18:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.560+0000 7f25445dd740 1 -- 172.21.15.57:0/965301262 wait complete. 2024-04-13T05:18:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.560+0000 7f25445dd740 1 -- start start 2024-04-13T05:18:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.560+0000 7f25445dd740 1 Processor -- start 2024-04-13T05:18:32.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.560+0000 7f25445dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7440 0x56382ad086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f25445dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f25445dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad06d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f25445dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1af600 con 0x56382acfcfa0 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f25445dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1b0a70 con 0x56382acf7440 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f25445dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f253c1afcd0 con 0x56382acf7e30 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f25445dd740 10 client.0 _ll_register_callbacks cb 0x7f252c00bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad06d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad06d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36902/0 (socket says 172.21.15.57:36902) 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 -- 172.21.15.57:0/2814826169 learned_addr learned my addr 172.21.15.57:0/2814826169 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253a000640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253aa00640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7440 0x56382ad086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 -- 172.21.15.57:0/2814826169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 msgr2=0x56382ad08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 -- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7440 msgr2=0x56382ad086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7440 0x56382ad086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.561+0000 7f253b400640 1 -- 172.21.15.57:0/2814826169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382acf3420 con 0x56382acfcfa0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.562+0000 7f253a000640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.562+0000 7f253b400640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad06d00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2534005f30 tx=0x7f2534020b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ea9253b6b167da server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.562+0000 7f251be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.562+0000 7f251be00640 1 -- 172.21.15.57:0/2814826169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534005220 con 0x56382acfcfa0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.562+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2534023070 con 0x56382acfcfa0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.562+0000 7f251be00640 1 -- 172.21.15.57:0/2814826169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f25340053c0 con 0x56382acfcfa0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f251be00640 1 -- 172.21.15.57:0/2814826169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f253403d050 con 0x56382acfcfa0 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 10 client.15675 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56382acf3720 con 0x56382acfcfa0 2024-04-13T05:18:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f251be00640 1 -- 172.21.15.57:0/2814826169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2534023070 con 0x56382acfcfa0 2024-04-13T05:18:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f251be00640 1 -- 172.21.15.57:0/2814826169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2534031030 con 0x56382acfcfa0 2024-04-13T05:18:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 10 client.15675 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:18:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 20 client.15675 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad74a90 0x56382ad76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 4 client.15675 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.563+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56382ac6aa40 con 0x56382ad74a90 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.564+0000 7f253aa00640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad74a90 0x56382ad76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.565+0000 7f253aa00640 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad74a90 0x56382ad76eb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d12ea5fc566f978c server_cookie=ef1ef88ff829510a in_seq=0 out_seq=0 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.565+0000 7f251be00640 10 client.15675 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.565+0000 7f251be00640 1 -- 172.21.15.57:0/2814826169 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56382ac6aa40 con 0x56382ad74a90 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.565+0000 7f251be00640 10 client.15675 handle_command_reply: tid=0 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 client.15675 shutdown 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 msgr2=0x56382ad06d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad06d00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2534005f30 tx=0x7f2534020b80 comp rx=0 tx=0).stop 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 shutdown_connections 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56382ad74a90 0x56382ad76eb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56382acfcfa0 0x56382ad06d00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56382acf7e30 0x56382ad08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 --2- 172.21.15.57:0/2814826169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56382acf7440 0x56382ad086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 >> 172.21.15.57:0/2814826169 conn(0x56382acf29b0 msgr2=0x56382ad14320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f251be00640 0 client.15675 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:18:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 shutdown_connections 2024-04-13T05:18:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 1 -- 172.21.15.57:0/2814826169 wait complete. 2024-04-13T05:18:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.566+0000 7f25445dd740 20 client.15675 trim_cache size 0 max 16384 2024-04-13T05:18:32.570 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f25200a3c30 msgr2=0x7f25200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f25200a3c30 0x7f25200a60f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2534005fd0 tx=0x7f2534009420 comp rx=0 tx=0).stop 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 msgr2=0x7f253c19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c19ec60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f252802c700 tx=0x7f25280328a0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 shutdown_connections 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f25200a3c30 0x7f25200a60f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f253c078560 0x7f253c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f253c077bc0 0x7f253c19ec60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 --2- 172.21.15.57:0/1420029184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f253c076980 0x7f253c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 >> 172.21.15.57:0/1420029184 conn(0x7f253c0ff670 msgr2=0x7f253c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 shutdown_connections 2024-04-13T05:18:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.567+0000 7f253be00640 1 -- 172.21.15.57:0/1420029184 wait complete. 2024-04-13T05:18:32.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:18:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 -- 172.21.15.57:0/1429933949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad0075fa0 msgr2=0x7f4ad00743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 --2- 172.21.15.57:0/1429933949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad0075fa0 0x7f4ad00743f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009900 tx=0x7f4ac002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 -- 172.21.15.57:0/1429933949 shutdown_connections 2024-04-13T05:18:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 --2- 172.21.15.57:0/1429933949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad00749c0 0x7f4ad0074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 --2- 172.21.15.57:0/1429933949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ad0109d60 0x7f4ad010c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 --2- 172.21.15.57:0/1429933949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad0075fa0 0x7f4ad00743f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 -- 172.21.15.57:0/1429933949 >> 172.21.15.57:0/1429933949 conn(0x7f4ad00fd460 msgr2=0x7f4ad00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 -- 172.21.15.57:0/1429933949 shutdown_connections 2024-04-13T05:18:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.817+0000 7f4ad5800640 1 -- 172.21.15.57:0/1429933949 wait complete. 2024-04-13T05:18:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.818+0000 7f4ad5800640 1 Processor -- start 2024-04-13T05:18:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.818+0000 7f4ad5800640 1 -- start start 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 0x7f4ad019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ad0075fa0 0x7f4ad019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad0109d60 0x7f4ad01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad5800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010fc60 con 0x7f4ad00749c0 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad5800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010fae0 con 0x7f4ad0109d60 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad5800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad010fde0 con 0x7f4ad0075fa0 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 0x7f4ad019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 0x7f4ad019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36920/0 (socket says 172.21.15.57:36920) 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 -- 172.21.15.57:0/3593355833 learned_addr learned my addr 172.21.15.57:0/3593355833 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acf400640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ad0075fa0 0x7f4ad019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad0109d60 0x7f4ad01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 -- 172.21.15.57:0/3593355833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ad0075fa0 msgr2=0x7f4ad019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ad0075fa0 0x7f4ad019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 -- 172.21.15.57:0/3593355833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad0109d60 msgr2=0x7f4ad01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad0109d60 0x7f4ad01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.819+0000 7f4acfe00640 1 -- 172.21.15.57:0/3593355833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01a1ef0 con 0x7f4ad00749c0 2024-04-13T05:18:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad0109d60 0x7f4ad01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4acfe00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 0x7f4ad019a2d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0000c00 tx=0x7f4ac002dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4accc00640 1 -- 172.21.15.57:0/3593355833 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0043020 con 0x7f4ad00749c0 2024-04-13T05:18:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01a2180 con 0x7f4ad00749c0 2024-04-13T05:18:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4accc00640 1 -- 172.21.15.57:0/3593355833 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ac002de30 con 0x7f4ad00749c0 2024-04-13T05:18:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4accc00640 1 -- 172.21.15.57:0/3593355833 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0003c40 con 0x7f4ad00749c0 2024-04-13T05:18:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.820+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a2720 con 0x7f4ad00749c0 2024-04-13T05:18:32.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.823+0000 7f4ad9bf1740 1 Processor -- start 2024-04-13T05:18:32.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 -- start start 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4accc00640 1 -- 172.21.15.57:0/3593355833 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ac0046020 con 0x7f4ad00749c0 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019df8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019dfd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e00430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a019d5e020 con 0x55a019dfdfa0 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a019d04950 con 0x55a019df8440 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a019d59fe0 con 0x55a019df8e30 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4acf400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e00430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4acf400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e00430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36932/0 (socket says 172.21.15.57:36932) 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.824+0000 7f4acf400640 1 -- 172.21.15.57:0/2609239567 learned_addr learned my addr 172.21.15.57:0/2609239567 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019df8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acfe00640 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019dfd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acf400640 1 -- 172.21.15.57:0/2609239567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 msgr2=0x55a019dfd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acf400640 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019dfd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acf400640 1 -- 172.21.15.57:0/2609239567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 msgr2=0x55a019df8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acf400640 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019df8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acf400640 1 -- 172.21.15.57:0/2609239567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a019a703f0 con 0x55a019dfdfa0 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019df8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.825+0000 7f4acf400640 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e00430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00bce0 tx=0x7f4abc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2a758ff27012d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4ac5600640 1 -- 172.21.15.57:0/2609239567 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc015070 con 0x55a019dfdfa0 2024-04-13T05:18:32.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4ac5600640 1 -- 172.21.15.57:0/2609239567 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4abc01a910 con 0x55a019dfdfa0 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4ac5600640 1 -- 172.21.15.57:0/2609239567 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0119d0 con 0x55a019dfdfa0 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4accc00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ab40a3c30 0x7f4ab40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4accc00640 1 -- 172.21.15.57:0/3593355833 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4ac00f2ad0 con 0x7f4ad00749c0 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ab40a3c30 0x7f4ab40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/2609239567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 msgr2=0x55a019e00430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.826+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e00430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00bce0 tx=0x7f4abc00f730 comp rx=0 tx=0).stop 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/2609239567 shutdown_connections 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e00430 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019dfd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/2609239567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019df8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/2609239567 >> 172.21.15.57:0/2609239567 conn(0x55a019df39b0 msgr2=0x55a019df3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/2609239567 shutdown_connections 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/2609239567 wait complete. 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ab40a3c30 0x7f4ab40a60f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a019df9690 tx=0x7f4ac8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.827+0000 7f4ad9bf1740 1 -- start start 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 Processor -- start 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 0x55a019e029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e02f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019dfdfa0 0x55a019e09ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a019e00f50 con 0x55a019df8440 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a019d17c80 con 0x55a019dfdfa0 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a019e010d0 con 0x55a019df8e30 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4ad9bf1740 10 client.0 _ll_register_callbacks cb 0x55a019d17670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4acfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 0x55a019e029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.828+0000 7f4acfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 0x55a019e029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36936/0 (socket says 172.21.15.57:36936) 2024-04-13T05:18:32.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 -- 172.21.15.57:0/3092909076 learned_addr learned my addr 172.21.15.57:0/3092909076 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019dfdfa0 0x55a019e09ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acf400640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e02f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 -- 172.21.15.57:0/3092909076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 msgr2=0x55a019e02f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e02f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 -- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019dfdfa0 msgr2=0x55a019e09ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019dfdfa0 0x55a019e09ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 -- 172.21.15.57:0/3092909076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a019b3a7d0 con 0x55a019df8440 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019dfdfa0 0x55a019e09ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acf400640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4acfe00640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 0x55a019e029e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ac002c740 tx=0x7f4ac0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca01fd70e947755d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4aba000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:32.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4aba000640 1 -- 172.21.15.57:0/3092909076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00be550 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.829+0000 7f4aba000640 1 -- 172.21.15.57:0/3092909076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4ac00bcd10 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc01a910 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4aba000640 1 -- 172.21.15.57:0/3092909076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ac00be370 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 10 client.15687 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a019e00f50 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4aba000640 1 -- 172.21.15.57:0/3092909076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4abc01a910 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4aba000640 1 -- 172.21.15.57:0/3092909076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4ac00ddbd0 con 0x55a019df8440 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 10 client.15687 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 20 client.15687 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e0e9e0 0x55a019e10e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 4 client.15687 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:32.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.830+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a019cfd750 con 0x55a019e0e9e0 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.831+0000 7f4acf400640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e0e9e0 0x55a019e10e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.832+0000 7f4acf400640 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e0e9e0 0x55a019e10e50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b833d7f3c8360c server_cookie=ac77873b04c30627 in_seq=0 out_seq=0 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.832+0000 7f4aba000640 10 client.15687 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4aba000640 1 -- 172.21.15.57:0/3092909076 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a019cfd750 con 0x55a019e0e9e0 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4aba000640 10 client.15687 handle_command_reply: tid=0 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 client.15687 shutdown 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 msgr2=0x55a019e029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 0x55a019e029e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ac002c740 tx=0x7f4ac0005e50 comp rx=0 tx=0).stop 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 shutdown_connections 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e0e9e0 0x55a019e10e50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019dfdfa0 0x55a019e09ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/3092909076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019df8440 0x55a019e029e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4aba000640 0 client.15687 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.834+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 >> 172.21.15.57:0/3092909076 conn(0x55a019df39b0 msgr2=0x55a019df2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.835+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 shutdown_connections 2024-04-13T05:18:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.835+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/3092909076 wait complete. 2024-04-13T05:18:32.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.835+0000 7f4ad9bf1740 20 client.15687 trim_cache size 0 max 16384 2024-04-13T05:18:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 Processor -- start 2024-04-13T05:18:32.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 -- start start 2024-04-13T05:18:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e02b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01af3f0 con 0x55a019dfdfa0 2024-04-13T05:18:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01accc0 con 0x55a019df8440 2024-04-13T05:18:32.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.850+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01aab10 con 0x55a019df8e30 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e02b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e02b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36946/0 (socket says 172.21.15.57:36946) 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1865092173 learned_addr learned my addr 172.21.15.57:0/1865092173 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4acf400640 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4acfe00640 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1865092173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 msgr2=0x55a019e09eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e09eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1865092173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 msgr2=0x55a019e025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1865092173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a019df9210 con 0x55a019dfdfa0 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.851+0000 7f4acfe00640 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.852+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e02b30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8012770 tx=0x7f4ac8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9832dad0061a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.852+0000 7f4ac4c00640 1 -- 172.21.15.57:0/1865092173 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8027430 con 0x55a019dfdfa0 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.852+0000 7f4ac4c00640 1 -- 172.21.15.57:0/1865092173 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ac801e8c0 con 0x55a019dfdfa0 2024-04-13T05:18:32.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.852+0000 7f4ac4c00640 1 -- 172.21.15.57:0/1865092173 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8026aa0 con 0x55a019dfdfa0 2024-04-13T05:18:32.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1865092173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 msgr2=0x55a019e02b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e02b30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8012770 tx=0x7f4ac8014000 comp rx=0 tx=0).stop 2024-04-13T05:18:32.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1865092173 shutdown_connections 2024-04-13T05:18:32.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e02b30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1865092173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1865092173 >> 172.21.15.57:0/1865092173 conn(0x55a019df39b0 msgr2=0x55a019df3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1865092173 shutdown_connections 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1865092173 wait complete. 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 -- start start 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.853+0000 7f4ad9bf1740 1 Processor -- start 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e07290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ab180 con 0x55a019dfdfa0 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ac5f0 con 0x55a019df8440 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ab850 con 0x55a019df8e30 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad9bf1740 10 client.0 _ll_register_callbacks cb 0x7f4ac00044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:32.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36958/0 (socket says 172.21.15.57:36958) 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1700568699 learned_addr learned my addr 172.21.15.57:0/1700568699 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.854+0000 7f4acfe00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e07290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4acf400640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1700568699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 msgr2=0x55a019e077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 msgr2=0x55a019e07290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e07290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad4e00640 1 -- 172.21.15.57:0/1700568699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a019e08dd0 con 0x55a019dfdfa0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4acfe00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e07290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad4e00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e087e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009e40 tx=0x7f4ac8012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6863618807a47ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4aba000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4aba000640 1 -- 172.21.15.57:0/1700568699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8034020 con 0x55a019dfdfa0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4aba000640 1 -- 172.21.15.57:0/1700568699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4ac801e610 con 0x55a019dfdfa0 2024-04-13T05:18:32.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.855+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac801e8c0 con 0x55a019dfdfa0 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4aba000640 1 -- 172.21.15.57:0/1700568699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ac803f050 con 0x55a019dfdfa0 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 10 client.15693 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a019e0a040 con 0x55a019dfdfa0 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4aba000640 1 -- 172.21.15.57:0/1700568699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ac801e8c0 con 0x55a019dfdfa0 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4aba000640 1 -- 172.21.15.57:0/1700568699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4ac802f030 con 0x55a019dfdfa0 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 10 client.15693 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 20 client.15693 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e755d0 0x55a019e779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 4 client.15693 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:18:32.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.856+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a019d6ba40 con 0x55a019e755d0 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.857+0000 7f4acfe00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e755d0 0x55a019e779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4acfe00640 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e755d0 0x55a019e779f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86794f1ef292e3a server_cookie=108a199ef3081794 in_seq=0 out_seq=0 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4aba000640 10 client.15693 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4aba000640 1 -- 172.21.15.57:0/1700568699 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a019d6ba40 con 0x55a019e755d0 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4aba000640 10 client.15693 handle_command_reply: tid=0 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4ad9bf1740 1 client.15693 shutdown 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 msgr2=0x55a019e087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.858+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e087e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009e40 tx=0x7f4ac8012450 comp rx=0 tx=0).stop 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 shutdown_connections 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a019e755d0 0x55a019e779f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a019dfdfa0 0x55a019e087e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a019df8e30 0x55a019e077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 --2- 172.21.15.57:0/1700568699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a019df8440 0x55a019e07290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4aba000640 0 client.15693 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:32.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 >> 172.21.15.57:0/1700568699 conn(0x55a019df39b0 msgr2=0x55a019e025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 shutdown_connections 2024-04-13T05:18:32.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 1 -- 172.21.15.57:0/1700568699 wait complete. 2024-04-13T05:18:32.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.859+0000 7f4ad9bf1740 20 client.15693 trim_cache size 0 max 16384 2024-04-13T05:18:32.864 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ab40a3c30 msgr2=0x7f4ab40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ab40a3c30 0x7f4ab40a60f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55a019df9690 tx=0x7f4ac8009550 comp rx=0 tx=0).stop 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 msgr2=0x7f4ad019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 0x7f4ad019a2d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0000c00 tx=0x7f4ac002dc40 comp rx=0 tx=0).stop 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 shutdown_connections 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ab40a3c30 0x7f4ab40a60f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ad0109d60 0x7f4ad01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ad0075fa0 0x7f4ad019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 --2- 172.21.15.57:0/3593355833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad00749c0 0x7f4ad019a2d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 >> 172.21.15.57:0/3593355833 conn(0x7f4ad00fd460 msgr2=0x7f4ad0105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 shutdown_connections 2024-04-13T05:18:32.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:32.860+0000 7f4ad5800640 1 -- 172.21.15.57:0/3593355833 wait complete. 2024-04-13T05:18:32.877 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:32 smithi057 ceph-mon[32331]: pgmap v484: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-13T05:18:32.877 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:32 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1934269422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:18:33.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:32 smithi189 ceph-mon[35531]: pgmap v484: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-13T05:18:33.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:32 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1934269422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:18:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:32 smithi083 ceph-mon[35133]: pgmap v484: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-13T05:18:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:32 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1934269422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:18:33.882 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:18:33.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:18:33.883 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:18:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 -- 172.21.15.57:0/3859038620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 msgr2=0x7f5eac078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 --2- 172.21.15.57:0/3859038620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac078e00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009ca0 tx=0x7f5ea402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 -- 172.21.15.57:0/3859038620 shutdown_connections 2024-04-13T05:18:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 --2- 172.21.15.57:0/3859038620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac078e00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 --2- 172.21.15.57:0/3859038620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eac077bc0 0x7f5eac078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 --2- 172.21.15.57:0/3859038620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 0x7f5eac076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 -- 172.21.15.57:0/3859038620 >> 172.21.15.57:0/3859038620 conn(0x7f5eac0ff770 msgr2=0x7f5eac101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 -- 172.21.15.57:0/3859038620 shutdown_connections 2024-04-13T05:18:34.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.122+0000 7f5eb2000640 1 -- 172.21.15.57:0/3859038620 wait complete. 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.123+0000 7f5eb2000640 1 Processor -- start 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.123+0000 7f5eb2000640 1 -- start start 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 0x7f5eac19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eac077bc0 0x7f5eac19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac1a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eb2000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac114160 con 0x7f5eac078560 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eb2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac113fe0 con 0x7f5eac076980 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eb2000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1142e0 con 0x7f5eac077bc0 2024-04-13T05:18:34.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac1a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac1a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36984/0 (socket says 172.21.15.57:36984) 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 -- 172.21.15.57:0/1061064299 learned_addr learned my addr 172.21.15.57:0/1061064299 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eac077bc0 0x7f5eac19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eab400640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 0x7f5eac19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 -- 172.21.15.57:0/1061064299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eac077bc0 msgr2=0x7f5eac19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eac077bc0 0x7f5eac19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 -- 172.21.15.57:0/1061064299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 msgr2=0x7f5eac19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.124+0000 7f5eabe00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 0x7f5eac19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.125+0000 7f5eabe00640 1 -- 172.21.15.57:0/1061064299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac1a6430 con 0x7f5eac078560 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.125+0000 7f5eab400640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 0x7f5eac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:18:34.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.125+0000 7f5eabe00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac1a5dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4032380 tx=0x7f5ea4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.125+0000 7f5ea3e00640 1 -- 172.21.15.57:0/1061064299 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4041020 con 0x7f5eac078560 2024-04-13T05:18:34.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.126+0000 7f5ea3e00640 1 -- 172.21.15.57:0/1061064299 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ea4003e20 con 0x7f5eac078560 2024-04-13T05:18:34.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.126+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac1a66c0 con 0x7f5eac078560 2024-04-13T05:18:34.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.126+0000 7f5ea3e00640 1 -- 172.21.15.57:0/1061064299 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4039040 con 0x7f5eac078560 2024-04-13T05:18:34.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.126+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac1a6bb0 con 0x7f5eac078560 2024-04-13T05:18:34.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.129+0000 7f5eb4f70740 1 Processor -- start 2024-04-13T05:18:34.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5ea3e00640 1 -- 172.21.15.57:0/1061064299 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5ea4044020 con 0x7f5eac078560 2024-04-13T05:18:34.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 -- start start 2024-04-13T05:18:34.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c5668e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56693770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c56696270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563c565f4bc0 con 0x563c5668e280 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c5659b660 con 0x563c56693de0 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c565f0b80 con 0x563c5668ec70 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.130+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c5668e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c5668e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36994/0 (socket says 172.21.15.57:36994) 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 -- 172.21.15.57:0/2574458167 learned_addr learned my addr 172.21.15.57:0/2574458167 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eab400640 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56693770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5ea3e00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e900a3bb0 0x7f5e900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c56696270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5ea3e00640 1 -- 172.21.15.57:0/1061064299 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5ea40f2b30 con 0x7f5eac078560 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 -- 172.21.15.57:0/2574458167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 msgr2=0x563c56693770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56693770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 -- 172.21.15.57:0/2574458167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 msgr2=0x563c56696270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c56696270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eaaa00640 1 -- 172.21.15.57:0/2574458167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c565be220 con 0x563c5668e280 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.131+0000 7f5eabe00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e900a3bb0 0x7f5e900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.132+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c5668e6a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e98013580 tx=0x7f5e9800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1db43bc9d358e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.132+0000 7f5e97e00640 1 -- 172.21.15.57:0/2574458167 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98019030 con 0x563c5668e280 2024-04-13T05:18:34.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.132+0000 7f5e97e00640 1 -- 172.21.15.57:0/2574458167 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e980118f0 con 0x563c5668e280 2024-04-13T05:18:34.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.132+0000 7f5e97e00640 1 -- 172.21.15.57:0/2574458167 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800e6d0 con 0x563c5668e280 2024-04-13T05:18:34.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2574458167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 msgr2=0x563c5668e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c5668e6a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e98013580 tx=0x7f5e9800b6a0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eabe00640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e900a3bb0 0x7f5e900a6070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40baed0 tx=0x7f5ea40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2574458167 shutdown_connections 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c56696270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56693770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2574458167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c5668e6a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2574458167 >> 172.21.15.57:0/2574458167 conn(0x563c566897f0 msgr2=0x563c56689bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2574458167 shutdown_connections 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.133+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2574458167 wait complete. 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 -- start start 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 Processor -- start 2024-04-13T05:18:34.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c56698780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56698cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 0x563c5669fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563c56696de0 con 0x563c56693de0 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c565aea30 con 0x563c5668e280 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c56696f60 con 0x563c5668ec70 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.134+0000 7f5eb4f70740 10 client.0 _ll_register_callbacks cb 0x563c56306c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 0x563c5669fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 0x563c5669fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36998/0 (socket says 172.21.15.57:36998) 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 -- 172.21.15.57:0/376228717 learned_addr learned my addr 172.21.15.57:0/376228717 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c56698780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eab400640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56698cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 -- 172.21.15.57:0/376228717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 msgr2=0x563c56698cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56698cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 -- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 msgr2=0x563c56698780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c56698780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 -- 172.21.15.57:0/376228717 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c563cf7e0 con 0x563c56693de0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 0x563c5669fd00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005c40 tx=0x7f5e98015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf2869158f8b016 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.135+0000 7f5e94c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5e94c00640 1 -- 172.21.15.57:0/376228717 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98019070 con 0x563c56693de0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5e94c00640 1 -- 172.21.15.57:0/376228717 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5e98011640 con 0x563c56693de0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e980118f0 con 0x563c56693de0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5e94c00640 1 -- 172.21.15.57:0/376228717 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e98011aa0 con 0x563c56693de0 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5eb4f70740 10 client.15705 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:34.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c5642ff70 con 0x563c56693de0 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5e94c00640 1 -- 172.21.15.57:0/376228717 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e980118f0 con 0x563c56693de0 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5e94c00640 1 -- 172.21.15.57:0/376228717 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e9800e9d0 con 0x563c56693de0 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.136+0000 7f5eb4f70740 10 client.15705 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.137+0000 7f5eb4f70740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.137+0000 7f5eb4f70740 20 client.15705 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.137+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c566a4230 0x563c566a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.137+0000 7f5eb4f70740 4 client.15705 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.137+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c566025e0 con 0x563c566a4230 2024-04-13T05:18:34.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.137+0000 7f5eab400640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c566a4230 0x563c566a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.138+0000 7f5eab400640 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c566a4230 0x563c566a6620 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa00c312a814d50 server_cookie=3bcab13001438881 in_seq=0 out_seq=0 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.138+0000 7f5e94c00640 10 client.15705 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.140+0000 7f5e94c00640 1 -- 172.21.15.57:0/376228717 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c566025e0 con 0x563c566a4230 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.140+0000 7f5e94c00640 10 client.15705 handle_command_reply: tid=0 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.140+0000 7f5eb4f70740 1 client.15705 shutdown 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.140+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 msgr2=0x563c5669fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.140+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 0x563c5669fd00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005c40 tx=0x7f5e98015040 comp rx=0 tx=0).stop 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 shutdown_connections 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c566a4230 0x563c566a6620 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c56693de0 0x563c5669fd00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c56698cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/376228717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c56698780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 >> 172.21.15.57:0/376228717 conn(0x563c566897f0 msgr2=0x563c566889f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5e94c00640 0 client.15705 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 shutdown_connections 2024-04-13T05:18:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 1 -- 172.21.15.57:0/376228717 wait complete. 2024-04-13T05:18:34.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.141+0000 7f5eb4f70740 20 client.15705 trim_cache size 0 max 16384 2024-04-13T05:18:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.161+0000 7f5eb4f70740 1 Processor -- start 2024-04-13T05:18:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 -- start start 2024-04-13T05:18:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c56688160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c5669fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c566886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1b3a80 con 0x563c5668e280 2024-04-13T05:18:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1b1350 con 0x563c56693de0 2024-04-13T05:18:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1af1a0 con 0x563c5668ec70 2024-04-13T05:18:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c56688160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c56688160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37006/0 (socket says 172.21.15.57:37006) 2024-04-13T05:18:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eaaa00640 1 -- 172.21.15.57:0/894638111 learned_addr learned my addr 172.21.15.57:0/894638111 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.162+0000 7f5eab400640 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c5669fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5eaaa00640 1 -- 172.21.15.57:0/894638111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 msgr2=0x563c5669fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c5669fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5eaaa00640 1 -- 172.21.15.57:0/894638111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 msgr2=0x563c566886a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c566886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5eaaa00640 1 -- 172.21.15.57:0/894638111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c562c7170 con 0x563c5668e280 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c56688160 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800ed90 tx=0x7f5e9800bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e60ea8e200b7bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5e97400640 1 -- 172.21.15.57:0/894638111 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98019040 con 0x563c5668e280 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5e97400640 1 -- 172.21.15.57:0/894638111 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e9800b620 con 0x563c5668e280 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.163+0000 7f5e97400640 1 -- 172.21.15.57:0/894638111 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011800 con 0x563c5668e280 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 -- 172.21.15.57:0/894638111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 msgr2=0x563c56688160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c56688160 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800ed90 tx=0x7f5e9800bb60 comp rx=0 tx=0).stop 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 -- 172.21.15.57:0/894638111 shutdown_connections 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c56693de0 0x563c566886a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5668ec70 0x563c5669fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/894638111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668e280 0x563c56688160 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 -- 172.21.15.57:0/894638111 >> 172.21.15.57:0/894638111 conn(0x563c566897f0 msgr2=0x563c566a6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 -- 172.21.15.57:0/894638111 shutdown_connections 2024-04-13T05:18:34.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.164+0000 7f5eb4f70740 1 -- 172.21.15.57:0/894638111 wait complete. 2024-04-13T05:18:34.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 -- start start 2024-04-13T05:18:34.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 Processor -- start 2024-04-13T05:18:34.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c5669d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 0x563c5669daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c56693de0 0x563c5669ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1af810 con 0x563c5668ec70 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1b0c80 con 0x563c5668e280 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.165+0000 7f5eb4f70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1afee0 con 0x563c56693de0 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eb4f70740 10 client.0 _ll_register_callbacks cb 0x7f5e98013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 0x563c5669daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 0x563c5669daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37020/0 (socket says 172.21.15.57:37020) 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 -- 172.21.15.57:0/2812479883 learned_addr learned my addr 172.21.15.57:0/2812479883 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c5669d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 -- 172.21.15.57:0/2812479883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c56693de0 msgr2=0x563c5669ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c56693de0 0x563c5669ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 -- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 msgr2=0x563c5669d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c5669d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 -- 172.21.15.57:0/2812479883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c565afbe0 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5eab400640 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 0x563c5669daa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00b370 tx=0x7f5e9c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b05aaa015f485cd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.166+0000 7f5e94c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5e94c00640 1 -- 172.21.15.57:0/2812479883 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c004030 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5e94c00640 1 -- 172.21.15.57:0/2812479883 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5e9c0041d0 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9800b620 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5e94c00640 1 -- 172.21.15.57:0/2812479883 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c02b050 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5eb4f70740 10 client.15711 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c5670a740 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5e94c00640 1 -- 172.21.15.57:0/2812479883 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e9800b620 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.167+0000 7f5e94c00640 1 -- 172.21.15.57:0/2812479883 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5e9c0037a0 con 0x563c5668ec70 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eb4f70740 10 client.15711 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eb4f70740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eb4f70740 20 client.15711 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:34.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c5670b130 0x563c5670d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eb4f70740 4 client.15711 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:18:34.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c565fc450 con 0x563c5670b130 2024-04-13T05:18:34.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.168+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c5670b130 0x563c5670d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.169+0000 7f5eaaa00640 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c5670b130 0x563c5670d550 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2975f059d615f432 server_cookie=d511efee44f9d17d in_seq=0 out_seq=0 2024-04-13T05:18:34.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.169+0000 7f5e94c00640 10 client.15711 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5e94c00640 1 -- 172.21.15.57:0/2812479883 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c565fc450 con 0x563c5670b130 2024-04-13T05:18:34.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5e94c00640 10 client.15711 handle_command_reply: tid=0 2024-04-13T05:18:34.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 client.15711 shutdown 2024-04-13T05:18:34.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 msgr2=0x563c5669daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 0x563c5669daa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00b370 tx=0x7f5e9c01bf90 comp rx=0 tx=0).stop 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 shutdown_connections 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563c5670b130 0x563c5670d550 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c56693de0 0x563c5669ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563c5668ec70 0x563c5669daa0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 --2- 172.21.15.57:0/2812479883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5668e280 0x563c5669d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5e94c00640 0 client.15711 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 >> 172.21.15.57:0/2812479883 conn(0x563c566897f0 msgr2=0x563c56688160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 shutdown_connections 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 1 -- 172.21.15.57:0/2812479883 wait complete. 2024-04-13T05:18:34.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.172+0000 7f5eb4f70740 20 client.15711 trim_cache size 0 max 16384 2024-04-13T05:18:34.176 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:18:34.176 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:18:34.176 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "e86b0b5d-43c6-4e3c-bbc9-eb7e8dac93d0", 2024-04-13T05:18:34.176 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:18:34.176 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e900a3bb0 msgr2=0x7f5e900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e900a3bb0 0x7f5e900a6070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40baed0 tx=0x7f5ea40095c0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 msgr2=0x7f5eac1a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac1a5dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4032380 tx=0x7f5ea4032820 comp rx=0 tx=0).stop 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 shutdown_connections 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e900a3bb0 0x7f5e900a6070 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eac078560 0x7f5eac1a5dc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eac077bc0 0x7f5eac19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1061064299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac076980 0x7f5eac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 >> 172.21.15.57:0/1061064299 conn(0x7f5eac0ff770 msgr2=0x7f5eac100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 shutdown_connections 2024-04-13T05:18:34.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.173+0000 7f5eb2000640 1 -- 172.21.15.57:0/1061064299 wait complete. 2024-04-13T05:18:34.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:18:34.193 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:18:34.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 -- 172.21.15.57:0/2031234343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 msgr2=0x7f13fc104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 --2- 172.21.15.57:0/2031234343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc104520 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009900 tx=0x7f13f002bc30 comp rx=0 tx=0).stop 2024-04-13T05:18:34.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 -- 172.21.15.57:0/2031234343 shutdown_connections 2024-04-13T05:18:34.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 --2- 172.21.15.57:0/2031234343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc105cb0 0x7f13fc10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 --2- 172.21.15.57:0/2031234343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13fc1052f0 0x7f13fc105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 --2- 172.21.15.57:0/2031234343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc104520 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 -- 172.21.15.57:0/2031234343 >> 172.21.15.57:0/2031234343 conn(0x7f13fc0ff670 msgr2=0x7f13fc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 -- 172.21.15.57:0/2031234343 shutdown_connections 2024-04-13T05:18:34.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.414+0000 7f1402200640 1 -- 172.21.15.57:0/2031234343 wait complete. 2024-04-13T05:18:34.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.415+0000 7f1402200640 1 Processor -- start 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.415+0000 7f1402200640 1 -- start start 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1402200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1402200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc1052f0 0x7f13fc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1402200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13fc105cb0 0x7f13fc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1402200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10fc90 con 0x7f13fc104100 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1402200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10fb10 con 0x7f13fc1052f0 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1402200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10fe10 con 0x7f13fc105cb0 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37038/0 (socket says 172.21.15.57:37038) 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 -- 172.21.15.57:0/1995690216 learned_addr learned my addr 172.21.15.57:0/1995690216 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f13fbe00640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc1052f0 0x7f13fc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 -- 172.21.15.57:0/1995690216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13fc105cb0 msgr2=0x7f13fc1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1401800640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13fc105cb0 0x7f13fc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13fc105cb0 0x7f13fc1a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 -- 172.21.15.57:0/1995690216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc1052f0 msgr2=0x7f13fc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc1052f0 0x7f13fc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.416+0000 7f1400e00640 1 -- 172.21.15.57:0/1995690216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc1a1fc0 con 0x7f13fc104100 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f13fbe00640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc1052f0 0x7f13fc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f1400e00640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc19a3a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f13f0000c00 tx=0x7f13f00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f13f9600640 1 -- 172.21.15.57:0/1995690216 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0043020 con 0x7f13fc104100 2024-04-13T05:18:34.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc1a2250 con 0x7f13fc104100 2024-04-13T05:18:34.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f13f9600640 1 -- 172.21.15.57:0/1995690216 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f00044c0 con 0x7f13fc104100 2024-04-13T05:18:34.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f13f9600640 1 -- 172.21.15.57:0/1995690216 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0003c60 con 0x7f13fc104100 2024-04-13T05:18:34.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.417+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc1a27a0 con 0x7f13fc104100 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.420+0000 7f1406788740 1 Processor -- start 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 -- start start 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e53b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e540e0 0x564fe0e58be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0e5b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x564fe0dba020 con 0x564fe0e59250 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564fe0d606d0 con 0x564fe0e536f0 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f1406788740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564fe0db5fe0 con 0x564fe0e540e0 2024-04-13T05:18:34.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f13f9600640 1 -- 172.21.15.57:0/1995690216 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f13f0046020 con 0x7f13fc104100 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0e5b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0e5b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37054/0 (socket says 172.21.15.57:37054) 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.421+0000 7f13fbe00640 1 -- 172.21.15.57:0/4064859060 learned_addr learned my addr 172.21.15.57:0/4064859060 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f1401800640 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e53b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13fbe00640 1 -- 172.21.15.57:0/4064859060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e540e0 msgr2=0x564fe0e58be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13fbe00640 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e540e0 0x564fe0e58be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13fbe00640 1 -- 172.21.15.57:0/4064859060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 msgr2=0x564fe0e53b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13fbe00640 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e53b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13fbe00640 1 -- 172.21.15.57:0/4064859060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe0b967d0 con 0x564fe0e59250 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13fbe00640 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0e5b6e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009c00 tx=0x7f13e8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ae3f708cd8b93d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13ee000640 1 -- 172.21.15.57:0/4064859060 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004030 con 0x564fe0e59250 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13ee000640 1 -- 172.21.15.57:0/4064859060 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13e80035f0 con 0x564fe0e59250 2024-04-13T05:18:34.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.422+0000 7f13ee000640 1 -- 172.21.15.57:0/4064859060 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8012690 con 0x564fe0e59250 2024-04-13T05:18:34.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f13f9600640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f13e40a3bb0 0x7f13e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f13f9600640 1 -- 172.21.15.57:0/1995690216 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f13f00f2a90 con 0x7f13fc104100 2024-04-13T05:18:34.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1406788740 1 -- 172.21.15.57:0/4064859060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 msgr2=0x564fe0e5b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1406788740 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0e5b6e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009c00 tx=0x7f13e8010740 comp rx=0 tx=0).stop 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1401800640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f13e40a3bb0 0x7f13e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1406788740 1 -- 172.21.15.57:0/4064859060 shutdown_connections 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1406788740 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0e5b6e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1406788740 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e540e0 0x564fe0e58be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.423+0000 7f1406788740 1 --2- 172.21.15.57:0/4064859060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e53b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.424+0000 7f1406788740 1 -- 172.21.15.57:0/4064859060 >> 172.21.15.57:0/4064859060 conn(0x564fe0e4ec60 msgr2=0x564fe0e4f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.424+0000 7f1406788740 1 -- 172.21.15.57:0/4064859060 shutdown_connections 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.424+0000 7f1406788740 1 -- 172.21.15.57:0/4064859060 wait complete. 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.424+0000 7f1406788740 1 -- start start 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.424+0000 7f1401800640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f13e40a3bb0 0x7f13e40a6070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x564fe0e54940 tx=0x7f13f4009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.424+0000 7f1406788740 1 Processor -- start 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e5da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e5df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x564fe0e5c200 con 0x564fe0e540e0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564fe0d73a90 con 0x564fe0e536f0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564fe0e5c380 con 0x564fe0e59250 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1406788740 10 client.0 _ll_register_callbacks cb 0x564fe0d73480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e5df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e5df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37060/0 (socket says 172.21.15.57:37060) 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f13fbe00640 1 -- 172.21.15.57:0/948938537 learned_addr learned my addr 172.21.15.57:0/948938537 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.425+0000 7f1400e00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e5da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f1401800640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13fbe00640 1 -- 172.21.15.57:0/948938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 msgr2=0x564fe0e64fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13fbe00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13fbe00640 1 -- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 msgr2=0x564fe0e5da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13fbe00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e5da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13fbe00640 1 -- 172.21.15.57:0/948938537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe0acc3f0 con 0x564fe0e540e0 2024-04-13T05:18:34.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f1400e00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e5da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f1401800640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13fbe00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e5df70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009f20 tx=0x7f13e8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85637f45f8e5f09a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13e2a00640 1 -- 172.21.15.57:0/948938537 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e802b060 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f13e2a00640 1 -- 172.21.15.57:0/948938537 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f13e8003fe0 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.426+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80035f0 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f13e2a00640 1 -- 172.21.15.57:0/948938537 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13e8032050 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 10 client.15723 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564fe0a8c0f0 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f13e2a00640 1 -- 172.21.15.57:0/948938537 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13e80035f0 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f13e2a00640 1 -- 172.21.15.57:0/948938537 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f13e800f040 con 0x564fe0e540e0 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 10 client.15723 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:34.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 20 client.15723 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:34.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0e695b0 0x564fe0e6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 4 client.15723 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:18:34.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.427+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe0d59600 con 0x564fe0e695b0 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.428+0000 7f1400e00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0e695b0 0x564fe0e6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.429+0000 7f1400e00640 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0e695b0 0x564fe0e6b9a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d5409b71b75624 server_cookie=617bb2ba7a38c5a6 in_seq=0 out_seq=0 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.429+0000 7f13e2a00640 10 client.15723 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f13e2a00640 1 -- 172.21.15.57:0/948938537 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fe0d59600 con 0x564fe0e695b0 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f13e2a00640 10 client.15723 handle_command_reply: tid=0 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 client.15723 shutdown 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 msgr2=0x564fe0e5df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e5df70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009f20 tx=0x7f13e8009f50 comp rx=0 tx=0).stop 2024-04-13T05:18:34.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 shutdown_connections 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0e695b0 0x564fe0e6b9a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e5df70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 --2- 172.21.15.57:0/948938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e5da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f13e2a00640 0 client.15723 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 >> 172.21.15.57:0/948938537 conn(0x564fe0e4ec60 msgr2=0x564fe0e4de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 shutdown_connections 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 1 -- 172.21.15.57:0/948938537 wait complete. 2024-04-13T05:18:34.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.431+0000 7f1406788740 20 client.15723 trim_cache size 0 max 16384 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 Processor -- start 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 -- start start 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1af470 con 0x564fe0e540e0 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1acd40 con 0x564fe0e536f0 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f1406788740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1aab90 con 0x564fe0e59250 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37076/0 (socket says 172.21.15.57:37076) 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.451+0000 7f13fbe00640 1 -- 172.21.15.57:0/3562865702 learned_addr learned my addr 172.21.15.57:0/3562865702 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13fbe00640 1 -- 172.21.15.57:0/3562865702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 msgr2=0x564fe0e64f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13fbe00640 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13fbe00640 1 -- 172.21.15.57:0/3562865702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 msgr2=0x564fe0e554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13fbe00640 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13fbe00640 1 -- 172.21.15.57:0/3562865702 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe0e54970 con 0x564fe0e540e0 2024-04-13T05:18:34.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13fbe00640 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e559e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009b20 tx=0x7f13e8003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130df0eba4507bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13ed600640 1 -- 172.21.15.57:0/3562865702 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e802b040 con 0x564fe0e540e0 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13ed600640 1 -- 172.21.15.57:0/3562865702 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13e8005520 con 0x564fe0e540e0 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.452+0000 7f13ed600640 1 -- 172.21.15.57:0/3562865702 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8012450 con 0x564fe0e540e0 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 -- 172.21.15.57:0/3562865702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 msgr2=0x564fe0e559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e559e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009b20 tx=0x7f13e8003e60 comp rx=0 tx=0).stop 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 -- 172.21.15.57:0/3562865702 shutdown_connections 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e59250 0x564fe0e64f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e540e0 0x564fe0e559e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 --2- 172.21.15.57:0/3562865702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e536f0 0x564fe0e554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 -- 172.21.15.57:0/3562865702 >> 172.21.15.57:0/3562865702 conn(0x564fe0e4ec60 msgr2=0x564fe0e49d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 -- 172.21.15.57:0/3562865702 shutdown_connections 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.453+0000 7f1406788740 1 -- 172.21.15.57:0/3562865702 wait complete. 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 -- start start 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 Processor -- start 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e536f0 0x564fe0e61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e540e0 0x564fe0e621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0ed62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1ab200 con 0x564fe0e59250 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1ac670 con 0x564fe0e540e0 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc1ab8d0 con 0x564fe0e536f0 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1406788740 10 client.0 _ll_register_callbacks cb 0x7f13e8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:18:34.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.454+0000 7f1401800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0ed62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0ed62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37086/0 (socket says 172.21.15.57:37086) 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 -- 172.21.15.57:0/3548718156 learned_addr learned my addr 172.21.15.57:0/3548718156 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1400e00640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e536f0 0x564fe0e61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f13fbe00640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e540e0 0x564fe0e621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 -- 172.21.15.57:0/3548718156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e536f0 msgr2=0x564fe0e61c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e536f0 0x564fe0e61c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 -- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e540e0 msgr2=0x564fe0e621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e540e0 0x564fe0e621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 -- 172.21.15.57:0/3548718156 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe0e62840 con 0x564fe0e59250 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1400e00640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e536f0 0x564fe0e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f13fbe00640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e540e0 0x564fe0e621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f1401800640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0ed62c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f13f4012680 tx=0x7f13f4012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b104e9e4b2f8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.455+0000 7f13e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:18:34.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f13e2a00640 1 -- 172.21.15.57:0/3548718156 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40119f0 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f13e2a00640 1 -- 172.21.15.57:0/3548718156 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f13f4011b90 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e8005520 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f13e2a00640 1 -- 172.21.15.57:0/3548718156 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13f4030050 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f1406788740 10 client.15729 fetch_fsmap learned FSMap version 25 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564fe0ed7a10 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f13e2a00640 1 -- 172.21.15.57:0/3548718156 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13e8005520 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.456+0000 7f13e2a00640 1 -- 172.21.15.57:0/3548718156 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f13f400f040 con 0x564fe0e59250 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1406788740 10 client.15729 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1406788740 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1406788740 20 client.15729 populate_metadata read hostname 'smithi057' 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1406788740 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0ed8420 0x564fe0eda840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1406788740 4 client.15729 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fe0dc7a40 con 0x564fe0ed8420 2024-04-13T05:18:34.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.457+0000 7f1400e00640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0ed8420 0x564fe0eda840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.458+0000 7f1400e00640 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0ed8420 0x564fe0eda840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c6cc0eb1e56eef server_cookie=6cb526bd1f31a162 in_seq=0 out_seq=0 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.458+0000 7f13e2a00640 10 client.15729 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.458+0000 7f13e2a00640 1 -- 172.21.15.57:0/3548718156 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564fe0dc7a40 con 0x564fe0ed8420 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.458+0000 7f13e2a00640 10 client.15729 handle_command_reply: tid=0 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 client.15729 shutdown 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 msgr2=0x564fe0ed62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0ed62c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f13f4012680 tx=0x7f13f4012a80 comp rx=0 tx=0).stop 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 shutdown_connections 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x564fe0ed8420 0x564fe0eda840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fe0e540e0 0x564fe0e621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564fe0e59250 0x564fe0ed62c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 --2- 172.21.15.57:0/3548718156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fe0e536f0 0x564fe0e61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f13e2a00640 0 client.15729 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 >> 172.21.15.57:0/3548718156 conn(0x564fe0e4ec60 msgr2=0x564fe0e51ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 shutdown_connections 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 1 -- 172.21.15.57:0/3548718156 wait complete. 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.459+0000 7f1406788740 20 client.15729 trim_cache size 0 max 16384 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout: "e86b0b5d-43c6-4e3c-bbc9-eb7e8dac93d0": { 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "e86b0b5d-43c6-4e3c-bbc9-eb7e8dac93d0", 2024-04-13T05:18:34.465 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:18:34.466 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:18:34.466 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:18:34.466 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:18:34.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f13e40a3bb0 msgr2=0x7f13e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f13e40a3bb0 0x7f13e40a6070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x564fe0e54940 tx=0x7f13f4009520 comp rx=0 tx=0).stop 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 msgr2=0x7f13fc19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc19a3a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f13f0000c00 tx=0x7f13f00040c0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 shutdown_connections 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f13e40a3bb0 0x7f13e40a6070 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13fc1052f0 0x7f13fc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13fc105cb0 0x7f13fc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 --2- 172.21.15.57:0/1995690216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13fc104100 0x7f13fc19a3a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 >> 172.21.15.57:0/1995690216 conn(0x7f13fc0ff670 msgr2=0x7f13fc108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 shutdown_connections 2024-04-13T05:18:34.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:18:34.460+0000 7f1402200640 1 -- 172.21.15.57:0/1995690216 wait complete. 2024-04-13T05:18:34.479 INFO:tasks.cephfs.filesystem:scrub status for tag:e86b0b5d-43c6-4e3c-bbc9-eb7e8dac93d0 - {'path': '/', 'tag': 'e86b0b5d-43c6-4e3c-bbc9-eb7e8dac93d0', 'options': 'recursive,force'} 2024-04-13T05:18:35.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:34 smithi057 ceph-mon[32331]: pgmap v485: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-13T05:18:35.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:34 smithi189 ceph-mon[35531]: pgmap v485: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-13T05:18:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:34 smithi083 ceph-mon[35133]: pgmap v485: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-13T05:18:36.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:35 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:18:36.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:35 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:36.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:35 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:18:36.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:35 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:35 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:18:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:35 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:35 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:18:36.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:35 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:36.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:35 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:18:36.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:35 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:36.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:35 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:18:36.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:35 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:18:37.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:36 smithi189 ceph-mon[35531]: pgmap v486: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-13T05:18:37.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:36 smithi083 ceph-mon[35133]: pgmap v486: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-13T05:18:37.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:36 smithi057 ceph-mon[32331]: pgmap v486: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-13T05:18:38.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:37 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:18:38.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:37 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:18:38.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:37 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:18:39.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:38 smithi189 ceph-mon[35531]: pgmap v487: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-13T05:18:39.153 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:38 smithi083 ceph-mon[35133]: pgmap v487: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-13T05:18:39.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:38 smithi057 ceph-mon[32331]: pgmap v487: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-13T05:18:41.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:40 smithi189 ceph-mon[35531]: pgmap v488: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-13T05:18:41.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:40 smithi083 ceph-mon[35133]: pgmap v488: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-13T05:18:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:40 smithi057 ceph-mon[32331]: pgmap v488: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-13T05:18:43.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:42 smithi189 ceph-mon[35531]: pgmap v489: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:18:43.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:42 smithi083 ceph-mon[35133]: pgmap v489: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:18:43.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:42 smithi057 ceph-mon[32331]: pgmap v489: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:18:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:44 smithi083 ceph-mon[35133]: pgmap v490: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:18:45.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:44 smithi057 ceph-mon[32331]: pgmap v490: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:18:45.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:44 smithi189 ceph-mon[35531]: pgmap v490: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:18:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:46 smithi083 ceph-mon[35133]: pgmap v491: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-13T05:18:47.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:46 smithi057 ceph-mon[32331]: pgmap v491: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-13T05:18:47.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:46 smithi189 ceph-mon[35531]: pgmap v491: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-13T05:18:49.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:48 smithi083 ceph-mon[35133]: pgmap v492: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:49.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:48 smithi057 ceph-mon[32331]: pgmap v492: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:48 smithi189 ceph-mon[35531]: pgmap v492: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:51.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:50 smithi083 ceph-mon[35133]: pgmap v493: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:18:51.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:50 smithi057 ceph-mon[32331]: pgmap v493: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:18:51.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:50 smithi189 ceph-mon[35531]: pgmap v493: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:18:52.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:51 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:18:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:51 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:18:52.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:51 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:18:53.184 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:52 smithi057 ceph-mon[32331]: pgmap v494: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:53.184 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:18:53.184 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:18:53.184 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:52 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:18:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:52 smithi083 ceph-mon[35133]: pgmap v494: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:18:53.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:18:53.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:52 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:18:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:52 smithi189 ceph-mon[35531]: pgmap v494: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:18:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:18:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:52 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:18:55.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:54 smithi083 ceph-mon[35133]: pgmap v495: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:18:55.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:54 smithi057 ceph-mon[32331]: pgmap v495: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:18:55.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:54 smithi189 ceph-mon[35531]: pgmap v495: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:18:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:56 smithi083 ceph-mon[35133]: pgmap v496: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:18:57.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:56 smithi057 ceph-mon[32331]: pgmap v496: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:18:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:56 smithi189 ceph-mon[35531]: pgmap v496: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:18:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:18:58 smithi083 ceph-mon[35133]: pgmap v497: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:59.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:18:58 smithi057 ceph-mon[32331]: pgmap v497: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:18:59.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:18:58 smithi189 ceph-mon[35531]: pgmap v497: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:01.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:00 smithi083 ceph-mon[35133]: pgmap v498: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:19:01.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:00 smithi057 ceph-mon[32331]: pgmap v498: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:19:01.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:00 smithi189 ceph-mon[35531]: pgmap v498: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:19:03.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:02 smithi083 ceph-mon[35133]: pgmap v499: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:02 smithi057 ceph-mon[32331]: pgmap v499: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:03.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:02 smithi189 ceph-mon[35531]: pgmap v499: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:04.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:19:04.481 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.686+0000 7fa81aa00640 1 -- 172.21.15.57:0/2289789078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814109da0 msgr2=0x7fa81410c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.686+0000 7fa81aa00640 1 --2- 172.21.15.57:0/2289789078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814109da0 0x7fa81410c230 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009ca0 tx=0x7fa80c02c180 comp rx=0 tx=0).stop 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 -- 172.21.15.57:0/2289789078 shutdown_connections 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 --2- 172.21.15.57:0/2289789078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa814109da0 0x7fa81410c230 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 --2- 172.21.15.57:0/2289789078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8140749a0 0x7fa814074e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 --2- 172.21.15.57:0/2289789078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa814075f80 0x7fa8140743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 -- 172.21.15.57:0/2289789078 >> 172.21.15.57:0/2289789078 conn(0x7fa8140fd460 msgr2=0x7fa8140ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 -- 172.21.15.57:0/2289789078 shutdown_connections 2024-04-13T05:19:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.687+0000 7fa81aa00640 1 -- 172.21.15.57:0/2289789078 wait complete. 2024-04-13T05:19:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.688+0000 7fa81aa00640 1 Processor -- start 2024-04-13T05:19:04.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.688+0000 7fa81aa00640 1 -- start start 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa81aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 0x7fa81419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa81aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa814075f80 0x7fa81419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa81aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa814109da0 0x7fa8141a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa81aa00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa81410fc40 con 0x7fa8140749a0 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa81aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa81410fac0 con 0x7fa814075f80 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa81aa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa81410fdc0 con 0x7fa814109da0 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa819600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 0x7fa81419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa819600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 0x7fa81419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34658/0 (socket says 172.21.15.57:34658) 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.689+0000 7fa819600640 1 -- 172.21.15.57:0/1967494197 learned_addr learned my addr 172.21.15.57:0/1967494197 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa818c00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa814075f80 0x7fa81419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa819600640 1 -- 172.21.15.57:0/1967494197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa814109da0 msgr2=0x7fa8141a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa819600640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa814109da0 0x7fa8141a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa819600640 1 -- 172.21.15.57:0/1967494197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa814075f80 msgr2=0x7fa81419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa819600640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa814075f80 0x7fa81419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa819600640 1 -- 172.21.15.57:0/1967494197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8141a1e50 con 0x7fa8140749a0 2024-04-13T05:19:04.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.690+0000 7fa819600640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 0x7fa81419a230 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d950 tx=0x7fa80800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.691+0000 7fa812000640 1 -- 172.21.15.57:0/1967494197 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808017070 con 0x7fa8140749a0 2024-04-13T05:19:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.691+0000 7fa812000640 1 -- 172.21.15.57:0/1967494197 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa808009c30 con 0x7fa8140749a0 2024-04-13T05:19:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.691+0000 7fa812000640 1 -- 172.21.15.57:0/1967494197 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80800f620 con 0x7fa8140749a0 2024-04-13T05:19:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.691+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8141a2140 con 0x7fa8140749a0 2024-04-13T05:19:04.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.692+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8141a2640 con 0x7fa8140749a0 2024-04-13T05:19:04.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 Processor -- start 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 -- start start 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 0x558c78bbfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bc26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558c78b21020 con 0x558c78bc0250 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558c78ac76d0 con 0x558c78bba6f0 2024-04-13T05:19:04.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.695+0000 7fa81efbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c78b1cfe0 con 0x558c78bbb0e0 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.696+0000 7fa818c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bc26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.696+0000 7fa818c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bc26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34674/0 (socket says 172.21.15.57:34674) 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.696+0000 7fa818c00640 1 -- 172.21.15.57:0/1047969487 learned_addr learned my addr 172.21.15.57:0/1047969487 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.696+0000 7fa81a000640 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa818c00640 1 -- 172.21.15.57:0/1047969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 msgr2=0x558c78bbfbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa818c00640 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 0x558c78bbfbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa812000640 1 -- 172.21.15.57:0/1967494197 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa8080109a0 con 0x7fa8140749a0 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa818c00640 1 -- 172.21.15.57:0/1047969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 msgr2=0x558c78bbab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa818c00640 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa818c00640 1 -- 172.21.15.57:0/1047969487 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c788fd7d0 con 0x558c78bc0250 2024-04-13T05:19:04.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.697+0000 7fa818c00640 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bc26e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa80000bc80 tx=0x7fa80000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57923fe8a6fc14c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.698+0000 7fa806600640 1 -- 172.21.15.57:0/1047969487 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800015070 con 0x558c78bc0250 2024-04-13T05:19:04.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.698+0000 7fa806600640 1 -- 172.21.15.57:0/1047969487 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa80001a910 con 0x558c78bc0250 2024-04-13T05:19:04.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.698+0000 7fa806600640 1 -- 172.21.15.57:0/1047969487 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000118d0 con 0x558c78bc0250 2024-04-13T05:19:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 -- 172.21.15.57:0/1047969487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 msgr2=0x558c78bc26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bc26e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa80000bc80 tx=0x7fa80000f730 comp rx=0 tx=0).stop 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 -- 172.21.15.57:0/1047969487 shutdown_connections 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bc26e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 0x558c78bbfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 --2- 172.21.15.57:0/1047969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 -- 172.21.15.57:0/1047969487 >> 172.21.15.57:0/1047969487 conn(0x558c78bb5c60 msgr2=0x558c78bb6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 -- 172.21.15.57:0/1047969487 shutdown_connections 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa81efbd740 1 -- 172.21.15.57:0/1047969487 wait complete. 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.699+0000 7fa812000640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7fc0a3b40 0x7fa7fc0a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.700+0000 7fa812000640 1 -- 172.21.15.57:0/1967494197 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa8080cfe40 con 0x7fa8140749a0 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.700+0000 7fa81efbd740 1 -- start start 2024-04-13T05:19:04.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.700+0000 7fa818c00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7fc0a3b40 0x7fa7fc0a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.700+0000 7fa81efbd740 1 Processor -- start 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bcd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 0x558c78bcd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bd4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558c78bc3200 con 0x558c78bc0250 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558c78adaa90 con 0x558c78bba6f0 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c78bc3380 con 0x558c78bbb0e0 2024-04-13T05:19:04.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81efbd740 10 client.0 _ll_register_callbacks cb 0x558c78ada480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bd4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bd4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34676/0 (socket says 172.21.15.57:34676) 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa81a000640 1 -- 172.21.15.57:0/3621372662 learned_addr learned my addr 172.21.15.57:0/3621372662 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.701+0000 7fa819600640 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bcd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81a000640 1 -- 172.21.15.57:0/3621372662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 msgr2=0x558c78bcd790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81a000640 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 0x558c78bcd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81a000640 1 -- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 msgr2=0x558c78bcd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81a000640 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bcd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81a000640 1 -- 172.21.15.57:0/3621372662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c788333f0 con 0x558c78bc0250 2024-04-13T05:19:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa818c00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7fc0a3b40 0x7fa7fc0a6000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa800000c00 tx=0x7fa80000be90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81a000640 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bd4760 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009ca0 tx=0x7fa80c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45d86217eafb6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa7faa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:04.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa7faa00640 1 -- 172.21.15.57:0/3621372662 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c043070 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa7faa00640 1 -- 172.21.15.57:0/3621372662 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa80c0328f0 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.702+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80001a910 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa7faa00640 1 -- 172.21.15.57:0/3621372662 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa80c04f050 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa81efbd740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c787f30f0 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa7faa00640 1 -- 172.21.15.57:0/3621372662 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa80001a910 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa7faa00640 1 -- 172.21.15.57:0/3621372662 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa80c0046e0 con 0x558c78bc0250 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa81efbd740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.703+0000 7fa81efbd740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.704+0000 7fa81efbd740 20 client.15741 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.704+0000 7fa81efbd740 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78bd9050 0x558c78bdb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.704+0000 7fa81efbd740 4 client.15741 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:04.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.704+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c78ac0600 con 0x558c78bd9050 2024-04-13T05:19:04.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.704+0000 7fa819600640 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78bd9050 0x558c78bdb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.705+0000 7fa819600640 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78bd9050 0x558c78bdb4f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f261b95a1222d60 server_cookie=309edd2b5919e63f in_seq=0 out_seq=0 2024-04-13T05:19:04.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.705+0000 7fa7faa00640 10 client.15741 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa7faa00640 1 -- 172.21.15.57:0/3621372662 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c78ac0600 con 0x558c78bd9050 2024-04-13T05:19:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa7faa00640 10 client.15741 handle_command_reply: tid=0 2024-04-13T05:19:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 client.15741 shutdown 2024-04-13T05:19:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 msgr2=0x558c78bd4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.709 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bd4760 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009ca0 tx=0x7fa80c009670 comp rx=0 tx=0).stop 2024-04-13T05:19:04.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 shutdown_connections 2024-04-13T05:19:04.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78bd9050 0x558c78bdb4f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bc0250 0x558c78bd4760 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bbb0e0 0x558c78bcd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 --2- 172.21.15.57:0/3621372662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bcd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 >> 172.21.15.57:0/3621372662 conn(0x558c78bb5c60 msgr2=0x558c78bb4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa7faa00640 0 client.15741 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:04.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.707+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 shutdown_connections 2024-04-13T05:19:04.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.708+0000 7fa81efbd740 1 -- 172.21.15.57:0/3621372662 wait complete. 2024-04-13T05:19:04.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.708+0000 7fa81efbd740 20 client.15741 trim_cache size 0 max 16384 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 Processor -- start 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 -- start start 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bbc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78bd4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141aafa0 con 0x558c78bbb0e0 2024-04-13T05:19:04.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a8870 con 0x558c78bba6f0 2024-04-13T05:19:04.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa81efbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a66c0 con 0x558c78bc0250 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa818c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bbc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa818c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bbc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34684/0 (socket says 172.21.15.57:34684) 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.727+0000 7fa818c00640 1 -- 172.21.15.57:0/2081147896 learned_addr learned my addr 172.21.15.57:0/2081147896 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa819600640 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa81a000640 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78bd4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa818c00640 1 -- 172.21.15.57:0/2081147896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 msgr2=0x558c78bd4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa818c00640 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78bd4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa818c00640 1 -- 172.21.15.57:0/2081147896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 msgr2=0x558c78bbc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa818c00640 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.728+0000 7fa818c00640 1 -- 172.21.15.57:0/2081147896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c78bb69d0 con 0x558c78bbb0e0 2024-04-13T05:19:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.729+0000 7fa818c00640 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bbc9e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa800005280 tx=0x7fa800022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2204a397b50e767c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.729+0000 7fa805c00640 1 -- 172.21.15.57:0/2081147896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800004d00 con 0x558c78bbb0e0 2024-04-13T05:19:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.729+0000 7fa805c00640 1 -- 172.21.15.57:0/2081147896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa80001a910 con 0x558c78bbb0e0 2024-04-13T05:19:04.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.729+0000 7fa805c00640 1 -- 172.21.15.57:0/2081147896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000116a0 con 0x558c78bbb0e0 2024-04-13T05:19:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 -- 172.21.15.57:0/2081147896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 msgr2=0x558c78bbc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bbc9e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa800005280 tx=0x7fa800022000 comp rx=0 tx=0).stop 2024-04-13T05:19:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 -- 172.21.15.57:0/2081147896 shutdown_connections 2024-04-13T05:19:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78bd4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bbc9e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2081147896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bbc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 -- 172.21.15.57:0/2081147896 >> 172.21.15.57:0/2081147896 conn(0x558c78bb5c60 msgr2=0x558c78bb6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 -- 172.21.15.57:0/2081147896 shutdown_connections 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 -- 172.21.15.57:0/2081147896 wait complete. 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.730+0000 7fa81efbd740 1 -- start start 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 Processor -- start 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bd1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bd1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78c3d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a6d30 con 0x558c78bbb0e0 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a81a0 con 0x558c78bba6f0 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8141a7400 con 0x558c78bc0250 2024-04-13T05:19:04.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa81efbd740 10 client.0 _ll_register_callbacks cb 0x7fa80c003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:04.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.731+0000 7fa818c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bd1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bd1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34690/0 (socket says 172.21.15.57:34690) 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 -- 172.21.15.57:0/2059358598 learned_addr learned my addr 172.21.15.57:0/2059358598 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa81a000640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78c3d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa819600640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bd1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 -- 172.21.15.57:0/2059358598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 msgr2=0x558c78c3d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78c3d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 -- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 msgr2=0x558c78bd1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bd1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa818c00640 1 -- 172.21.15.57:0/2059358598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c78c3db60 con 0x558c78bbb0e0 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.732+0000 7fa819600640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bd1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:04.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.733+0000 7fa818c00640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bd1ba0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa8000059f0 tx=0x7fa80001a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dd2aab6ac574b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.733+0000 7fa7faa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.733+0000 7fa7faa00640 1 -- 172.21.15.57:0/2059358598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80003e060 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.733+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80001a910 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.733+0000 7fa7faa00640 1 -- 172.21.15.57:0/2059358598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa80001acd0 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa7faa00640 1 -- 172.21.15.57:0/2059358598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa800045050 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa81efbd740 10 client.15747 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c78c3ed80 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa7faa00640 1 -- 172.21.15.57:0/2059358598 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa80001a910 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa7faa00640 1 -- 172.21.15.57:0/2059358598 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa800023070 con 0x558c78bbb0e0 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa81efbd740 10 client.15747 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa81efbd740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:04.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.734+0000 7fa81efbd740 20 client.15747 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:04.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.735+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78c3f4c0 0x558c78c418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.735+0000 7fa81efbd740 4 client.15747 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:19:04.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.735+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c78b2ea40 con 0x558c78c3f4c0 2024-04-13T05:19:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.735+0000 7fa819600640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78c3f4c0 0x558c78c418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.736+0000 7fa819600640 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78c3f4c0 0x558c78c418e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1d92fbc0c3e2d3 server_cookie=83e15b5c70e7110f in_seq=0 out_seq=0 2024-04-13T05:19:04.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.736+0000 7fa7faa00640 10 client.15747 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa7faa00640 1 -- 172.21.15.57:0/2059358598 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558c78b2ea40 con 0x558c78c3f4c0 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa7faa00640 10 client.15747 handle_command_reply: tid=0 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 client.15747 shutdown 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 msgr2=0x558c78bd1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bd1ba0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa8000059f0 tx=0x7fa80001a670 comp rx=0 tx=0).stop 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 shutdown_connections 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558c78c3f4c0 0x558c78c418e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c78bc0250 0x558c78c3d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558c78bbb0e0 0x558c78bd1ba0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 --2- 172.21.15.57:0/2059358598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c78bba6f0 0x558c78bd1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 >> 172.21.15.57:0/2059358598 conn(0x558c78bb5c60 msgr2=0x558c78bbc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.737+0000 7fa7faa00640 0 client.15747 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.738+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 shutdown_connections 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.738+0000 7fa81efbd740 1 -- 172.21.15.57:0/2059358598 wait complete. 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.738+0000 7fa81efbd740 20 client.15747 trim_cache size 0 max 16384 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:19:04.742 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:19:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7fc0a3b40 msgr2=0x7fa7fc0a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7fc0a3b40 0x7fa7fc0a6000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa800000c00 tx=0x7fa80000be90 comp rx=0 tx=0).stop 2024-04-13T05:19:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 msgr2=0x7fa81419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 0x7fa81419a230 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d950 tx=0x7fa80800de10 comp rx=0 tx=0).stop 2024-04-13T05:19:04.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 shutdown_connections 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa7fc0a3b40 0x7fa7fc0a6000 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa814075f80 0x7fa81419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa814109da0 0x7fa8141a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 --2- 172.21.15.57:0/1967494197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8140749a0 0x7fa81419a230 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 >> 172.21.15.57:0/1967494197 conn(0x7fa8140fd460 msgr2=0x7fa814105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 shutdown_connections 2024-04-13T05:19:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.739+0000 7fa81aa00640 1 -- 172.21.15.57:0/1967494197 wait complete. 2024-04-13T05:19:04.766 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:19:04.766 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:19:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.949+0000 7fdd5f200640 1 -- 172.21.15.57:0/3922841302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601006b0 msgr2=0x7fdd60102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.949+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3922841302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601006b0 0x7fdd60102ae0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009900 tx=0x7fdd5002bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 -- 172.21.15.57:0/3922841302 shutdown_connections 2024-04-13T05:19:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3922841302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd60103020 0x7fdd60105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3922841302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601059d0 0x7fdd60105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3922841302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601006b0 0x7fdd60102ae0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 -- 172.21.15.57:0/3922841302 >> 172.21.15.57:0/3922841302 conn(0x7fdd600fc290 msgr2=0x7fdd600fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 -- 172.21.15.57:0/3922841302 shutdown_connections 2024-04-13T05:19:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.950+0000 7fdd5f200640 1 -- 172.21.15.57:0/3922841302 wait complete. 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 Processor -- start 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 -- start start 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601006b0 0x7fdd6019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd60103020 0x7fdd6019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 0x7fdd601a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60113fb0 con 0x7fdd601059d0 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60113e30 con 0x7fdd60103020 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.951+0000 7fdd5f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60114130 con 0x7fdd601006b0 2024-04-13T05:19:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.952+0000 7fdd5e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 0x7fdd601a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.952+0000 7fdd5e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 0x7fdd601a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55872/0 (socket says 172.21.15.57:55872) 2024-04-13T05:19:04.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.952+0000 7fdd5e800640 1 -- 172.21.15.57:0/3973767689 learned_addr learned my addr 172.21.15.57:0/3973767689 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:04.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.952+0000 7fdd5d400640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd60103020 0x7fdd6019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.952+0000 7fdd5de00640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601006b0 0x7fdd6019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.953+0000 7fdd5e800640 1 -- 172.21.15.57:0/3973767689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601006b0 msgr2=0x7fdd6019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.953+0000 7fdd5e800640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601006b0 0x7fdd6019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.953+0000 7fdd5e800640 1 -- 172.21.15.57:0/3973767689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd60103020 msgr2=0x7fdd6019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.953+0000 7fdd5e800640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd60103020 0x7fdd6019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.953+0000 7fdd5e800640 1 -- 172.21.15.57:0/3973767689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd601a6260 con 0x7fdd601059d0 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.953+0000 7fdd5e800640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 0x7fdd601a5bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800eaa0 tx=0x7fdd5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.954+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800cdb0 con 0x7fdd601059d0 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.954+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd58005020 con 0x7fdd601059d0 2024-04-13T05:19:04.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.954+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580116d0 con 0x7fdd601059d0 2024-04-13T05:19:04.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.954+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd601a6550 con 0x7fdd601059d0 2024-04-13T05:19:04.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.954+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd601a6aa0 con 0x7fdd601059d0 2024-04-13T05:19:04.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.956+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd600fdd80 con 0x7fdd601059d0 2024-04-13T05:19:04.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.959+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd580051d0 con 0x7fdd601059d0 2024-04-13T05:19:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.960+0000 7fdd56a00640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd440a3c30 0x7fdd440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.960+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdd580d0640 con 0x7fdd601059d0 2024-04-13T05:19:04.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.961+0000 7fdd5de00640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd440a3c30 0x7fdd440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:04.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.961+0000 7fdd5de00640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd440a3c30 0x7fdd440a60f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd5002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:04.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:04.969+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd58016030 con 0x7fdd601059d0 2024-04-13T05:19:05.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:04 smithi083 ceph-mon[35133]: pgmap v500: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:19:05.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:04 smithi057 ceph-mon[32331]: pgmap v500: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:19:05.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:04 smithi189 ceph-mon[35531]: pgmap v500: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:19:05.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.457+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd60062bc0 con 0x7fdd601059d0 2024-04-13T05:19:05.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.458+0000 7fdd56a00640 1 -- 172.21.15.57:0/3973767689 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7fdd5809cc80 con 0x7fdd601059d0 2024-04-13T05:19:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:19:05.461 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:17:49.577177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:19:05.461 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-04-13T05:19:05.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd440a3c30 msgr2=0x7fdd440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd440a3c30 0x7fdd440a60f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd5002c7e0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 msgr2=0x7fdd601a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 0x7fdd601a5bf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800eaa0 tx=0x7fdd5800ef60 comp rx=0 tx=0).stop 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 shutdown_connections 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdd440a3c30 0x7fdd440a60f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd60103020 0x7fdd6019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd601059d0 0x7fdd601a5bf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 --2- 172.21.15.57:0/3973767689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd601006b0 0x7fdd6019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 >> 172.21.15.57:0/3973767689 conn(0x7fdd600fc290 msgr2=0x7fdd60103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 shutdown_connections 2024-04-13T05:19:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.461+0000 7fdd5f200640 1 -- 172.21.15.57:0/3973767689 wait complete. 2024-04-13T05:19:05.489 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 -- 172.21.15.57:0/942698525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c109da0 msgr2=0x7fb54c10c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 --2- 172.21.15.57:0/942698525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c109da0 0x7fb54c10c230 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb544009ca0 tx=0x7fb54402c180 comp rx=0 tx=0).stop 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 -- 172.21.15.57:0/942698525 shutdown_connections 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 --2- 172.21.15.57:0/942698525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c0749a0 0x7fb54c074e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 --2- 172.21.15.57:0/942698525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c109da0 0x7fb54c10c230 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 --2- 172.21.15.57:0/942698525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c075f80 0x7fb54c0743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 -- 172.21.15.57:0/942698525 >> 172.21.15.57:0/942698525 conn(0x7fb54c0fd460 msgr2=0x7fb54c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 -- 172.21.15.57:0/942698525 shutdown_connections 2024-04-13T05:19:05.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.680+0000 7fb54be00640 1 -- 172.21.15.57:0/942698525 wait complete. 2024-04-13T05:19:05.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.681+0000 7fb54be00640 1 Processor -- start 2024-04-13T05:19:05.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.681+0000 7fb54be00640 1 -- start start 2024-04-13T05:19:05.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 0x7fb54c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c075f80 0x7fb54c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c109da0 0x7fb54c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10fc40 con 0x7fb54c0749a0 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10fac0 con 0x7fb54c075f80 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c10fdc0 con 0x7fb54c109da0 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 0x7fb54c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 0x7fb54c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55882/0 (socket says 172.21.15.57:55882) 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54aa00640 1 -- 172.21.15.57:0/294061545 learned_addr learned my addr 172.21.15.57:0/294061545 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.682+0000 7fb54a000640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c075f80 0x7fb54c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54b400640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c109da0 0x7fb54c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54aa00640 1 -- 172.21.15.57:0/294061545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c109da0 msgr2=0x7fb54c1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54aa00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c109da0 0x7fb54c1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54aa00640 1 -- 172.21.15.57:0/294061545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c075f80 msgr2=0x7fb54c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54aa00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c075f80 0x7fb54c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54aa00640 1 -- 172.21.15.57:0/294061545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c1a1e90 con 0x7fb54c0749a0 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54b400640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c109da0 0x7fb54c1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.683+0000 7fb54aa00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 0x7fb54c19a270 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d770 tx=0x7fb53c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.684+0000 7fb550a00640 1 -- 172.21.15.57:0/294061545 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c017070 con 0x7fb54c0749a0 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.684+0000 7fb550a00640 1 -- 172.21.15.57:0/294061545 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb53c0054d0 con 0x7fb54c0749a0 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.684+0000 7fb550a00640 1 -- 172.21.15.57:0/294061545 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004c80 con 0x7fb54c0749a0 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.684+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c1a2180 con 0x7fb54c0749a0 2024-04-13T05:19:05.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.684+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a2680 con 0x7fb54c0749a0 2024-04-13T05:19:05.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.688+0000 7fb550a00640 1 -- 172.21.15.57:0/294061545 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb53c004e20 con 0x7fb54c0749a0 2024-04-13T05:19:05.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.689+0000 7fb554de4740 1 Processor -- start 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.689+0000 7fb554de4740 1 -- start start 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb554de4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080cad8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb554de4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080cadd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb554de4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080cae0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb554de4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56080ca3e020 con 0x56080cad8440 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb554de4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56080c9e4950 con 0x56080caddfa0 2024-04-13T05:19:05.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb554de4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56080ca39fe0 con 0x56080cad8e30 2024-04-13T05:19:05.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb54b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080cad8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.690+0000 7fb550a00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5300a3cb0 0x7fb5300a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080cad8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55890/0 (socket says 172.21.15.57:55890) 2024-04-13T05:19:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 -- 172.21.15.57:0/2653885430 learned_addr learned my addr 172.21.15.57:0/2653885430 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54aa00640 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080cadd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54a000640 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080cae0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb550a00640 1 -- 172.21.15.57:0/294061545 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb53c0d0170 con 0x7fb54c0749a0 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5300a3cb0 0x7fb5300a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 -- 172.21.15.57:0/2653885430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 msgr2=0x56080cadd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080cadd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 -- 172.21.15.57:0/2653885430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 msgr2=0x56080cae0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080cae0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.691+0000 7fb54b400640 1 -- 172.21.15.57:0/2653885430 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080c7503f0 con 0x56080cad8440 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.692+0000 7fb54b400640 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080cad8860 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb54402c790 tx=0x7fb544003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21dd0777e4ed10f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.692+0000 7fb537e00640 1 -- 172.21.15.57:0/2653885430 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544039040 con 0x56080cad8440 2024-04-13T05:19:05.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.692+0000 7fb537e00640 1 -- 172.21.15.57:0/2653885430 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5440044f0 con 0x56080cad8440 2024-04-13T05:19:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.692+0000 7fb537e00640 1 -- 172.21.15.57:0/2653885430 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440348a0 con 0x56080cad8440 2024-04-13T05:19:05.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.692+0000 7fb54b400640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5300a3cb0 0x7fb5300a6170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb544046000 tx=0x7fb544048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.693+0000 7fb554de4740 1 -- 172.21.15.57:0/2653885430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 msgr2=0x56080cad8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.693+0000 7fb554de4740 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080cad8860 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb54402c790 tx=0x7fb544003dc0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 -- 172.21.15.57:0/2653885430 shutdown_connections 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080cae0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080cadd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 --2- 172.21.15.57:0/2653885430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080cad8860 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 -- 172.21.15.57:0/2653885430 >> 172.21.15.57:0/2653885430 conn(0x56080cad39b0 msgr2=0x56080cad3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 -- 172.21.15.57:0/2653885430 shutdown_connections 2024-04-13T05:19:05.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 -- 172.21.15.57:0/2653885430 wait complete. 2024-04-13T05:19:05.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.694+0000 7fb554de4740 1 -- start start 2024-04-13T05:19:05.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 Processor -- start 2024-04-13T05:19:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080caeafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caeb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080caf2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56080cae0ef0 con 0x56080cad8440 2024-04-13T05:19:05.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56080c9f7c80 con 0x56080caddfa0 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56080cae1070 con 0x56080cad8e30 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.695+0000 7fb554de4740 10 client.0 _ll_register_callbacks cb 0x56080c9f7670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080caeafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080caeafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55896/0 (socket says 172.21.15.57:55896) 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 -- 172.21.15.57:0/2966604811 learned_addr learned my addr 172.21.15.57:0/2966604811 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 -- 172.21.15.57:0/2966604811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 msgr2=0x56080caeb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caeb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 -- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 msgr2=0x56080caf2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080caf2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 -- 172.21.15.57:0/2966604811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080c81a7d0 con 0x56080cad8440 2024-04-13T05:19:05.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.696+0000 7fb54aa00640 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080caeafb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb53c09bfe0 tx=0x7fb53c0209b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aebdee7db407a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb534c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb534c00640 1 -- 172.21.15.57:0/2966604811 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c016170 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb534c00640 1 -- 172.21.15.57:0/2966604811 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb53c0b5540 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5440044f0 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb534c00640 1 -- 172.21.15.57:0/2966604811 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb53c09c750 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb554de4740 10 client.15765 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.697+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56080caf5910 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb534c00640 1 -- 172.21.15.57:0/2966604811 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5440044f0 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb534c00640 1 -- 172.21.15.57:0/2966604811 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb53c0a0070 con 0x56080cad8440 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb554de4740 10 client.15765 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:05.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb554de4740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:19:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb554de4740 20 client.15765 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb554de4740 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080caf6fa0 0x56080caf9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb554de4740 4 client.15765 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:05.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.698+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56080c9dd750 con 0x56080caf6fa0 2024-04-13T05:19:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.699+0000 7fb54a000640 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080caf6fa0 0x56080caf9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.699+0000 7fb54a000640 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080caf6fa0 0x56080caf9460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eafb334277db9be4 server_cookie=7a57ebfd5b2eb584 in_seq=0 out_seq=0 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.699+0000 7fb534c00640 10 client.15765 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.701+0000 7fb534c00640 1 -- 172.21.15.57:0/2966604811 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56080c9dd750 con 0x56080caf6fa0 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.701+0000 7fb534c00640 10 client.15765 handle_command_reply: tid=0 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.701+0000 7fb554de4740 1 client.15765 shutdown 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 msgr2=0x56080caeafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080caeafb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb53c09bfe0 tx=0x7fb53c0209b0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 shutdown_connections 2024-04-13T05:19:05.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080caf6fa0 0x56080caf9460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080caddfa0 0x56080caf2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caeb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 --2- 172.21.15.57:0/2966604811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080cad8440 0x56080caeafb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb534c00640 0 client.15765 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 >> 172.21.15.57:0/2966604811 conn(0x56080cad39b0 msgr2=0x56080cad2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 shutdown_connections 2024-04-13T05:19:05.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 1 -- 172.21.15.57:0/2966604811 wait complete. 2024-04-13T05:19:05.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.702+0000 7fb554de4740 20 client.15765 trim_cache size 0 max 16384 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.718+0000 7fb554de4740 1 Processor -- start 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 -- start start 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8440 0x56080cadd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8e30 0x56080caf24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caeabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1ab080 con 0x56080caddfa0 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a8950 con 0x56080cad8e30 2024-04-13T05:19:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb554de4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a67a0 con 0x56080cad8440 2024-04-13T05:19:05.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb54b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caeabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb54b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caeabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55910/0 (socket says 172.21.15.57:55910) 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb54b400640 1 -- 172.21.15.57:0/1041395215 learned_addr learned my addr 172.21.15.57:0/1041395215 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.719+0000 7fb54aa00640 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8440 0x56080cadd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54a000640 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8e30 0x56080caf24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54b400640 1 -- 172.21.15.57:0/1041395215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8440 msgr2=0x56080cadd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54b400640 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8440 0x56080cadd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54b400640 1 -- 172.21.15.57:0/1041395215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8e30 msgr2=0x56080caf24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54b400640 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8e30 0x56080caf24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54b400640 1 -- 172.21.15.57:0/1041395215 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080cacf410 con 0x56080caddfa0 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb54b400640 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caeabc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb54402c870 tx=0x7fb5440098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ebbdd4e24b5555 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.720+0000 7fb537400640 1 -- 172.21.15.57:0/1041395215 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544039030 con 0x56080caddfa0 2024-04-13T05:19:05.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb537400640 1 -- 172.21.15.57:0/1041395215 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5440044f0 con 0x56080caddfa0 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb537400640 1 -- 172.21.15.57:0/1041395215 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544034a40 con 0x56080caddfa0 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 -- 172.21.15.57:0/1041395215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 msgr2=0x56080caeabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caeabc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb54402c870 tx=0x7fb5440098d0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 -- 172.21.15.57:0/1041395215 shutdown_connections 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8e30 0x56080caf24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caeabc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 --2- 172.21.15.57:0/1041395215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8440 0x56080cadd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.721+0000 7fb554de4740 1 -- 172.21.15.57:0/1041395215 >> 172.21.15.57:0/1041395215 conn(0x56080cad39b0 msgr2=0x56080caceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.722+0000 7fb554de4740 1 -- 172.21.15.57:0/1041395215 shutdown_connections 2024-04-13T05:19:05.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.722+0000 7fb554de4740 1 -- 172.21.15.57:0/1041395215 wait complete. 2024-04-13T05:19:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.722+0000 7fb554de4740 1 -- start start 2024-04-13T05:19:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.722+0000 7fb554de4740 1 Processor -- start 2024-04-13T05:19:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8440 0x56080caef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caefcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caf0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a6e10 con 0x56080caddfa0 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a8280 con 0x56080cad8440 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1a74e0 con 0x56080cad8e30 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb554de4740 10 client.0 _ll_register_callbacks cb 0x7fb53c098310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb54b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caf0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb54b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caf0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55912/0 (socket says 172.21.15.57:55912) 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb54b400640 1 -- 172.21.15.57:0/824561550 learned_addr learned my addr 172.21.15.57:0/824561550 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb54a000640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caefcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.723+0000 7fb54aa00640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8440 0x56080caef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb54b400640 1 -- 172.21.15.57:0/824561550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 msgr2=0x56080caefcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb54b400640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caefcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb54b400640 1 -- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8440 msgr2=0x56080caef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb54b400640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8440 0x56080caef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb54b400640 1 -- 172.21.15.57:0/824561550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080caf12f0 con 0x56080caddfa0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb54b400640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caf0d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb54402c1c0 tx=0x7fb544034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68e8b9f96aab93c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb534c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb534c00640 1 -- 172.21.15.57:0/824561550 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544039040 con 0x56080caddfa0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb534c00640 1 -- 172.21.15.57:0/824561550 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb544034780 con 0x56080caddfa0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.724+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5440044f0 con 0x56080caddfa0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.725+0000 7fb534c00640 1 -- 172.21.15.57:0/824561550 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb544034980 con 0x56080caddfa0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.725+0000 7fb554de4740 10 client.15771 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.725+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56080caf25f0 con 0x56080caddfa0 2024-04-13T05:19:05.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.725+0000 7fb534c00640 1 -- 172.21.15.57:0/824561550 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5440044f0 con 0x56080caddfa0 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.725+0000 7fb534c00640 1 -- 172.21.15.57:0/824561550 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb54404a070 con 0x56080caddfa0 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.726+0000 7fb554de4740 10 client.15771 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.726+0000 7fb554de4740 10 client.15771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.726+0000 7fb554de4740 20 client.15771 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.726+0000 7fb554de4740 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080cb5d610 0x56080cb5fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.726+0000 7fb554de4740 4 client.15771 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:19:05.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.726+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56080ca4ba40 con 0x56080cb5d610 2024-04-13T05:19:05.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.727+0000 7fb54aa00640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080cb5d610 0x56080cb5fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.727+0000 7fb54aa00640 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080cb5d610 0x56080cb5fa00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2d4e0f000a68997 server_cookie=d7888be2ad110631 in_seq=0 out_seq=0 2024-04-13T05:19:05.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.727+0000 7fb534c00640 10 client.15771 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:05.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb534c00640 1 -- 172.21.15.57:0/824561550 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56080ca4ba40 con 0x56080cb5d610 2024-04-13T05:19:05.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb534c00640 10 client.15771 handle_command_reply: tid=0 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 client.15771 shutdown 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 msgr2=0x56080caf0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caf0d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb54402c1c0 tx=0x7fb544034440 comp rx=0 tx=0).stop 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 shutdown_connections 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56080cb5d610 0x56080cb5fa00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080caddfa0 0x56080caf0d00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080cad8e30 0x56080caefcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 --2- 172.21.15.57:0/824561550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080cad8440 0x56080caef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 >> 172.21.15.57:0/824561550 conn(0x56080cad39b0 msgr2=0x56080caeabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.728+0000 7fb534c00640 0 client.15771 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:05.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.729+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 shutdown_connections 2024-04-13T05:19:05.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.729+0000 7fb554de4740 1 -- 172.21.15.57:0/824561550 wait complete. 2024-04-13T05:19:05.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.729+0000 7fb554de4740 20 client.15771 trim_cache size 0 max 16384 2024-04-13T05:19:05.735 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:19:05.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.729+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5300a3cb0 msgr2=0x7fb5300a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5300a3cb0 0x7fb5300a6170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb544046000 tx=0x7fb544048000 comp rx=0 tx=0).stop 2024-04-13T05:19:05.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 msgr2=0x7fb54c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 0x7fb54c19a270 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d770 tx=0x7fb53c00dc30 comp rx=0 tx=0).stop 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 shutdown_connections 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb5300a3cb0 0x7fb5300a6170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c075f80 0x7fb54c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb54c109da0 0x7fb54c1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 --2- 172.21.15.57:0/294061545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb54c0749a0 0x7fb54c19a270 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 >> 172.21.15.57:0/294061545 conn(0x7fb54c0fd460 msgr2=0x7fb54c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 shutdown_connections 2024-04-13T05:19:05.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.730+0000 7fb54be00640 1 -- 172.21.15.57:0/294061545 wait complete. 2024-04-13T05:19:05.748 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:19:05.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 -- 172.21.15.57:0/1863625526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4103080 msgr2=0x7fdfb4103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/1863625526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4103080 0x7fdfb4103500 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009900 tx=0x7fdfa002bc40 comp rx=0 tx=0).stop 2024-04-13T05:19:05.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 -- 172.21.15.57:0/1863625526 shutdown_connections 2024-04-13T05:19:05.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/1863625526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfb4103a40 0x7fdfb410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/1863625526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4103080 0x7fdfb4103500 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/1863625526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb4101ea0 0x7fdfb41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.935+0000 7fdfb3e00640 1 -- 172.21.15.57:0/1863625526 >> 172.21.15.57:0/1863625526 conn(0x7fdfb40fd460 msgr2=0x7fdfb40ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.936+0000 7fdfb3e00640 1 -- 172.21.15.57:0/1863625526 shutdown_connections 2024-04-13T05:19:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.936+0000 7fdfb3e00640 1 -- 172.21.15.57:0/1863625526 wait complete. 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 Processor -- start 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 -- start start 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 0x7fdfb4195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb4103080 0x7fdfb4196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfb4103a40 0x7fdfb419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410b890 con 0x7fdfb4101ea0 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410b710 con 0x7fdfb4103a40 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb410ba10 con 0x7fdfb4103080 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 0x7fdfb4195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 0x7fdfb4195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55940/0 (socket says 172.21.15.57:55940) 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.937+0000 7fdfb2a00640 1 -- 172.21.15.57:0/3134293262 learned_addr learned my addr 172.21.15.57:0/3134293262 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb3400640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfb4103a40 0x7fdfb419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb2a00640 1 -- 172.21.15.57:0/3134293262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb4103080 msgr2=0x7fdfb4196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb4103080 0x7fdfb4196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb2a00640 1 -- 172.21.15.57:0/3134293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfb4103a40 msgr2=0x7fdfb419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfb4103a40 0x7fdfb419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb2a00640 1 -- 172.21.15.57:0/3134293262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb419dae0 con 0x7fdfb4101ea0 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.938+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 0x7fdfb4195ec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400d950 tx=0x7fdfa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.939+0000 7fdfab400640 1 -- 172.21.15.57:0/3134293262 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4017070 con 0x7fdfb4101ea0 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.939+0000 7fdfab400640 1 -- 172.21.15.57:0/3134293262 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfa4009c30 con 0x7fdfb4101ea0 2024-04-13T05:19:05.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.939+0000 7fdfab400640 1 -- 172.21.15.57:0/3134293262 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400f620 con 0x7fdfb4101ea0 2024-04-13T05:19:05.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.940+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb419ddd0 con 0x7fdfb4101ea0 2024-04-13T05:19:05.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.940+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb419e1e0 con 0x7fdfb4101ea0 2024-04-13T05:19:05.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.943+0000 7fdfbc404740 1 Processor -- start 2024-04-13T05:19:05.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 -- start start 2024-04-13T05:19:05.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d08957860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d0895c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0895cfa0 0x561d0895f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561d088bd020 con 0x561d08957440 2024-04-13T05:19:05.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d08863950 con 0x561d0895cfa0 2024-04-13T05:19:05.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.944+0000 7fdfbc404740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561d088b8fe0 con 0x561d08957e30 2024-04-13T05:19:05.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d08957860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d08957860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55950/0 (socket says 172.21.15.57:55950) 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 -- 172.21.15.57:0/3189458574 learned_addr learned my addr 172.21.15.57:0/3189458574 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb2000640 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0895cfa0 0x561d0895f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 -- 172.21.15.57:0/3189458574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 msgr2=0x561d0895c930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d0895c930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 -- 172.21.15.57:0/3189458574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0895cfa0 msgr2=0x561d0895f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0895cfa0 0x561d0895f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfb3400640 1 -- 172.21.15.57:0/3189458574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d085cf3f0 con 0x561d08957440 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.945+0000 7fdfab400640 1 -- 172.21.15.57:0/3134293262 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdfa401a070 con 0x7fdfb4101ea0 2024-04-13T05:19:05.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.946+0000 7fdfb3400640 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d08957860 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00dcb0 tx=0x7fdfac013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f5feaf8470cd07 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.946+0000 7fdf9f400640 1 -- 172.21.15.57:0/3189458574 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac020030 con 0x561d08957440 2024-04-13T05:19:05.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.946+0000 7fdf9f400640 1 -- 172.21.15.57:0/3189458574 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfac013e40 con 0x561d08957440 2024-04-13T05:19:05.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.946+0000 7fdf9f400640 1 -- 172.21.15.57:0/3189458574 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac005440 con 0x561d08957440 2024-04-13T05:19:05.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.947+0000 7fdfbc404740 1 -- 172.21.15.57:0/3189458574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 msgr2=0x561d08957860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.947+0000 7fdfbc404740 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d08957860 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00dcb0 tx=0x7fdfac013520 comp rx=0 tx=0).stop 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.948+0000 7fdfbc404740 1 -- 172.21.15.57:0/3189458574 shutdown_connections 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.948+0000 7fdfbc404740 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0895cfa0 0x561d0895f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.948+0000 7fdfbc404740 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d0895c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.948+0000 7fdfbc404740 1 --2- 172.21.15.57:0/3189458574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d08957860 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.948+0000 7fdfbc404740 1 -- 172.21.15.57:0/3189458574 >> 172.21.15.57:0/3189458574 conn(0x561d089529b0 msgr2=0x561d08952db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.948+0000 7fdfab400640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdf980a3bb0 0x7fdf980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.949+0000 7fdfab400640 1 -- 172.21.15.57:0/3134293262 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdfa40d0090 con 0x7fdfb4101ea0 2024-04-13T05:19:05.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.949+0000 7fdfbc404740 1 -- 172.21.15.57:0/3189458574 shutdown_connections 2024-04-13T05:19:05.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.949+0000 7fdfbc404740 1 -- 172.21.15.57:0/3189458574 wait complete. 2024-04-13T05:19:05.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.950+0000 7fdfb2000640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdf980a3bb0 0x7fdf980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.950+0000 7fdfbc404740 1 -- start start 2024-04-13T05:19:05.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.950+0000 7fdfbc404740 1 Processor -- start 2024-04-13T05:19:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d08961f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08968f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561d0895ff50 con 0x561d0895cfa0 2024-04-13T05:19:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d08876c80 con 0x561d08957440 2024-04-13T05:19:05.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561d089600d0 con 0x561d08957e30 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfbc404740 10 client.0 _ll_register_callbacks cb 0x561d08876670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08968f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08968f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55964/0 (socket says 172.21.15.57:55964) 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb3400640 1 -- 172.21.15.57:0/2412317094 learned_addr learned my addr 172.21.15.57:0/2412317094 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb2000640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdf980a3bb0 0x7fdf980a6070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdfa002c850 tx=0x7fdfa0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb2000640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d08961f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.951+0000 7fdfb3400640 1 -- 172.21.15.57:0/2412317094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 msgr2=0x561d08961f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdfb3400640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d08961f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdfb3400640 1 -- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957440 msgr2=0x561d089619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdfb3400640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdfb3400640 1 -- 172.21.15.57:0/2412317094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d086997d0 con 0x561d0895cfa0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdfb3400640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08968f20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdfac01cc30 tx=0x7fdfac01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6045998af1b1ec87 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdf93e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:05.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdf93e00640 1 -- 172.21.15.57:0/2412317094 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0055d0 con 0x561d0895cfa0 2024-04-13T05:19:05.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.952+0000 7fdf93e00640 1 -- 172.21.15.57:0/2412317094 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdfac005770 con 0x561d0895cfa0 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac013e40 con 0x561d0895cfa0 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdf93e00640 1 -- 172.21.15.57:0/2412317094 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfac00d8b0 con 0x561d0895cfa0 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdfbc404740 10 client.15783 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d0896c2b0 con 0x561d0895cfa0 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdf93e00640 1 -- 172.21.15.57:0/2412317094 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfac013e40 con 0x561d0895cfa0 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdf93e00640 1 -- 172.21.15.57:0/2412317094 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdfac037420 con 0x561d0895cfa0 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdfbc404740 10 client.15783 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.953+0000 7fdfbc404740 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.954+0000 7fdfbc404740 20 client.15783 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:05.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.954+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d0896d600 0x561d0896f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.954+0000 7fdfbc404740 4 client.15783 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.954+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d0885c750 con 0x561d0896d600 2024-04-13T05:19:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.954+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d0896d600 0x561d0896f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.955+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d0896d600 0x561d0896f9f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4240157f71877e43 server_cookie=b7d204541aef0606 in_seq=0 out_seq=0 2024-04-13T05:19:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.955+0000 7fdf93e00640 10 client.15783 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.957+0000 7fdf93e00640 1 -- 172.21.15.57:0/2412317094 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d0885c750 con 0x561d0896d600 2024-04-13T05:19:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.957+0000 7fdf93e00640 10 client.15783 handle_command_reply: tid=0 2024-04-13T05:19:05.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.957+0000 7fdfbc404740 1 client.15783 shutdown 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 msgr2=0x561d08968f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08968f20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdfac01cc30 tx=0x7fdfac01cc60 comp rx=0 tx=0).stop 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 shutdown_connections 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d0896d600 0x561d0896f9f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08968f20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957e30 0x561d08961f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2412317094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 >> 172.21.15.57:0/2412317094 conn(0x561d089529b0 msgr2=0x561d08951cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdf93e00640 0 client.15783 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 shutdown_connections 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 1 -- 172.21.15.57:0/2412317094 wait complete. 2024-04-13T05:19:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.958+0000 7fdfbc404740 20 client.15783 trim_cache size 0 max 16384 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.986+0000 7fdfbc404740 1 Processor -- start 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.986+0000 7fdfbc404740 1 -- start start 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d08968ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d0895cfa0 0x561d08961f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfbc404740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ab050 con 0x561d08957440 2024-04-13T05:19:05.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfbc404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a8920 con 0x561d08957e30 2024-04-13T05:19:05.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfbc404740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a6770 con 0x561d0895cfa0 2024-04-13T05:19:05.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfb2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfb2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55972/0 (socket says 172.21.15.57:55972) 2024-04-13T05:19:05.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.987+0000 7fdfb2a00640 1 -- 172.21.15.57:0/717754379 learned_addr learned my addr 172.21.15.57:0/717754379 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2000640 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d08968ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb3400640 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d0895cfa0 0x561d08961f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2a00640 1 -- 172.21.15.57:0/717754379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d0895cfa0 msgr2=0x561d08961f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d0895cfa0 0x561d08961f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2a00640 1 -- 172.21.15.57:0/717754379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 msgr2=0x561d08968ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d08968ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2a00640 1 -- 172.21.15.57:0/717754379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d08957080 con 0x561d08957440 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb3400640 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d0895cfa0 0x561d08961f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.988+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d089619f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005fd0 tx=0x7fdfa409bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb5793b9f8293ac server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.989+0000 7fdf9ea00640 1 -- 172.21.15.57:0/717754379 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40cd8a0 con 0x561d08957440 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.989+0000 7fdf9ea00640 1 -- 172.21.15.57:0/717754379 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfa40ba070 con 0x561d08957440 2024-04-13T05:19:05.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.989+0000 7fdf9ea00640 1 -- 172.21.15.57:0/717754379 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4096ae0 con 0x561d08957440 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 -- 172.21.15.57:0/717754379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 msgr2=0x561d089619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d089619f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005fd0 tx=0x7fdfa409bd40 comp rx=0 tx=0).stop 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 -- 172.21.15.57:0/717754379 shutdown_connections 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d08968ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d0895cfa0 0x561d08961f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 --2- 172.21.15.57:0/717754379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d08957440 0x561d089619f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 -- 172.21.15.57:0/717754379 >> 172.21.15.57:0/717754379 conn(0x561d089529b0 msgr2=0x561d08952d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:05.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 -- 172.21.15.57:0/717754379 shutdown_connections 2024-04-13T05:19:05.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.990+0000 7fdfbc404740 1 -- 172.21.15.57:0/717754379 wait complete. 2024-04-13T05:19:05.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 -- start start 2024-04-13T05:19:05.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 Processor -- start 2024-04-13T05:19:05.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957440 0x561d089663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d089668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08967900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a6de0 con 0x561d0895cfa0 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a8250 con 0x561d08957e30 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a74b0 con 0x561d08957440 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.991+0000 7fdfbc404740 10 client.0 _ll_register_callbacks cb 0x7fdfac004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08967900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08967900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55986/0 (socket says 172.21.15.57:55986) 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 -- 172.21.15.57:0/2100930078 learned_addr learned my addr 172.21.15.57:0/2100930078 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb2000640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d089668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 -- 172.21.15.57:0/2100930078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957440 msgr2=0x561d089663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957440 0x561d089663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 -- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 msgr2=0x561d089668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:05.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d089668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb3400640 1 -- 172.21.15.57:0/2100930078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d08970020 con 0x561d0895cfa0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.992+0000 7fdfb2000640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d089668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.993+0000 7fdfb3400640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08967900 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdfac000f80 tx=0x7fdfac037a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58469887a4ac676 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.993+0000 7fdf93e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.993+0000 7fdf93e00640 1 -- 172.21.15.57:0/2100930078 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac020070 con 0x561d0895cfa0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.993+0000 7fdf93e00640 1 -- 172.21.15.57:0/2100930078 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdfac0053d0 con 0x561d0895cfa0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.993+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa40ba070 con 0x561d0895cfa0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.993+0000 7fdf93e00640 1 -- 172.21.15.57:0/2100930078 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfac0055f0 con 0x561d0895cfa0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.994+0000 7fdfbc404740 10 client.15789 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.994+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d08969020 con 0x561d0895cfa0 2024-04-13T05:19:05.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.994+0000 7fdf93e00640 1 -- 172.21.15.57:0/2100930078 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfa40ba070 con 0x561d0895cfa0 2024-04-13T05:19:05.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdf93e00640 1 -- 172.21.15.57:0/2100930078 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdfac025440 con 0x561d0895cfa0 2024-04-13T05:19:05.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfbc404740 10 client.15789 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:05.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfbc404740 10 client.15789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:19:05.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfbc404740 20 client.15789 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:05.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d089d4060 0x561d089d6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:05.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfbc404740 4 client.15789 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:19:05.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d088caa40 con 0x561d089d4060 2024-04-13T05:19:05.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.995+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d089d4060 0x561d089d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:05.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.996+0000 7fdfb2a00640 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d089d4060 0x561d089d6480 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c64f96cb8d2f153b server_cookie=91be18526767a063 in_seq=0 out_seq=0 2024-04-13T05:19:05.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.996+0000 7fdf93e00640 10 client.15789 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdf93e00640 1 -- 172.21.15.57:0/2100930078 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d088caa40 con 0x561d089d4060 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdf93e00640 10 client.15789 handle_command_reply: tid=0 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 client.15789 shutdown 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 msgr2=0x561d08967900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08967900 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdfac000f80 tx=0x7fdfac037a60 comp rx=0 tx=0).stop 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 shutdown_connections 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561d089d4060 0x561d089d6480 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d08957e30 0x561d089668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d0895cfa0 0x561d08967900 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 --2- 172.21.15.57:0/2100930078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d08957440 0x561d089663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 >> 172.21.15.57:0/2100930078 conn(0x561d089529b0 msgr2=0x561d08961990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.997+0000 7fdf93e00640 0 client.15789 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.998+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 shutdown_connections 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.998+0000 7fdfbc404740 1 -- 172.21.15.57:0/2100930078 wait complete. 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.998+0000 7fdfbc404740 20 client.15789 trim_cache size 0 max 16384 2024-04-13T05:19:06.001 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:19:06.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdf980a3bb0 msgr2=0x7fdf980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdf980a3bb0 0x7fdf980a6070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdfa002c850 tx=0x7fdfa0000bc0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 msgr2=0x7fdfb4195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 0x7fdfb4195ec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400d950 tx=0x7fdfa400de10 comp rx=0 tx=0).stop 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 shutdown_connections 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdf980a3bb0 0x7fdf980a6070 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfb4103a40 0x7fdfb419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfb4103080 0x7fdfb4196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 --2- 172.21.15.57:0/3134293262 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdfb4101ea0 0x7fdfb4195ec0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 >> 172.21.15.57:0/3134293262 conn(0x7fdfb40fd460 msgr2=0x7fdfb4106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 shutdown_connections 2024-04-13T05:19:06.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:05.999+0000 7fdfb3e00640 1 -- 172.21.15.57:0/3134293262 wait complete. 2024-04-13T05:19:06.022 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:05 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3973767689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:19:06.026 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:19:06.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 -- 172.21.15.57:0/1679944196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f01052f0 msgr2=0x7f54f0105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 --2- 172.21.15.57:0/1679944196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f01052f0 0x7f54f0105770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009900 tx=0x7f54dc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:06.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 -- 172.21.15.57:0/1679944196 shutdown_connections 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 --2- 172.21.15.57:0/1679944196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f0105cb0 0x7f54f010c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 --2- 172.21.15.57:0/1679944196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f01052f0 0x7f54f0105770 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 --2- 172.21.15.57:0/1679944196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f0104100 0x7f54f0104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 -- 172.21.15.57:0/1679944196 >> 172.21.15.57:0/1679944196 conn(0x7f54f00ff670 msgr2=0x7f54f0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 -- 172.21.15.57:0/1679944196 shutdown_connections 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.221+0000 7f54f6c00640 1 -- 172.21.15.57:0/1679944196 wait complete. 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.222+0000 7f54f6c00640 1 Processor -- start 2024-04-13T05:19:06.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.222+0000 7f54f6c00640 1 -- start start 2024-04-13T05:19:06.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f0104100 0x7f54f019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f01052f0 0x7f54f019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 0x7f54f01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010fc90 con 0x7f54f0105cb0 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010fb10 con 0x7f54f0104100 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010fe10 con 0x7f54f01052f0 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 0x7f54f01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 0x7f54f01a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56004/0 (socket says 172.21.15.57:56004) 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f6200640 1 -- 172.21.15.57:0/3974533574 learned_addr learned my addr 172.21.15.57:0/3974533574 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f5800640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f0104100 0x7f54f019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.223+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f01052f0 0x7f54f019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.224+0000 7f54f6200640 1 -- 172.21.15.57:0/3974533574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f01052f0 msgr2=0x7f54f019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.224+0000 7f54f6200640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f01052f0 0x7f54f019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.224+0000 7f54f6200640 1 -- 172.21.15.57:0/3974533574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f0104100 msgr2=0x7f54f019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.224+0000 7f54f6200640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f0104100 0x7f54f019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.224+0000 7f54f6200640 1 -- 172.21.15.57:0/3974533574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f01a1ec0 con 0x7f54f0105cb0 2024-04-13T05:19:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.225+0000 7f54f6200640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 0x7f54f01a1850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f54e800cce0 tx=0x7f54e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.225+0000 7f54ee000640 1 -- 172.21.15.57:0/3974533574 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e800f450 con 0x7f54f0105cb0 2024-04-13T05:19:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.225+0000 7f54ee000640 1 -- 172.21.15.57:0/3974533574 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e800faa0 con 0x7f54f0105cb0 2024-04-13T05:19:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.225+0000 7f54ee000640 1 -- 172.21.15.57:0/3974533574 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8018b70 con 0x7f54f0105cb0 2024-04-13T05:19:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.225+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f01a21b0 con 0x7f54f0105cb0 2024-04-13T05:19:06.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.225+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a2680 con 0x7f54f0105cb0 2024-04-13T05:19:06.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.230+0000 7f54fb03a740 1 Processor -- start 2024-04-13T05:19:06.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54ee000640 1 -- 172.21.15.57:0/3974533574 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f54e8018d10 con 0x7f54f0105cb0 2024-04-13T05:19:06.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 -- start start 2024-04-13T05:19:06.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2ceb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cf2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561bd2c48020 con 0x561bd2ceb640 2024-04-13T05:19:06.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561bd2bee950 con 0x561bd2ceac50 2024-04-13T05:19:06.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.231+0000 7f54fb03a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561bd2c43fe0 con 0x561bd2cf07b0 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.232+0000 7f54f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.232+0000 7f54f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56010/0 (socket says 172.21.15.57:56010) 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.232+0000 7f54f5800640 1 -- 172.21.15.57:0/3605915802 learned_addr learned my addr 172.21.15.57:0/3605915802 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.232+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2ceb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.232+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cf2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54ee000640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54d80a3c80 0x7f54d80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f5800640 1 -- 172.21.15.57:0/3605915802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 msgr2=0x561bd2cf2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f5800640 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cf2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f5800640 1 -- 172.21.15.57:0/3605915802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 msgr2=0x561bd2ceb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f5800640 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2ceb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f5800640 1 -- 172.21.15.57:0/3605915802 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd295a3f0 con 0x561bd2ceb640 2024-04-13T05:19:06.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54ee000640 1 -- 172.21.15.57:0/3974533574 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f54e80d0ac0 con 0x7f54f0105cb0 2024-04-13T05:19:06.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f6200640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54d80a3c80 0x7f54d80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.233+0000 7f54f5800640 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf0140 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f54e400bcb0 tx=0x7f54e40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2ea5bad04cb0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:06.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.234+0000 7f54e2a00640 1 -- 172.21.15.57:0/3605915802 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4004020 con 0x561bd2ceb640 2024-04-13T05:19:06.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.234+0000 7f54e2a00640 1 -- 172.21.15.57:0/3605915802 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40054d0 con 0x561bd2ceb640 2024-04-13T05:19:06.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.234+0000 7f54e2a00640 1 -- 172.21.15.57:0/3605915802 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4004cc0 con 0x561bd2ceb640 2024-04-13T05:19:06.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 -- 172.21.15.57:0/3605915802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 msgr2=0x561bd2cf0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf0140 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f54e400bcb0 tx=0x7f54e40107a0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54f6200640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54d80a3c80 0x7f54d80a6140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f54e8098e60 tx=0x7f54e8085f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 -- 172.21.15.57:0/3605915802 shutdown_connections 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cf2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf0140 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3605915802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2ceb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 -- 172.21.15.57:0/3605915802 >> 172.21.15.57:0/3605915802 conn(0x561bd2ce61c0 msgr2=0x561bd2ce65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.235+0000 7f54fb03a740 1 -- 172.21.15.57:0/3605915802 shutdown_connections 2024-04-13T05:19:06.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.236+0000 7f54fb03a740 1 -- 172.21.15.57:0/3605915802 wait complete. 2024-04-13T05:19:06.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.236+0000 7f54fb03a740 1 -- start start 2024-04-13T05:19:06.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.236+0000 7f54fb03a740 1 Processor -- start 2024-04-13T05:19:06.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cfc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561bd2cf3760 con 0x561bd2ceb640 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561bd2c01c80 con 0x561bd2ceac50 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561bd2cf38e0 con 0x561bd2cf07b0 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54fb03a740 10 client.0 _ll_register_callbacks cb 0x561bd2c01670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54f4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56026/0 (socket says 172.21.15.57:56026) 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54f5800640 1 -- 172.21.15.57:0/95115940 learned_addr learned my addr 172.21.15.57:0/95115940 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:06.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.237+0000 7f54f4e00640 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cfc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54f4e00640 1 -- 172.21.15.57:0/95115940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 msgr2=0x561bd2cfc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54f4e00640 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cfc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54f4e00640 1 -- 172.21.15.57:0/95115940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 msgr2=0x561bd2cf5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54f4e00640 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54f4e00640 1 -- 172.21.15.57:0/95115940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd2a247d0 con 0x561bd2ceac50 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54f4e00640 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf51f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009850 tx=0x7f54dc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9127fe556cb7a5a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.238+0000 7f54d7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.239+0000 7f54d7400640 1 -- 172.21.15.57:0/95115940 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0046c0 con 0x561bd2ceac50 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.239+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e40054d0 con 0x561bd2ceac50 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.239+0000 7f54d7400640 1 -- 172.21.15.57:0/95115940 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f54dc004860 con 0x561bd2ceac50 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.239+0000 7f54d7400640 1 -- 172.21.15.57:0/95115940 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54dc055050 con 0x561bd2ceac50 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.239+0000 7f54fb03a740 10 client.35327 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:06.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.239+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561bd2cffba0 con 0x561bd2ceac50 2024-04-13T05:19:06.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.240+0000 7f54d7400640 1 -- 172.21.15.57:0/95115940 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40054d0 con 0x561bd2ceac50 2024-04-13T05:19:06.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.240+0000 7f54d7400640 1 -- 172.21.15.57:0/95115940 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54dc058020 con 0x561bd2ceac50 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54fb03a740 10 client.35327 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54fb03a740 10 client.35327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54fb03a740 20 client.35327 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54fb03a740 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54fb03a740 4 client.35327 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd2be7750 con 0x561bd2d011f0 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.241+0000 7f54f5800640 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.242+0000 7f54f5800640 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d03660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e0ba1734335aea server_cookie=e8b1372872776c07 in_seq=0 out_seq=0 2024-04-13T05:19:06.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.242+0000 7f54d7400640 10 client.35327 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54d7400640 1 -- 172.21.15.57:0/95115940 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bd2be7750 con 0x561bd2d011f0 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54d7400640 10 client.35327 handle_command_reply: tid=0 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54fb03a740 1 client.35327 shutdown 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 msgr2=0x561bd2cf51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54fb03a740 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf51f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009850 tx=0x7f54dc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 shutdown_connections 2024-04-13T05:19:06.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.244+0000 7f54fb03a740 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d03660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cfc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54d7400640 0 client.35327 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 1 --2- 172.21.15.57:0/95115940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf51f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 >> 172.21.15.57:0/95115940 conn(0x561bd2ce61c0 msgr2=0x561bd2cdccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 shutdown_connections 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 1 -- 172.21.15.57:0/95115940 wait complete. 2024-04-13T05:19:06.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.245+0000 7f54fb03a740 20 client.35327 trim_cache size 0 max 16384 2024-04-13T05:19:06.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:05 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3973767689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:19:06.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.259+0000 7f54fb03a740 1 Processor -- start 2024-04-13T05:19:06.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.259+0000 7f54fb03a740 1 -- start start 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.259+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54fb03a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aafe0 con 0x561bd2ceb640 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54fb03a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01a88b0 con 0x561bd2ceac50 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54fb03a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01a6700 con 0x561bd2cf07b0 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56036/0 (socket says 172.21.15.57:56036) 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 -- 172.21.15.57:0/3627390980 learned_addr learned my addr 172.21.15.57:0/3627390980 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 -- 172.21.15.57:0/3627390980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 msgr2=0x561bd2cebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 -- 172.21.15.57:0/3627390980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 msgr2=0x561bd2cf3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.260+0000 7f54f5800640 1 -- 172.21.15.57:0/3627390980 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd2ceba20 con 0x561bd2ceb640 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.261+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:06.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.261+0000 7f54f5800640 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf41c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005d00 tx=0x7f54e4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde6d59f240d80a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.261+0000 7f54e2000640 1 -- 172.21.15.57:0/3627390980 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4030070 con 0x561bd2ceb640 2024-04-13T05:19:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.261+0000 7f54e2000640 1 -- 172.21.15.57:0/3627390980 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e402b170 con 0x561bd2ceb640 2024-04-13T05:19:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.261+0000 7f54e2000640 1 -- 172.21.15.57:0/3627390980 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4003850 con 0x561bd2ceb640 2024-04-13T05:19:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 -- 172.21.15.57:0/3627390980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 msgr2=0x561bd2cf41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf41c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005d00 tx=0x7f54e4013040 comp rx=0 tx=0).stop 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 -- 172.21.15.57:0/3627390980 shutdown_connections 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2cf07b0 0x561bd2cebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2ceb640 0x561bd2cf41c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3627390980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 -- 172.21.15.57:0/3627390980 >> 172.21.15.57:0/3627390980 conn(0x561bd2ce61c0 msgr2=0x561bd2ce65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 -- 172.21.15.57:0/3627390980 shutdown_connections 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.262+0000 7f54fb03a740 1 -- 172.21.15.57:0/3627390980 wait complete. 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.263+0000 7f54fb03a740 1 -- start start 2024-04-13T05:19:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.263+0000 7f54fb03a740 1 Processor -- start 2024-04-13T05:19:06.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 0x561bd2cf96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2cf07b0 0x561bd2cfa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01a6d70 con 0x561bd2cf07b0 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01a81e0 con 0x561bd2ceac50 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01a7440 con 0x561bd2ceb640 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54fb03a740 10 client.0 _ll_register_callbacks cb 0x7f54dc0051c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54f5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 0x561bd2cf96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54f4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2cf07b0 0x561bd2cfa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54f5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 0x561bd2cf96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60322/0 (socket says 172.21.15.57:60322) 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54f5800640 1 -- 172.21.15.57:0/3991310509 learned_addr learned my addr 172.21.15.57:0/3991310509 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.264+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f5800640 1 -- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 msgr2=0x561bd2cf9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f5800640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f5800640 1 -- 172.21.15.57:0/3991310509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2cf07b0 msgr2=0x561bd2cfa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f5800640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2cf07b0 0x561bd2cfa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f5800640 1 -- 172.21.15.57:0/3991310509 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd2cfacc0 con 0x561bd2ceb640 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2cf07b0 0x561bd2cfa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54f5800640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 0x561bd2cf96c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005930 tx=0x7f54e400bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6806434d46418e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.265+0000 7f54d7400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:19:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.266+0000 7f54d7400640 1 -- 172.21.15.57:0/3991310509 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4039060 con 0x561bd2ceb640 2024-04-13T05:19:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.266+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e402b170 con 0x561bd2ceb640 2024-04-13T05:19:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.266+0000 7f54d7400640 1 -- 172.21.15.57:0/3991310509 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f54e40044d0 con 0x561bd2ceb640 2024-04-13T05:19:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.267+0000 7f54d7400640 1 -- 172.21.15.57:0/3991310509 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54e4040050 con 0x561bd2ceb640 2024-04-13T05:19:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.267+0000 7f54fb03a740 10 client.25282 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.267+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561bd2cfbfc0 con 0x561bd2ceb640 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54d7400640 1 -- 172.21.15.57:0/3991310509 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e402b170 con 0x561bd2ceb640 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54d7400640 1 -- 172.21.15.57:0/3991310509 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54e4003c80 con 0x561bd2ceb640 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54fb03a740 10 client.25282 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54fb03a740 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54fb03a740 20 client.25282 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54fb03a740 4 client.25282 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:19:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.268+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bd2c55a40 con 0x561bd2d011f0 2024-04-13T05:19:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.269+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.269+0000 7f54f4e00640 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d69d80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab505647a16aba8 server_cookie=70ff7ef3e7d34aaf in_seq=0 out_seq=0 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.269+0000 7f54d7400640 10 client.25282 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54d7400640 1 -- 172.21.15.57:0/3991310509 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bd2c55a40 con 0x561bd2d011f0 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54d7400640 10 client.25282 handle_command_reply: tid=0 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 client.25282 shutdown 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 msgr2=0x561bd2cf96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 0x561bd2cf96c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005930 tx=0x7f54e400bda0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 shutdown_connections 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561bd2d011f0 0x561bd2d69d80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bd2cf07b0 0x561bd2cfa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd2ceb640 0x561bd2cf96c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54d7400640 0 client.25282 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 --2- 172.21.15.57:0/3991310509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd2ceac50 0x561bd2cf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 >> 172.21.15.57:0/3991310509 conn(0x561bd2ce61c0 msgr2=0x561bd2cecbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 shutdown_connections 2024-04-13T05:19:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.270+0000 7f54fb03a740 1 -- 172.21.15.57:0/3991310509 wait complete. 2024-04-13T05:19:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.271+0000 7f54fb03a740 20 client.25282 trim_cache size 0 max 16384 2024-04-13T05:19:06.274 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.271+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54d80a3c80 msgr2=0x7f54d80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.271+0000 7f54f6c00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54d80a3c80 0x7f54d80a6140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f54e8098e60 tx=0x7f54e8085f90 comp rx=0 tx=0).stop 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.271+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 msgr2=0x7f54f01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.271+0000 7f54f6c00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 0x7f54f01a1850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f54e800cce0 tx=0x7f54e800a620 comp rx=0 tx=0).stop 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 shutdown_connections 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54d80a3c80 0x7f54d80a6140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54f0105cb0 0x7f54f01a1850 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f01052f0 0x7f54f019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 --2- 172.21.15.57:0/3974533574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54f0104100 0x7f54f019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 >> 172.21.15.57:0/3974533574 conn(0x7f54f00ff670 msgr2=0x7f54f0108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 shutdown_connections 2024-04-13T05:19:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:06.272+0000 7f54f6c00640 1 -- 172.21.15.57:0/3974533574 wait complete. 2024-04-13T05:19:06.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:05 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3973767689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:19:07.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:06 smithi083 ceph-mon[35133]: pgmap v501: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:19:07.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:06 smithi057 ceph-mon[32331]: pgmap v501: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:19:07.298 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:19:07.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:19:07.299 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:19:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:06 smithi189 ceph-mon[35531]: pgmap v501: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:19:07.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 -- 172.21.15.57:0/3995239548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 msgr2=0x7fd434074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 --2- 172.21.15.57:0/3995239548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434074e00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd420009900 tx=0x7fd42002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 -- 172.21.15.57:0/3995239548 shutdown_connections 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 --2- 172.21.15.57:0/3995239548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd434109da0 0x7fd43410c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 --2- 172.21.15.57:0/3995239548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434074e00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 --2- 172.21.15.57:0/3995239548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd434075f80 0x7fd4340743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 -- 172.21.15.57:0/3995239548 >> 172.21.15.57:0/3995239548 conn(0x7fd4340fd460 msgr2=0x7fd4340ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 -- 172.21.15.57:0/3995239548 shutdown_connections 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.588+0000 7fd433200640 1 -- 172.21.15.57:0/3995239548 wait complete. 2024-04-13T05:19:07.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.589+0000 7fd433200640 1 Processor -- start 2024-04-13T05:19:07.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 -- start start 2024-04-13T05:19:07.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434105290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd434075f80 0x7fd4341057d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd434109da0 0x7fd434101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd43410fc60 con 0x7fd4340749a0 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd43410fae0 con 0x7fd434109da0 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd433200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd43410fde0 con 0x7fd434075f80 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd431e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434105290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd431e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434105290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56054/0 (socket says 172.21.15.57:56054) 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.590+0000 7fd431e00640 1 -- 172.21.15.57:0/1912243199 learned_addr learned my addr 172.21.15.57:0/1912243199 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.591+0000 7fd432800640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd434109da0 0x7fd434101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.591+0000 7fd431e00640 1 -- 172.21.15.57:0/1912243199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd434075f80 msgr2=0x7fd4341057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.591+0000 7fd431e00640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd434075f80 0x7fd4341057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.591+0000 7fd431e00640 1 -- 172.21.15.57:0/1912243199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd434109da0 msgr2=0x7fd434101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.591+0000 7fd431e00640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd434109da0 0x7fd434101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.591+0000 7fd431e00640 1 -- 172.21.15.57:0/1912243199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4341026c0 con 0x7fd4340749a0 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.592+0000 7fd431e00640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434105290 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c920 tx=0x7fd42400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.592+0000 7fd42aa00640 1 -- 172.21.15.57:0/1912243199 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424007c70 con 0x7fd4340749a0 2024-04-13T05:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.592+0000 7fd42aa00640 1 -- 172.21.15.57:0/1912243199 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd424007e10 con 0x7fd4340749a0 2024-04-13T05:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.592+0000 7fd42aa00640 1 -- 172.21.15.57:0/1912243199 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424018820 con 0x7fd4340749a0 2024-04-13T05:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.592+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341a66f0 con 0x7fd4340749a0 2024-04-13T05:19:07.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.592+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341a6b00 con 0x7fd4340749a0 2024-04-13T05:19:07.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.596+0000 7fd43bdb1740 1 Processor -- start 2024-04-13T05:19:07.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 -- start start 2024-04-13T05:19:07.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b53358420 0x558b53358840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358e10 0x558b5335d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 0x558b53360410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558b532bebc0 con 0x558b5335df80 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b53265660 con 0x558b53358420 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.597+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b532bab80 con 0x558b53358e10 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.598+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 0x558b53360410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.598+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 0x558b53360410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56068/0 (socket says 172.21.15.57:56068) 2024-04-13T05:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.598+0000 7fd431400640 1 -- 172.21.15.57:0/3182524910 learned_addr learned my addr 172.21.15.57:0/3182524910 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.598+0000 7fd42aa00640 1 -- 172.21.15.57:0/1912243199 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd4240189c0 con 0x7fd4340749a0 2024-04-13T05:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.598+0000 7fd432800640 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b53358420 0x558b53358840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.598+0000 7fd431e00640 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358e10 0x558b5335d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.599+0000 7fd431400640 1 -- 172.21.15.57:0/3182524910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358e10 msgr2=0x558b5335d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.599+0000 7fd431400640 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358e10 0x558b5335d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.599+0000 7fd431400640 1 -- 172.21.15.57:0/3182524910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b53358420 msgr2=0x558b53358840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.599+0000 7fd431400640 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b53358420 0x558b53358840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.599+0000 7fd431400640 1 -- 172.21.15.57:0/3182524910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b53288220 con 0x558b5335df80 2024-04-13T05:19:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.600+0000 7fd431400640 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 0x558b53360410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd420036040 tx=0x7fd420009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d31e7ecba2dc56 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.600+0000 7fd41f400640 1 -- 172.21.15.57:0/3182524910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420042020 con 0x558b5335df80 2024-04-13T05:19:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.600+0000 7fd41f400640 1 -- 172.21.15.57:0/3182524910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd42002ce30 con 0x558b5335df80 2024-04-13T05:19:07.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.600+0000 7fd41f400640 1 -- 172.21.15.57:0/3182524910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420003c90 con 0x558b5335df80 2024-04-13T05:19:07.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.601+0000 7fd42aa00640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4180a3c30 0x7fd4180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.601+0000 7fd42aa00640 1 -- 172.21.15.57:0/1912243199 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd4240d08a0 con 0x7fd4340749a0 2024-04-13T05:19:07.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.601+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3182524910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 msgr2=0x558b53360410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.601+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 0x558b53360410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd420036040 tx=0x7fd420009610 comp rx=0 tx=0).stop 2024-04-13T05:19:07.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.601+0000 7fd432800640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4180a3c30 0x7fd4180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3182524910 shutdown_connections 2024-04-13T05:19:07.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b5335df80 0x558b53360410 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358e10 0x558b5335d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3182524910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b53358420 0x558b53358840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3182524910 >> 172.21.15.57:0/3182524910 conn(0x558b53353990 msgr2=0x558b53353d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3182524910 shutdown_connections 2024-04-13T05:19:07.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3182524910 wait complete. 2024-04-13T05:19:07.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.602+0000 7fd432800640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4180a3c30 0x7fd4180a60f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x558b53359670 tx=0x7fd42c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.603+0000 7fd43bdb1740 1 -- start start 2024-04-13T05:19:07.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.603+0000 7fd43bdb1740 1 Processor -- start 2024-04-13T05:19:07.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53373930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53373e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b5337ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558b53371ee0 con 0x558b53358e10 2024-04-13T05:19:07.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b53278a30 con 0x558b5335df80 2024-04-13T05:19:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b53372060 con 0x558b53358420 2024-04-13T05:19:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd43bdb1740 10 client.0 _ll_register_callbacks cb 0x558b52fd0c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53373e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53373e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56076/0 (socket says 172.21.15.57:56076) 2024-04-13T05:19:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd432800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b5337ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.604+0000 7fd431400640 1 -- 172.21.15.57:0/3804990309 learned_addr learned my addr 172.21.15.57:0/3804990309 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd431e00640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53373930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd431400640 1 -- 172.21.15.57:0/3804990309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 msgr2=0x558b53373930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd431400640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53373930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd431400640 1 -- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 msgr2=0x558b5337ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd431400640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b5337ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd431400640 1 -- 172.21.15.57:0/3804990309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b530997e0 con 0x558b53358e10 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.605+0000 7fd432800640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b5337ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.606+0000 7fd431400640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53373e70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4200042f0 tx=0x7fd420004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a648b7c147eb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.606+0000 7fd413e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.606+0000 7fd413e00640 1 -- 172.21.15.57:0/3804990309 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420042030 con 0x558b53358e10 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.606+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42002ce30 con 0x558b53358e10 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.606+0000 7fd413e00640 1 -- 172.21.15.57:0/3804990309 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd420034440 con 0x558b53358e10 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.606+0000 7fd413e00640 1 -- 172.21.15.57:0/3804990309 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd420059050 con 0x558b53358e10 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.607+0000 7fd43bdb1740 10 client.15819 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.607+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b530f9f70 con 0x558b53358e10 2024-04-13T05:19:07.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.607+0000 7fd413e00640 1 -- 172.21.15.57:0/3804990309 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd42002ce30 con 0x558b53358e10 2024-04-13T05:19:07.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.607+0000 7fd413e00640 1 -- 172.21.15.57:0/3804990309 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4200534b0 con 0x558b53358e10 2024-04-13T05:19:07.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.607+0000 7fd43bdb1740 10 client.15819 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.607+0000 7fd43bdb1740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.608+0000 7fd43bdb1740 20 client.15819 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.608+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b5337f380 0x558b533817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.608+0000 7fd43bdb1740 4 client.15819 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.608+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b532cc5e0 con 0x558b5337f380 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.608+0000 7fd431e00640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b5337f380 0x558b533817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.609+0000 7fd431e00640 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b5337f380 0x558b533817d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e8717e2545ebdd server_cookie=b9778204c416efe1 in_seq=0 out_seq=0 2024-04-13T05:19:07.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.609+0000 7fd413e00640 10 client.15819 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.611+0000 7fd413e00640 1 -- 172.21.15.57:0/3804990309 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b532cc5e0 con 0x558b5337f380 2024-04-13T05:19:07.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.611+0000 7fd413e00640 10 client.15819 handle_command_reply: tid=0 2024-04-13T05:19:07.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.611+0000 7fd43bdb1740 1 client.15819 shutdown 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.611+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 msgr2=0x558b53373e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.611+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53373e70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4200042f0 tx=0x7fd420004320 comp rx=0 tx=0).stop 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 shutdown_connections 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b5337f380 0x558b533817d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd413e00640 0 client.15819 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b5337ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53373e70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3804990309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53373930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 >> 172.21.15.57:0/3804990309 conn(0x558b53353990 msgr2=0x558b53352b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 shutdown_connections 2024-04-13T05:19:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3804990309 wait complete. 2024-04-13T05:19:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.612+0000 7fd43bdb1740 20 client.15819 trim_cache size 0 max 16384 2024-04-13T05:19:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.649+0000 7fd43bdb1740 1 Processor -- start 2024-04-13T05:19:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.649+0000 7fd43bdb1740 1 -- start start 2024-04-13T05:19:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53357300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b5337ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53386090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341aafb0 con 0x558b53358e10 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a8880 con 0x558b5335df80 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a2000 con 0x558b53358420 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b5337ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b5337ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56080/0 (socket says 172.21.15.57:56080) 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd431400640 1 -- 172.21.15.57:0/2344379527 learned_addr learned my addr 172.21.15.57:0/2344379527 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd431e00640 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53357300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd432800640 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53386090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.650+0000 7fd431400640 1 -- 172.21.15.57:0/2344379527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 msgr2=0x558b53357300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd431400640 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53357300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd431400640 1 -- 172.21.15.57:0/2344379527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 msgr2=0x558b53386090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd431400640 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53386090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd431400640 1 -- 172.21.15.57:0/2344379527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b52f91170 con 0x558b53358e10 2024-04-13T05:19:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd431400640 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b5337ae00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd420009230 tx=0x7fd420005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cbd0c246bf499c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd41ea00640 1 -- 172.21.15.57:0/2344379527 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420042040 con 0x558b53358e10 2024-04-13T05:19:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd41ea00640 1 -- 172.21.15.57:0/2344379527 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd420004390 con 0x558b53358e10 2024-04-13T05:19:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.651+0000 7fd41ea00640 1 -- 172.21.15.57:0/2344379527 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420053340 con 0x558b53358e10 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 -- 172.21.15.57:0/2344379527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 msgr2=0x558b5337ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b5337ae00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd420009230 tx=0x7fd420005d50 comp rx=0 tx=0).stop 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 -- 172.21.15.57:0/2344379527 shutdown_connections 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53386090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b5337ae00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/2344379527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53357300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.652+0000 7fd43bdb1740 1 -- 172.21.15.57:0/2344379527 >> 172.21.15.57:0/2344379527 conn(0x558b53353990 msgr2=0x558b53356bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.653+0000 7fd43bdb1740 1 -- 172.21.15.57:0/2344379527 shutdown_connections 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.653+0000 7fd43bdb1740 1 -- 172.21.15.57:0/2344379527 wait complete. 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.653+0000 7fd43bdb1740 1 -- start start 2024-04-13T05:19:07.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.653+0000 7fd43bdb1740 1 Processor -- start 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53378770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53378cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53379cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a6d90 con 0x558b53358e10 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a81b0 con 0x558b5335df80 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a7410 con 0x558b53358420 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd43bdb1740 10 client.0 _ll_register_callbacks cb 0x7fd42c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:07.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53378cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd431400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53378cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56082/0 (socket says 172.21.15.57:56082) 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.654+0000 7fd431400640 1 -- 172.21.15.57:0/3247413537 learned_addr learned my addr 172.21.15.57:0/3247413537 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd432800640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53379cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431e00640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53378770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431400640 1 -- 172.21.15.57:0/3247413537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 msgr2=0x558b53378770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431400640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53378770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431400640 1 -- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 msgr2=0x558b53379cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431400640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53379cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431400640 1 -- 172.21.15.57:0/3247413537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b53279be0 con 0x558b53358e10 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd432800640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53379cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd431400640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53378cb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd420003c60 tx=0x7fd420009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90cafe5fbb15c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.655+0000 7fd413e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.656+0000 7fd413e00640 1 -- 172.21.15.57:0/3247413537 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420042070 con 0x558b53358e10 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.656+0000 7fd413e00640 1 -- 172.21.15.57:0/3247413537 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd420053a20 con 0x558b53358e10 2024-04-13T05:19:07.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.656+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd420004390 con 0x558b53358e10 2024-04-13T05:19:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.656+0000 7fd413e00640 1 -- 172.21.15.57:0/3247413537 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd42006d050 con 0x558b53358e10 2024-04-13T05:19:07.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 10 client.15825 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b533e5a00 con 0x558b53358e10 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd413e00640 1 -- 172.21.15.57:0/3247413537 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd420004390 con 0x558b53358e10 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd413e00640 1 -- 172.21.15.57:0/3247413537 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd420034540 con 0x558b53358e10 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 10 client.15825 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 20 client.15825 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b533e63c0 0x558b533e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 4 client.15825 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.657+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b532c6450 con 0x558b533e63c0 2024-04-13T05:19:07.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.658+0000 7fd431e00640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b533e63c0 0x558b533e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.659+0000 7fd413e00640 10 client.15825 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.659+0000 7fd431e00640 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b533e63c0 0x558b533e87e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f53f56b84dc9d9a server_cookie=223e0a76a54a502d in_seq=0 out_seq=0 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.662+0000 7fd413e00640 1 -- 172.21.15.57:0/3247413537 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b532c6450 con 0x558b533e63c0 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.662+0000 7fd413e00640 10 client.15825 handle_command_reply: tid=0 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 client.15825 shutdown 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 msgr2=0x558b53378cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53378cb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd420003c60 tx=0x7fd420009260 comp rx=0 tx=0).stop 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 shutdown_connections 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x558b533e63c0 0x558b533e87e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd413e00640 0 client.15825 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b5335df80 0x558b53379cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558b53358e10 0x558b53378cb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 --2- 172.21.15.57:0/3247413537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b53358420 0x558b53378770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.663+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 >> 172.21.15.57:0/3247413537 conn(0x558b53353990 msgr2=0x558b53357480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.664+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 shutdown_connections 2024-04-13T05:19:07.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.664+0000 7fd43bdb1740 1 -- 172.21.15.57:0/3247413537 wait complete. 2024-04-13T05:19:07.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.664+0000 7fd43bdb1740 20 client.15825 trim_cache size 0 max 16384 2024-04-13T05:19:07.667 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:19:07.667 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:19:07.667 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "53dcb1ed-d8f1-42f3-8b1b-e52f54379124", 2024-04-13T05:19:07.667 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:19:07.667 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4180a3c30 msgr2=0x7fd4180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4180a3c30 0x7fd4180a60f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x558b53359670 tx=0x7fd42c009600 comp rx=0 tx=0).stop 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 msgr2=0x7fd434105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434105290 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c920 tx=0x7fd42400cde0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 shutdown_connections 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd4180a3c30 0x7fd4180a60f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd434109da0 0x7fd434101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd434075f80 0x7fd4341057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 --2- 172.21.15.57:0/1912243199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd4340749a0 0x7fd434105290 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 >> 172.21.15.57:0/1912243199 conn(0x7fd4340fd460 msgr2=0x7fd434105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 shutdown_connections 2024-04-13T05:19:07.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.665+0000 7fd433200640 1 -- 172.21.15.57:0/1912243199 wait complete. 2024-04-13T05:19:07.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:19:07.690 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:19:07.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 -- 172.21.15.57:0/1282737611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 msgr2=0x7f2c60103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 --2- 172.21.15.57:0/1282737611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c60103670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009900 tx=0x7f2c4c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 -- 172.21.15.57:0/1282737611 shutdown_connections 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 --2- 172.21.15.57:0/1282737611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60103bb0 0x7f2c6010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 --2- 172.21.15.57:0/1282737611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c60103670 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 --2- 172.21.15.57:0/1282737611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c60102000 0x7f2c60102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.877+0000 7f2c67000640 1 -- 172.21.15.57:0/1282737611 >> 172.21.15.57:0/1282737611 conn(0x7f2c600fd570 msgr2=0x7f2c600ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.878+0000 7f2c67000640 1 -- 172.21.15.57:0/1282737611 shutdown_connections 2024-04-13T05:19:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.878+0000 7f2c67000640 1 -- 172.21.15.57:0/1282737611 wait complete. 2024-04-13T05:19:07.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 Processor -- start 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 -- start start 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60102000 0x7f2c6019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c6019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c60103bb0 0x7f2c601a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010db40 con 0x7f2c601031f0 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010d9c0 con 0x7f2c60103bb0 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c67000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010dcc0 con 0x7f2c60102000 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c5f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c6019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.879+0000 7f2c5f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c6019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56094/0 (socket says 172.21.15.57:56094) 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 -- 172.21.15.57:0/3874894878 learned_addr learned my addr 172.21.15.57:0/3874894878 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60102000 0x7f2c6019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 -- 172.21.15.57:0/3874894878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60102000 msgr2=0x7f2c6019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60102000 0x7f2c6019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 -- 172.21.15.57:0/3874894878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c60103bb0 msgr2=0x7f2c601a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c60103bb0 0x7f2c601a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 -- 172.21.15.57:0/3874894878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c601a2050 con 0x7f2c601031f0 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60102000 0x7f2c6019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.880+0000 7f2c5f400640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c6019a970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006010 tx=0x7f2c4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.881+0000 7f2c5cc00640 1 -- 172.21.15.57:0/3874894878 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c043020 con 0x7f2c601031f0 2024-04-13T05:19:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.881+0000 7f2c5cc00640 1 -- 172.21.15.57:0/3874894878 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c004440 con 0x7f2c601031f0 2024-04-13T05:19:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.881+0000 7f2c5cc00640 1 -- 172.21.15.57:0/3874894878 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c003bf0 con 0x7f2c601031f0 2024-04-13T05:19:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.881+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c601a22e0 con 0x7f2c601031f0 2024-04-13T05:19:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.881+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c601a2830 con 0x7f2c601031f0 2024-04-13T05:19:07.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.885+0000 7f2c69fbe740 1 Processor -- start 2024-04-13T05:19:07.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 -- start start 2024-04-13T05:19:07.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a4001758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400175ec0 0x55a40017a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a40017d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a4000dc020 con 0x55a400175ec0 2024-04-13T05:19:07.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a4000826d0 con 0x55a40017b030 2024-04-13T05:19:07.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.886+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a4000d7fe0 con 0x55a4001754d0 2024-04-13T05:19:07.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.887+0000 7f2c5cc00640 1 -- 172.21.15.57:0/3874894878 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2c4c046020 con 0x7f2c601031f0 2024-04-13T05:19:07.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.887+0000 7f2c65000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a4001758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.887+0000 7f2c65000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a4001758f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60390/0 (socket says 172.21.15.57:60390) 2024-04-13T05:19:07.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.887+0000 7f2c65000640 1 -- 172.21.15.57:0/2068460949 learned_addr learned my addr 172.21.15.57:0/2068460949 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.887+0000 7f2c5f400640 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a40017d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400175ec0 0x55a40017a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c65000640 1 -- 172.21.15.57:0/2068460949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 msgr2=0x55a40017d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c65000640 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a40017d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c65000640 1 -- 172.21.15.57:0/2068460949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400175ec0 msgr2=0x55a40017a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c65000640 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400175ec0 0x55a40017a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c65000640 1 -- 172.21.15.57:0/2068460949 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3ffeb87d0 con 0x55a4001754d0 2024-04-13T05:19:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.888+0000 7f2c65000640 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a4001758f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d80 tx=0x7f2c5800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5800831b333e5951 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.889+0000 7f2c55600640 1 -- 172.21.15.57:0/2068460949 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58016070 con 0x55a4001754d0 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.889+0000 7f2c5cc00640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2c440a3d00 0x7f2c440a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.889+0000 7f2c55600640 1 -- 172.21.15.57:0/2068460949 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c58005020 con 0x55a4001754d0 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.889+0000 7f2c55600640 1 -- 172.21.15.57:0/2068460949 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58011670 con 0x55a4001754d0 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.889+0000 7f2c5cc00640 1 -- 172.21.15.57:0/3874894878 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2c4c0f2b70 con 0x7f2c601031f0 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2c440a3d00 0x7f2c440a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2068460949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 msgr2=0x55a4001758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a4001758f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d80 tx=0x7f2c5800e6c0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2068460949 shutdown_connections 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a40017d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400175ec0 0x55a40017a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2068460949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a4001758f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2068460949 >> 172.21.15.57:0/2068460949 conn(0x55a400170a40 msgr2=0x55a400170e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.890+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2068460949 shutdown_connections 2024-04-13T05:19:07.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.891+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2c440a3d00 0x7f2c440a61c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a4001762c0 tx=0x7f2c50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.891+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2068460949 wait complete. 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.891+0000 7f2c69fbe740 1 -- start start 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.891+0000 7f2c69fbe740 1 Processor -- start 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a40017f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a40017fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a40017b030 0x55a400186b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a40017dfe0 con 0x55a40017b030 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a400095a90 con 0x55a400175ec0 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a40017e160 con 0x55a4001754d0 2024-04-13T05:19:07.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.892+0000 7f2c69fbe740 10 client.0 _ll_register_callbacks cb 0x55a400095480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:07.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a40017fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a40017f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c65000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a40017b030 0x55a400186b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a40017fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:48732/0 (socket says 172.21.15.57:48732) 2024-04-13T05:19:07.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 -- 172.21.15.57:0/2804138085 learned_addr learned my addr 172.21.15.57:0/2804138085 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 -- 172.21.15.57:0/2804138085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 msgr2=0x55a40017f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a40017f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 -- 172.21.15.57:0/2804138085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a40017b030 msgr2=0x55a400186b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a40017b030 0x55a400186b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c5f400640 1 -- 172.21.15.57:0/2804138085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3ffdee3f0 con 0x55a400175ec0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.893+0000 7f2c65000640 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a40017b030 0x55a400186b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.894+0000 7f2c5f400640 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a40017fae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0bb420 tx=0x7f2c4c0bad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0dc791691be575 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.894+0000 7f2c4a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.894+0000 7f2c4a000640 1 -- 172.21.15.57:0/2804138085 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0fa040 con 0x55a400175ec0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.895+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c58005020 con 0x55a400175ec0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.895+0000 7f2c4a000640 1 -- 172.21.15.57:0/2804138085 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2c4c0f0380 con 0x55a400175ec0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.895+0000 7f2c4a000640 1 -- 172.21.15.57:0/2804138085 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c4c005560 con 0x55a400175ec0 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.895+0000 7f2c69fbe740 10 client.35357 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:07.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.895+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a3ffdae0f0 con 0x55a400175ec0 2024-04-13T05:19:07.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c4a000640 1 -- 172.21.15.57:0/2804138085 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c58005020 con 0x55a400175ec0 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c4a000640 1 -- 172.21.15.57:0/2804138085 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c4c0c7230 con 0x55a400175ec0 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c69fbe740 10 client.35357 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c69fbe740 10 client.35357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c69fbe740 20 client.35357 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a40018b3f0 0x55a40018d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c69fbe740 4 client.35357 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.896+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a40007b600 con 0x55a40018b3f0 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.897+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a40018b3f0 0x55a40018d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.898+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a40018b3f0 0x55a40018d800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb613631e8d80dd9 server_cookie=1aeb5414b40d6a56 in_seq=0 out_seq=0 2024-04-13T05:19:07.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.898+0000 7f2c4a000640 10 client.35357 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c4a000640 1 -- 172.21.15.57:0/2804138085 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a40007b600 con 0x55a40018b3f0 2024-04-13T05:19:07.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c4a000640 10 client.35357 handle_command_reply: tid=0 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 client.35357 shutdown 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 msgr2=0x55a40017fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a40017fae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0bb420 tx=0x7f2c4c0bad70 comp rx=0 tx=0).stop 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 shutdown_connections 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a40018b3f0 0x55a40018d800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a40017fae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a40017b030 0x55a400186b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/2804138085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4001754d0 0x55a40017f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c4a000640 0 client.35357 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 >> 172.21.15.57:0/2804138085 conn(0x55a400170a40 msgr2=0x55a40016fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 shutdown_connections 2024-04-13T05:19:07.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.900+0000 7f2c69fbe740 1 -- 172.21.15.57:0/2804138085 wait complete. 2024-04-13T05:19:07.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.901+0000 7f2c69fbe740 20 client.35357 trim_cache size 0 max 16384 2024-04-13T05:19:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 Processor -- start 2024-04-13T05:19:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 -- start start 2024-04-13T05:19:07.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 0x55a40017ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a400186ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a40017b030 0x55a40016f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601af590 con 0x55a4001754d0 2024-04-13T05:19:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ace60 con 0x55a400175ec0 2024-04-13T05:19:07.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.921+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601aacb0 con 0x55a40017b030 2024-04-13T05:19:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 0x55a40017ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 0x55a40017ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56122/0 (socket says 172.21.15.57:56122) 2024-04-13T05:19:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 -- 172.21.15.57:0/1946803924 learned_addr learned my addr 172.21.15.57:0/1946803924 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5f400640 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a400186ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c65000640 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a40017b030 0x55a40016f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 -- 172.21.15.57:0/1946803924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a40017b030 msgr2=0x55a40016f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a40017b030 0x55a40016f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 -- 172.21.15.57:0/1946803924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 msgr2=0x55a400186ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a400186ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5fe00640 1 -- 172.21.15.57:0/1946803924 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a40018de80 con 0x55a4001754d0 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.922+0000 7f2c5f400640 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a400186ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.923+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 0x55a40017ebc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009640 tx=0x7f2c50015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6de4a879c67aa6f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.923+0000 7f2c54c00640 1 -- 172.21.15.57:0/1946803924 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500283b0 con 0x55a4001754d0 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.923+0000 7f2c54c00640 1 -- 172.21.15.57:0/1946803924 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c5001b8b0 con 0x55a4001754d0 2024-04-13T05:19:07.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.923+0000 7f2c54c00640 1 -- 172.21.15.57:0/1946803924 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50027920 con 0x55a4001754d0 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.924+0000 7f2c69fbe740 1 -- 172.21.15.57:0/1946803924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 msgr2=0x55a40017ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.924+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 0x55a40017ebc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009640 tx=0x7f2c50015000 comp rx=0 tx=0).stop 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 -- 172.21.15.57:0/1946803924 shutdown_connections 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a400175ec0 0x55a400186ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a40017b030 0x55a40016f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/1946803924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4001754d0 0x55a40017ebc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 -- 172.21.15.57:0/1946803924 >> 172.21.15.57:0/1946803924 conn(0x55a400170a40 msgr2=0x55a40017f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 -- 172.21.15.57:0/1946803924 shutdown_connections 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 -- 172.21.15.57:0/1946803924 wait complete. 2024-04-13T05:19:07.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.925+0000 7f2c69fbe740 1 -- start start 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.926+0000 7f2c69fbe740 1 Processor -- start 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.926+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a400175ec0 0x55a400184750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.926+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a400184c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.926+0000 7f2c69fbe740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 0x55a400186120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.926+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ab320 con 0x55a400185ca0 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.926+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ac790 con 0x55a40017b030 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c69fbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ab9f0 con 0x55a400175ec0 2024-04-13T05:19:07.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c69fbe740 10 client.0 _ll_register_callbacks cb 0x7f2c4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:07.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c65000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 0x55a400186120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c65000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 0x55a400186120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56130/0 (socket says 172.21.15.57:56130) 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c65000640 1 -- 172.21.15.57:0/818496393 learned_addr learned my addr 172.21.15.57:0/818496393 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a400175ec0 0x55a400184750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.927+0000 7f2c5f400640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a400184c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c65000640 1 -- 172.21.15.57:0/818496393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a400175ec0 msgr2=0x55a400184750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c65000640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a400175ec0 0x55a400184750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c65000640 1 -- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 msgr2=0x55a400184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c65000640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a400184c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c65000640 1 -- 172.21.15.57:0/818496393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a400175c60 con 0x55a400185ca0 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c5f400640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a400184c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.928+0000 7f2c65000640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 0x55a400186120 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2c580096c0 tx=0x7f2c58009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f018b45c0bddef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:07.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.929+0000 7f2c4a000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:07.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.930+0000 7f2c4a000640 1 -- 172.21.15.57:0/818496393 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58028020 con 0x55a400185ca0 2024-04-13T05:19:07.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.930+0000 7f2c4a000640 1 -- 172.21.15.57:0/818496393 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2c58004450 con 0x55a400185ca0 2024-04-13T05:19:07.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.930+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5001b8b0 con 0x55a400185ca0 2024-04-13T05:19:07.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.931+0000 7f2c4a000640 1 -- 172.21.15.57:0/818496393 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c58016040 con 0x55a400185ca0 2024-04-13T05:19:07.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.931+0000 7f2c69fbe740 10 client.15843 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.931+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a40016b030 con 0x55a400185ca0 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.931+0000 7f2c4a000640 1 -- 172.21.15.57:0/818496393 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c5001b8b0 con 0x55a400185ca0 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c4a000640 1 -- 172.21.15.57:0/818496393 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c5800f040 con 0x55a400185ca0 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c69fbe740 10 client.15843 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c69fbe740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c69fbe740 20 client.15843 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a4001ec640 0x55a4001eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c69fbe740 4 client.15843 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:19:07.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.932+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a40017f390 con 0x55a4001ec640 2024-04-13T05:19:07.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.933+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a4001ec640 0x55a4001eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:07.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.934+0000 7f2c5fe00640 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a4001ec640 0x55a4001eea60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891aeff2ea950d6c server_cookie=6dae58126b78a599 in_seq=0 out_seq=0 2024-04-13T05:19:07.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.934+0000 7f2c4a000640 10 client.15843 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.934+0000 7f2c4a000640 1 -- 172.21.15.57:0/818496393 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a40017f390 con 0x55a4001ec640 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.934+0000 7f2c4a000640 10 client.15843 handle_command_reply: tid=0 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.934+0000 7f2c69fbe740 1 client.15843 shutdown 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 msgr2=0x55a400186120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 0x55a400186120 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2c580096c0 tx=0x7f2c58009af0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 shutdown_connections 2024-04-13T05:19:07.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a4001ec640 0x55a4001eea60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a40017b030 0x55a400184c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a400185ca0 0x55a400186120 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 --2- 172.21.15.57:0/818496393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a400175ec0 0x55a400184750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 >> 172.21.15.57:0/818496393 conn(0x55a400170a40 msgr2=0x55a400173ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c4a000640 0 client.15843 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.935+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 shutdown_connections 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.936+0000 7f2c69fbe740 1 -- 172.21.15.57:0/818496393 wait complete. 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.936+0000 7f2c69fbe740 20 client.15843 trim_cache size 0 max 16384 2024-04-13T05:19:07.939 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: "53dcb1ed-d8f1-42f3-8b1b-e52f54379124": { 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "53dcb1ed-d8f1-42f3-8b1b-e52f54379124", 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:19:07.940 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:19:07.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2c440a3d00 msgr2=0x7f2c440a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2c440a3d00 0x7f2c440a61c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55a4001762c0 tx=0x7f2c50005f90 comp rx=0 tx=0).stop 2024-04-13T05:19:07.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 msgr2=0x7f2c6019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:07.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c6019a970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006010 tx=0x7f2c4c004040 comp rx=0 tx=0).stop 2024-04-13T05:19:07.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 shutdown_connections 2024-04-13T05:19:07.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2c440a3d00 0x7f2c440a61c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c60103bb0 0x7f2c601a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2c601031f0 0x7f2c6019a970 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 --2- 172.21.15.57:0/3874894878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c60102000 0x7f2c6019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:07.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 >> 172.21.15.57:0/3874894878 conn(0x7f2c600fd570 msgr2=0x7f2c601089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:07.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 shutdown_connections 2024-04-13T05:19:07.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:07.937+0000 7f2c67000640 1 -- 172.21.15.57:0/3874894878 wait complete. 2024-04-13T05:19:07.963 INFO:tasks.cephfs.filesystem:scrub status for tag:53dcb1ed-d8f1-42f3-8b1b-e52f54379124 - {'path': '/', 'tag': '53dcb1ed-d8f1-42f3-8b1b-e52f54379124', 'options': 'recursive,force'} 2024-04-13T05:19:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:08 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:19:09.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:08 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:09.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:08 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:19:09.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:08 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:09.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:08 smithi083 ceph-mon[35133]: pgmap v502: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:08 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:19:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:08 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:08 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:19:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:08 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:08 smithi057 ceph-mon[32331]: pgmap v502: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:08 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:19:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:08 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:08 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:19:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:08 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:08 smithi189 ceph-mon[35531]: pgmap v502: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:11.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:10 smithi083 ceph-mon[35133]: pgmap v503: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-13T05:19:11.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:10 smithi057 ceph-mon[32331]: pgmap v503: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-13T05:19:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:10 smithi189 ceph-mon[35531]: pgmap v503: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-13T05:19:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:12 smithi083 ceph-mon[35133]: pgmap v504: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:12 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:19:13.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:12 smithi057 ceph-mon[32331]: pgmap v504: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:13.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:12 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:19:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:12 smithi189 ceph-mon[35531]: pgmap v504: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:12 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:19:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:14 smithi083 ceph-mon[35133]: pgmap v505: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:15.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:14 smithi057 ceph-mon[32331]: pgmap v505: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:15.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:14 smithi189 ceph-mon[35531]: pgmap v505: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:16 smithi083 ceph-mon[35133]: pgmap v506: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-13T05:19:17.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:16 smithi057 ceph-mon[32331]: pgmap v506: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-13T05:19:17.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:16 smithi189 ceph-mon[35531]: pgmap v506: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-13T05:19:19.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:18 smithi083 ceph-mon[35133]: pgmap v507: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:19.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:18 smithi057 ceph-mon[32331]: pgmap v507: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:19.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:18 smithi189 ceph-mon[35531]: pgmap v507: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-13T05:19:21.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:20 smithi083 ceph-mon[35133]: pgmap v508: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-13T05:19:21.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:20 smithi057 ceph-mon[32331]: pgmap v508: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-13T05:19:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:20 smithi189 ceph-mon[35531]: pgmap v508: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-13T05:19:23.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:22 smithi083 ceph-mon[35133]: pgmap v509: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-13T05:19:23.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:22 smithi057 ceph-mon[32331]: pgmap v509: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-13T05:19:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:22 smithi189 ceph-mon[35531]: pgmap v509: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-13T05:19:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:24 smithi083 ceph-mon[35133]: pgmap v510: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:19:25.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:24 smithi057 ceph-mon[32331]: pgmap v510: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:19:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:24 smithi189 ceph-mon[35531]: pgmap v510: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-13T05:19:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:26 smithi083 ceph-mon[35133]: pgmap v511: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:19:27.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:26 smithi057 ceph-mon[32331]: pgmap v511: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:19:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:26 smithi189 ceph-mon[35531]: pgmap v511: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:19:28.770 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 58% (7210/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 96% (11870/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-13T05:19:29.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:29 smithi057 ceph-mon[32331]: pgmap v512: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:29.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:29 smithi189 ceph-mon[35531]: pgmap v512: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:29.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:29 smithi083 ceph-mon[35133]: pgmap v512: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to '218a691417ef57c5d22a8d5cf3c5f41f7b542838'. 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.694 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:29.695 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at 218a691417e Merge PR #52623 into wip-vshankar-testing-20240412.042930 2024-04-13T05:19:29.708 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-13T05:19:29.773 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-13T05:19:29.781 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-13T05:19:29.781 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-13T05:19:30.683 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-13T05:19:30.819 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-13T05:19:30.915 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-13T05:19:30.922 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-13T05:19:30.922 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-13T05:19:31.012 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-13T05:19:31.023 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:19:31.023 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-13T05:19:31.081 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-04-13T05:19:31.082 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-04-13T05:19:31.082 DEBUG:teuthology.orchestra.run.smithi057:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=218a691417ef57c5d22a8d5cf3c5f41f7b542838 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-04-13T05:19:31.184 INFO:tasks.workunit.client.0.smithi057.stdout:getting blogbench 2024-04-13T05:19:31.184 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'getting blogbench' 2024-04-13T05:19:31.184 INFO:tasks.workunit.client.0.smithi057.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-04-13T05:19:31.225 INFO:tasks.workunit.client.0.smithi057.stderr:--2024-04-13 05:19:31-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-04-13T05:19:31.235 INFO:tasks.workunit.client.0.smithi057.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-13T05:19:31.256 INFO:tasks.workunit.client.0.smithi057.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-13T05:19:31.277 INFO:tasks.workunit.client.0.smithi057.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-13T05:19:31.277 INFO:tasks.workunit.client.0.smithi057.stderr:Length: 103355 (101K) [application/octet-stream] 2024-04-13T05:19:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:31 smithi057 ceph-mon[32331]: pgmap v513: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:19:31.284 INFO:tasks.workunit.client.0.smithi057.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-04-13T05:19:31.295 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:31.320 INFO:tasks.workunit.client.0.smithi057.stderr: 0K .......... .......... .......... .......... .......... 49% 1.41M 0s 2024-04-13T05:19:31.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:31 smithi189 ceph-mon[35531]: pgmap v513: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:19:31.339 INFO:tasks.workunit.client.0.smithi057.stderr: 50K .......... .......... .......... .......... .......... 99% 2.57M 0s 2024-04-13T05:19:31.340 INFO:tasks.workunit.client.0.smithi057.stderr: 100K 100% 1.74T=0.05s 2024-04-13T05:19:31.340 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:31.342 INFO:tasks.workunit.client.0.smithi057.stderr:2024-04-13 05:19:31 (1.84 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-04-13T05:19:31.342 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-13T05:19:31.344 INFO:tasks.workunit.client.0.smithi057.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-04-13T05:19:31.499 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/ 2024-04-13T05:19:31.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:31 smithi083 ceph-mon[35133]: pgmap v513: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:19:31.505 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/m4/ 2024-04-13T05:19:31.510 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-04-13T05:19:31.519 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/ 2024-04-13T05:19:31.527 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/blogbench.8 2024-04-13T05:19:31.535 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/Makefile.am 2024-04-13T05:19:31.543 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/Makefile.in 2024-04-13T05:19:31.555 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/ 2024-04-13T05:19:31.564 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/mysnprintf.c 2024-04-13T05:19:31.573 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/mysnprintf.h 2024-04-13T05:19:31.582 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench.c 2024-04-13T05:19:31.593 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench.h 2024-04-13T05:19:31.602 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/fakesnprintf.c 2024-04-13T05:19:31.614 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/fakesnprintf.h 2024-04-13T05:19:31.625 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench_p.h 2024-04-13T05:19:31.637 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-04-13T05:19:31.648 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-04-13T05:19:31.658 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/Makefile.am 2024-04-13T05:19:31.667 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/Makefile.in 2024-04-13T05:19:31.678 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/process.c 2024-04-13T05:19:31.689 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/globals.h 2024-04-13T05:19:31.697 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/commenter.c 2024-04-13T05:19:31.706 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/rewriter.c 2024-04-13T05:19:31.716 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/reader.c 2024-04-13T05:19:31.726 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/writer.c 2024-04-13T05:19:31.736 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/helpers.c 2024-04-13T05:19:31.745 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/NEWS 2024-04-13T05:19:31.757 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/compile 2024-04-13T05:19:31.766 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/depcomp 2024-04-13T05:19:31.776 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/aclocal.m4 2024-04-13T05:19:31.789 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/README 2024-04-13T05:19:31.798 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/configure 2024-04-13T05:19:31.845 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/configure.ac 2024-04-13T05:19:31.855 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.guess 2024-04-13T05:19:31.869 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/install-sh 2024-04-13T05:19:31.879 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.sub 2024-04-13T05:19:31.891 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/missing 2024-04-13T05:19:31.902 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/mkinstalldirs 2024-04-13T05:19:31.910 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/Makefile.am 2024-04-13T05:19:31.919 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/Makefile.in 2024-04-13T05:19:31.928 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.h.in 2024-04-13T05:19:31.938 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/AUTHORS 2024-04-13T05:19:31.948 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/INSTALL 2024-04-13T05:19:31.959 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/ChangeLog 2024-04-13T05:19:31.970 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/COPYING 2024-04-13T05:19:31.983 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd blogbench-1.0/ 2024-04-13T05:19:31.992 INFO:tasks.workunit.client.0.smithi057.stdout:making blogbench 2024-04-13T05:19:31.993 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'making blogbench' 2024-04-13T05:19:31.993 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./configure 2024-04-13T05:19:32.425 INFO:tasks.workunit.client.0.smithi057.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-13T05:19:33.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:33 smithi057 ceph-mon[32331]: pgmap v514: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:33 smithi189 ceph-mon[35531]: pgmap v514: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:33.461 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether build environment is sane... yes 2024-04-13T05:19:33.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:33 smithi083 ceph-mon[35133]: pgmap v514: 129 pgs: 129 active+clean; 658 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:19:33.521 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gawk... gawk 2024-04-13T05:19:33.596 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether make sets $(MAKE)... yes 2024-04-13T05:19:33.671 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gcc... gcc 2024-04-13T05:19:33.938 INFO:tasks.workunit.client.0.smithi057.stdout:checking for C compiler default output file name... a.out 2024-04-13T05:19:33.955 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether the C compiler works... yes 2024-04-13T05:19:33.970 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether we are cross compiling... no 2024-04-13T05:19:34.130 INFO:tasks.workunit.client.0.smithi057.stdout:checking for suffix of executables... 2024-04-13T05:19:34.291 INFO:tasks.workunit.client.0.smithi057.stdout:checking for suffix of object files... o 2024-04-13T05:19:34.469 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether we are using the GNU C compiler... yes 2024-04-13T05:19:34.653 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether gcc accepts -g... yes 2024-04-13T05:19:34.879 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gcc option to accept ANSI C... none needed 2024-04-13T05:19:35.132 INFO:tasks.workunit.client.0.smithi057.stdout:checking for style of include used by make... GNU 2024-04-13T05:19:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:35 smithi057 ceph-mon[32331]: pgmap v515: 129 pgs: 129 active+clean; 783 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-04-13T05:19:35.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:35 smithi189 ceph-mon[35531]: pgmap v515: 129 pgs: 129 active+clean; 783 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-04-13T05:19:35.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:35 smithi083 ceph-mon[35133]: pgmap v515: 129 pgs: 129 active+clean; 783 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-04-13T05:19:35.752 INFO:tasks.workunit.client.0.smithi057.stdout:checking dependency style of gcc... gcc3 2024-04-13T05:19:35.754 INFO:tasks.workunit.client.0.smithi057.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-13T05:19:36.027 INFO:tasks.workunit.client.0.smithi057.stdout:checking how to run the C preprocessor... gcc -E 2024-04-13T05:19:36.248 INFO:tasks.workunit.client.0.smithi057.stdout:checking for egrep... grep -E 2024-04-13T05:19:36.300 INFO:tasks.workunit.client.0.smithi057.stdout:checking for AIX... no 2024-04-13T05:19:36.638 INFO:tasks.workunit.client.0.smithi057.stdout:checking for library containing strerror... none required 2024-04-13T05:19:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:37 smithi189 ceph-mon[35531]: pgmap v516: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-04-13T05:19:37.433 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ANSI C header files... yes 2024-04-13T05:19:37.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:37 smithi083 ceph-mon[35133]: pgmap v516: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-04-13T05:19:37.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:37 smithi057 ceph-mon[32331]: pgmap v516: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-04-13T05:19:37.643 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-13T05:19:37.860 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/types.h... yes 2024-04-13T05:19:37.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:19:37.964 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:19:38.090 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/stat.h... yes 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 -- 172.21.15.57:0/2963024427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84077bc0 msgr2=0x7f2a84078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 --2- 172.21.15.57:0/2963024427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84077bc0 0x7f2a84078020 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009900 tx=0x7f2a7002bc40 comp rx=0 tx=0).stop 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 -- 172.21.15.57:0/2963024427 shutdown_connections 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 --2- 172.21.15.57:0/2963024427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 0x7f2a84078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 --2- 172.21.15.57:0/2963024427 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84077bc0 0x7f2a84078020 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 --2- 172.21.15.57:0/2963024427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84076980 0x7f2a84076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 -- 172.21.15.57:0/2963024427 >> 172.21.15.57:0/2963024427 conn(0x7f2a840ff770 msgr2=0x7f2a84101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 -- 172.21.15.57:0/2963024427 shutdown_connections 2024-04-13T05:19:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.184+0000 7f2a8b400640 1 -- 172.21.15.57:0/2963024427 wait complete. 2024-04-13T05:19:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.185+0000 7f2a8b400640 1 Processor -- start 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.185+0000 7f2a8b400640 1 -- start start 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a8b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 0x7f2a8419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84077bc0 0x7f2a8419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 0x7f2a841a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a8b400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84114160 con 0x7f2a84076980 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a8b400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84113fe0 con 0x7f2a84078560 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a8b400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841142e0 con 0x7f2a84077bc0 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a88a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 0x7f2a8419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a88a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 0x7f2a8419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59422/0 (socket says 172.21.15.57:59422) 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a88a00640 1 -- 172.21.15.57:0/695522859 learned_addr learned my addr 172.21.15.57:0/695522859 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a89400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 0x7f2a841a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a83e00640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84077bc0 0x7f2a8419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.186+0000 7f2a88a00640 1 -- 172.21.15.57:0/695522859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84077bc0 msgr2=0x7f2a8419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a88a00640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84077bc0 0x7f2a8419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a88a00640 1 -- 172.21.15.57:0/695522859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 msgr2=0x7f2a841a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a88a00640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 0x7f2a841a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a88a00640 1 -- 172.21.15.57:0/695522859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a841a64a0 con 0x7f2a84076980 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a83e00640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84077bc0 0x7f2a8419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a89400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 0x7f2a841a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a88a00640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 0x7f2a8419e880 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d770 tx=0x7f2a7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a81600640 1 -- 172.21.15.57:0/695522859 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78017070 con 0x7f2a84076980 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a841a6790 con 0x7f2a84076980 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a81600640 1 -- 172.21.15.57:0/695522859 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a780054d0 con 0x7f2a84076980 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a81600640 1 -- 172.21.15.57:0/695522859 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78004cf0 con 0x7f2a84076980 2024-04-13T05:19:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.187+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a841a6c90 con 0x7f2a84076980 2024-04-13T05:19:38.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 Processor -- start 2024-04-13T05:19:38.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a81600640 1 -- 172.21.15.57:0/695522859 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2a78004e90 con 0x7f2a84076980 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 -- start start 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae7298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae72e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae7314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5589ae690020 con 0x5589ae72f030 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5589ae6366d0 con 0x5589ae7294d0 2024-04-13T05:19:38.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.191+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589ae68bfe0 con 0x5589ae729ec0 2024-04-13T05:19:38.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae7314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae7314c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59434/0 (socket says 172.21.15.57:59434) 2024-04-13T05:19:38.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 -- 172.21.15.57:0/3201638290 learned_addr learned my addr 172.21.15.57:0/3201638290 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a89400640 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae7298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a88a00640 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae72e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 -- 172.21.15.57:0/3201638290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 msgr2=0x5589ae72e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae72e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 -- 172.21.15.57:0/3201638290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 msgr2=0x5589ae7298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae7298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.192+0000 7f2a83e00640 1 -- 172.21.15.57:0/3201638290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ae46c7d0 con 0x5589ae72f030 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.193+0000 7f2a83e00640 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae7314c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a70005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d136d43d89a37e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.193+0000 7f2a75c00640 1 -- 172.21.15.57:0/3201638290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70043020 con 0x5589ae72f030 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.193+0000 7f2a75c00640 1 -- 172.21.15.57:0/3201638290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a70004040 con 0x5589ae72f030 2024-04-13T05:19:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.193+0000 7f2a75c00640 1 -- 172.21.15.57:0/3201638290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700046c0 con 0x5589ae72f030 2024-04-13T05:19:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a81600640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a6c0a3bb0 0x7f2a6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a81600640 1 -- 172.21.15.57:0/695522859 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2a780d0320 con 0x7f2a84076980 2024-04-13T05:19:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a89400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a6c0a3bb0 0x7f2a6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3201638290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 msgr2=0x5589ae7314c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae7314c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a70005e00 comp rx=0 tx=0).stop 2024-04-13T05:19:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3201638290 shutdown_connections 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae7314c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae72e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3201638290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae7298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3201638290 >> 172.21.15.57:0/3201638290 conn(0x5589ae724a40 msgr2=0x5589ae724e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.194+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3201638290 shutdown_connections 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.195+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3201638290 wait complete. 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.195+0000 7f2a89400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a6c0a3bb0 0x7f2a6c0a6070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5589ae72a720 tx=0x7f2a7c009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.195+0000 7f2a8e25a740 1 -- start start 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.195+0000 7f2a8e25a740 1 Processor -- start 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae7294d0 0x5589ae733740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 0x5589ae733c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae73acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5589ae731f80 con 0x5589ae729ec0 2024-04-13T05:19:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5589ae649a90 con 0x5589ae72f030 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589ae732100 con 0x5589ae7294d0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a8e25a740 10 client.0 _ll_register_callbacks cb 0x5589ae649480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a83e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 0x5589ae733c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a83e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 0x5589ae733c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59436/0 (socket says 172.21.15.57:59436) 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a83e00640 1 -- 172.21.15.57:0/2338460914 learned_addr learned my addr 172.21.15.57:0/2338460914 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.196+0000 7f2a88a00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae7294d0 0x5589ae733740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a89400640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae73acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a83e00640 1 -- 172.21.15.57:0/2338460914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae7294d0 msgr2=0x5589ae733740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a83e00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae7294d0 0x5589ae733740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a83e00640 1 -- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 msgr2=0x5589ae73acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a83e00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae73acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a83e00640 1 -- 172.21.15.57:0/2338460914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ae3a23f0 con 0x5589ae729ec0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a88a00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae7294d0 0x5589ae733740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a83e00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 0x5589ae733c80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a7002cc20 tx=0x7f2a7002cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41556ce9196dfff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a6a000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.197+0000 7f2a6a000640 1 -- 172.21.15.57:0/2338460914 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7004c070 con 0x5589ae729ec0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a6a000640 1 -- 172.21.15.57:0/2338460914 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2a70003a00 con 0x5589ae729ec0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a70004040 con 0x5589ae729ec0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a6a000640 1 -- 172.21.15.57:0/2338460914 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a70043040 con 0x5589ae729ec0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a8e25a740 10 client.15855 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589ae3620f0 con 0x5589ae729ec0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a6a000640 1 -- 172.21.15.57:0/2338460914 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a70004040 con 0x5589ae729ec0 2024-04-13T05:19:38.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a6a000640 1 -- 172.21.15.57:0/2338460914 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a70004dd0 con 0x5589ae729ec0 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a8e25a740 10 client.15855 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.198+0000 7f2a8e25a740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.199+0000 7f2a8e25a740 20 client.15855 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.199+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae73f1b0 0x5589ae7415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.199+0000 7f2a8e25a740 4 client.15855 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.199+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ae62f600 con 0x5589ae73f1b0 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.199+0000 7f2a88a00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae73f1b0 0x5589ae7415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.200+0000 7f2a88a00640 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae73f1b0 0x5589ae7415a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=644725c764e76fa0 server_cookie=5d9eb3b345d77b5 in_seq=0 out_seq=0 2024-04-13T05:19:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.200+0000 7f2a6a000640 10 client.15855 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:38.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.203+0000 7f2a6a000640 1 -- 172.21.15.57:0/2338460914 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589ae62f600 con 0x5589ae73f1b0 2024-04-13T05:19:38.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.203+0000 7f2a6a000640 10 client.15855 handle_command_reply: tid=0 2024-04-13T05:19:38.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.203+0000 7f2a8e25a740 1 client.15855 shutdown 2024-04-13T05:19:38.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.203+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 msgr2=0x5589ae733c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.203+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 0x5589ae733c80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a7002cc20 tx=0x7f2a7002cc50 comp rx=0 tx=0).stop 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 shutdown_connections 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae73f1b0 0x5589ae7415a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae73acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae729ec0 0x5589ae733c80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/2338460914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae7294d0 0x5589ae733740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 >> 172.21.15.57:0/2338460914 conn(0x5589ae724a40 msgr2=0x5589ae723b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a6a000640 0 client.15855 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 shutdown_connections 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 1 -- 172.21.15.57:0/2338460914 wait complete. 2024-04-13T05:19:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.204+0000 7f2a8e25a740 20 client.15855 trim_cache size 0 max 16384 2024-04-13T05:19:38.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 Processor -- start 2024-04-13T05:19:38.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 -- start start 2024-04-13T05:19:38.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 0x5589ae7232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae73ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae723830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841b3a70 con 0x5589ae7294d0 2024-04-13T05:19:38.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841b1340 con 0x5589ae72f030 2024-04-13T05:19:38.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.224+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841af190 con 0x5589ae729ec0 2024-04-13T05:19:38.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 0x5589ae7232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 0x5589ae7232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59440/0 (socket says 172.21.15.57:59440) 2024-04-13T05:19:38.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 -- 172.21.15.57:0/3761345116 learned_addr learned my addr 172.21.15.57:0/3761345116 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a89400640 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae723830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a83e00640 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae73ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 -- 172.21.15.57:0/3761345116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 msgr2=0x5589ae73ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae73ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 -- 172.21.15.57:0/3761345116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 msgr2=0x5589ae723830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae723830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.225+0000 7f2a88a00640 1 -- 172.21.15.57:0/3761345116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ae7345e0 con 0x5589ae7294d0 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.226+0000 7f2a88a00640 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 0x5589ae7232f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a7809c190 tx=0x7f2a780cd8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413e4fec3eea27bd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.226+0000 7f2a75200640 1 -- 172.21.15.57:0/3761345116 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78020040 con 0x5589ae7294d0 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.226+0000 7f2a75200640 1 -- 172.21.15.57:0/3761345116 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a78009c30 con 0x5589ae7294d0 2024-04-13T05:19:38.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.226+0000 7f2a75200640 1 -- 172.21.15.57:0/3761345116 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780cdb30 con 0x5589ae7294d0 2024-04-13T05:19:38.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3761345116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 msgr2=0x5589ae7232f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 0x5589ae7232f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a7809c190 tx=0x7f2a780cd8f0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3761345116 shutdown_connections 2024-04-13T05:19:38.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae72f030 0x5589ae723830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae73ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/3761345116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae7294d0 0x5589ae7232f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3761345116 >> 172.21.15.57:0/3761345116 conn(0x5589ae724a40 msgr2=0x5589ae72b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3761345116 shutdown_connections 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 -- 172.21.15.57:0/3761345116 wait complete. 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.227+0000 7f2a8e25a740 1 -- start start 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 Processor -- start 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae738010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae738550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae739510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841af800 con 0x5589ae72f030 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841b0c70 con 0x5589ae7294d0 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841afed0 con 0x5589ae729ec0 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a8e25a740 10 client.0 _ll_register_callbacks cb 0x7f2a70004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a89400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae739510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a89400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae739510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59448/0 (socket says 172.21.15.57:59448) 2024-04-13T05:19:38.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.228+0000 7f2a89400640 1 -- 172.21.15.57:0/120462822 learned_addr learned my addr 172.21.15.57:0/120462822 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a88a00640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae738010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a83e00640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae738550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a89400640 1 -- 172.21.15.57:0/120462822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 msgr2=0x5589ae738550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a89400640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae738550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a89400640 1 -- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 msgr2=0x5589ae738010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a89400640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae738010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a89400640 1 -- 172.21.15.57:0/120462822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ae71f030 con 0x5589ae72f030 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a89400640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae739510 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c012940 tx=0x7f2a7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502988ab6cf29ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a6a000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.229+0000 7f2a6a000640 1 -- 172.21.15.57:0/120462822 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c021dd0 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a6a000640 1 -- 172.21.15.57:0/120462822 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2a7c01f450 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589ae69da40 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a6a000640 1 -- 172.21.15.57:0/120462822 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c01f650 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 10 client.15861 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589ae73ad10 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a6a000640 1 -- 172.21.15.57:0/120462822 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5589ae69da40 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a6a000640 1 -- 172.21.15.57:0/120462822 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a7c028070 con 0x5589ae72f030 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 10 client.15861 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 20 client.15861 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae7a5cb0 0x5589ae7a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 4 client.15861 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:19:38.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.230+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589ae6978b0 con 0x5589ae7a5cb0 2024-04-13T05:19:38.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.231+0000 7f2a88a00640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae7a5cb0 0x5589ae7a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.232+0000 7f2a88a00640 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae7a5cb0 0x5589ae7a80d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff403dfda8583a6 server_cookie=cdc9733f1237f0c9 in_seq=0 out_seq=0 2024-04-13T05:19:38.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.232+0000 7f2a6a000640 10 client.15861 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a6a000640 1 -- 172.21.15.57:0/120462822 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5589ae6978b0 con 0x5589ae7a5cb0 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a6a000640 10 client.15861 handle_command_reply: tid=0 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 client.15861 shutdown 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 msgr2=0x5589ae739510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae739510 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c012940 tx=0x7f2a7c004a40 comp rx=0 tx=0).stop 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 shutdown_connections 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5589ae7a5cb0 0x5589ae7a80d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5589ae72f030 0x5589ae739510 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589ae729ec0 0x5589ae738550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 --2- 172.21.15.57:0/120462822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5589ae7294d0 0x5589ae738010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.233+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 >> 172.21.15.57:0/120462822 conn(0x5589ae724a40 msgr2=0x5589ae727ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a6a000640 0 client.15861 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 shutdown_connections 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8e25a740 1 -- 172.21.15.57:0/120462822 wait complete. 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8e25a740 20 client.15861 trim_cache size 0 max 16384 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:19:38.238 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:19:38.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a6c0a3bb0 msgr2=0x7f2a6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8b400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a6c0a3bb0 0x7f2a6c0a6070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5589ae72a720 tx=0x7f2a7c009590 comp rx=0 tx=0).stop 2024-04-13T05:19:38.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 msgr2=0x7f2a8419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.234+0000 7f2a8b400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 0x7f2a8419e880 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d770 tx=0x7f2a7800dc30 comp rx=0 tx=0).stop 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 shutdown_connections 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2a6c0a3bb0 0x7f2a6c0a6070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a84078560 0x7f2a841a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a84077bc0 0x7f2a8419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 --2- 172.21.15.57:0/695522859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a84076980 0x7f2a8419e880 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 >> 172.21.15.57:0/695522859 conn(0x7f2a840ff770 msgr2=0x7f2a84100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 shutdown_connections 2024-04-13T05:19:38.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.235+0000 7f2a8b400640 1 -- 172.21.15.57:0/695522859 wait complete. 2024-04-13T05:19:38.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:19:38.257 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:19:38.302 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdlib.h... yes 2024-04-13T05:19:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 -- 172.21.15.57:0/973888933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388102fb0 msgr2=0x7fc388103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 --2- 172.21.15.57:0/973888933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388102fb0 0x7fc388103430 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc374009900 tx=0x7fc37402bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 -- 172.21.15.57:0/973888933 shutdown_connections 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 --2- 172.21.15.57:0/973888933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 0x7fc38810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 --2- 172.21.15.57:0/973888933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388102fb0 0x7fc388103430 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 --2- 172.21.15.57:0/973888933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc388101e60 0x7fc388102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 -- 172.21.15.57:0/973888933 >> 172.21.15.57:0/973888933 conn(0x7fc3880fd3d0 msgr2=0x7fc3880ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 -- 172.21.15.57:0/973888933 shutdown_connections 2024-04-13T05:19:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.460+0000 7fc38d400640 1 -- 172.21.15.57:0/973888933 wait complete. 2024-04-13T05:19:38.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 Processor -- start 2024-04-13T05:19:38.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 -- start start 2024-04-13T05:19:38.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 0x7fc388075d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc388102fb0 0x7fc3880743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 0x7fc3880748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc38810da00 con 0x7fc388101e60 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc38810d880 con 0x7fc388103970 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.461+0000 7fc38d400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc38810db80 con 0x7fc388102fb0 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 0x7fc388075d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 0x7fc388075d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59464/0 (socket says 172.21.15.57:59464) 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 -- 172.21.15.57:0/432061328 learned_addr learned my addr 172.21.15.57:0/432061328 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc388102fb0 0x7fc3880743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc38ca00640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 0x7fc3880748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 -- 172.21.15.57:0/432061328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc388102fb0 msgr2=0x7fc3880743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc388102fb0 0x7fc3880743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 -- 172.21.15.57:0/432061328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 msgr2=0x7fc3880748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 0x7fc3880748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc387e00640 1 -- 172.21.15.57:0/432061328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3880751b0 con 0x7fc388101e60 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.462+0000 7fc38ca00640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 0x7fc3880748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.463+0000 7fc387e00640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 0x7fc388075d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc37800d770 tx=0x7fc37800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.463+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378017070 con 0x7fc388101e60 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.463+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3881a21f0 con 0x7fc388101e60 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.463+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881a2710 con 0x7fc388101e60 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.463+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3780054d0 con 0x7fc388101e60 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.463+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378020060 con 0x7fc388101e60 2024-04-13T05:19:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.465+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3680052c0 con 0x7fc388101e60 2024-04-13T05:19:38.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.467+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc378009c30 con 0x7fc388101e60 2024-04-13T05:19:38.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.468+0000 7fc384c00640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc36c0a3c30 0x7fc36c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.469+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc3780d04d0 con 0x7fc388101e60 2024-04-13T05:19:38.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.469+0000 7fc387400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc36c0a3c30 0x7fc36c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.470+0000 7fc387400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc36c0a3c30 0x7fc36c0a60f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc37402c240 tx=0x7fc374005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.474+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc37809cab0 con 0x7fc388101e60 2024-04-13T05:19:38.503 INFO:tasks.workunit.client.0.smithi057.stdout:checking for string.h... yes 2024-04-13T05:19:38.718 INFO:tasks.workunit.client.0.smithi057.stdout:checking for memory.h... yes 2024-04-13T05:19:38.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.741+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3680055b0 con 0x7fc388101e60 2024-04-13T05:19:38.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.742+0000 7fc384c00640 1 -- 172.21.15.57:0/432061328 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11855 (secure 0 0 0) 0x7fc37809c3b0 con 0x7fc388101e60 2024-04-13T05:19:38.745 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:19:38.745 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:17:49.577177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:19:38.745 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc36c0a3c30 msgr2=0x7fc36c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc36c0a3c30 0x7fc36c0a60f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc37402c240 tx=0x7fc374005fd0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 msgr2=0x7fc388075d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 0x7fc388075d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc37800d770 tx=0x7fc37800dc30 comp rx=0 tx=0).stop 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 shutdown_connections 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc36c0a3c30 0x7fc36c0a60f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388103970 0x7fc3880748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc388102fb0 0x7fc3880743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 --2- 172.21.15.57:0/432061328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc388101e60 0x7fc388075d20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.745+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 >> 172.21.15.57:0/432061328 conn(0x7fc3880fd3d0 msgr2=0x7fc3881064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.746+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 shutdown_connections 2024-04-13T05:19:38.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.746+0000 7fc38d400640 1 -- 172.21.15.57:0/432061328 wait complete. 2024-04-13T05:19:38.767 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:19:38.916 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strings.h... yes 2024-04-13T05:19:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 -- 172.21.15.57:0/465857435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4448107d00 msgr2=0x7f444810a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 --2- 172.21.15.57:0/465857435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4448107d00 0x7f444810a150 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4440009ca0 tx=0x7f444002c100 comp rx=0 tx=0).stop 2024-04-13T05:19:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 -- 172.21.15.57:0/465857435 shutdown_connections 2024-04-13T05:19:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 --2- 172.21.15.57:0/465857435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44481053d0 0x7f44481077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 --2- 172.21.15.57:0/465857435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4448107d00 0x7f444810a150 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 --2- 172.21.15.57:0/465857435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4448102aa0 0x7f4448104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 -- 172.21.15.57:0/465857435 >> 172.21.15.57:0/465857435 conn(0x7f44480fc4b0 msgr2=0x7f44480fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 -- 172.21.15.57:0/465857435 shutdown_connections 2024-04-13T05:19:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.971+0000 7f444fc00640 1 -- 172.21.15.57:0/465857435 wait complete. 2024-04-13T05:19:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.972+0000 7f444fc00640 1 Processor -- start 2024-04-13T05:19:38.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.972+0000 7f444fc00640 1 -- start start 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444fc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4448102aa0 0x7f4448195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444fc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 0x7f4448196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4448107d00 0x7f44481a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444fc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f444810db60 con 0x7f44481053d0 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f444810d9e0 con 0x7f4448107d00 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444fc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f444810dce0 con 0x7f4448102aa0 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f4447e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 0x7f4448196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f4447e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 0x7f4448196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59492/0 (socket says 172.21.15.57:59492) 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f4447e00640 1 -- 172.21.15.57:0/3252410012 learned_addr learned my addr 172.21.15.57:0/3252410012 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444d200640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4448102aa0 0x7f4448195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.973+0000 7f444dc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4448107d00 0x7f44481a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4447e00640 1 -- 172.21.15.57:0/3252410012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4448102aa0 msgr2=0x7f4448195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4447e00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4448102aa0 0x7f4448195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4447e00640 1 -- 172.21.15.57:0/3252410012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4448107d00 msgr2=0x7f44481a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4447e00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4448107d00 0x7f44481a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4447e00640 1 -- 172.21.15.57:0/3252410012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44481a6430 con 0x7f44481053d0 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f444dc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4448107d00 0x7f44481a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4447e00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 0x7f4448196520 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f443400c9a0 tx=0x7f443400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4445600640 1 -- 172.21.15.57:0/3252410012 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434014070 con 0x7f44481053d0 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4445600640 1 -- 172.21.15.57:0/3252410012 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4434004440 con 0x7f44481053d0 2024-04-13T05:19:38.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44481a6720 con 0x7f44481053d0 2024-04-13T05:19:38.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f4445600640 1 -- 172.21.15.57:0/3252410012 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443400f450 con 0x7f44481053d0 2024-04-13T05:19:38.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.974+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481a6c70 con 0x7f44481053d0 2024-04-13T05:19:38.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 Processor -- start 2024-04-13T05:19:38.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 -- start start 2024-04-13T05:19:38.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4445600640 1 -- 172.21.15.57:0/3252410012 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4434017070 con 0x7f44481053d0 2024-04-13T05:19:38.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c5d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c60240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5567b7bbe020 con 0x5567b7c58250 2024-04-13T05:19:38.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567b7b64950 con 0x5567b7c5ddb0 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.978+0000 7f4452b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567b7bb9fe0 con 0x5567b7c58c40 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c58670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59494/0 (socket says 172.21.15.57:59494) 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 -- 172.21.15.57:0/3460581346 learned_addr learned my addr 172.21.15.57:0/3460581346 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444d200640 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c5d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f4447e00640 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c60240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 -- 172.21.15.57:0/3460581346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 msgr2=0x5567b7c5d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c5d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 -- 172.21.15.57:0/3460581346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 msgr2=0x5567b7c60240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c60240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.979+0000 7f444dc00640 1 -- 172.21.15.57:0/3460581346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b78d03f0 con 0x5567b7c58250 2024-04-13T05:19:38.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.980+0000 7f444dc00640 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c58670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4440005e30 tx=0x7f4440005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cf146bcd524911 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.980+0000 7f443a000640 1 -- 172.21.15.57:0/3460581346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440039050 con 0x5567b7c58250 2024-04-13T05:19:38.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.980+0000 7f443a000640 1 -- 172.21.15.57:0/3460581346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4440045070 con 0x5567b7c58250 2024-04-13T05:19:38.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.980+0000 7f443a000640 1 -- 172.21.15.57:0/3460581346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400042a0 con 0x5567b7c58250 2024-04-13T05:19:38.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.980+0000 7f4445600640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44300a3bb0 0x7f44300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4445600640 1 -- 172.21.15.57:0/3252410012 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f44340d03f0 con 0x7f44481053d0 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f444d200640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44300a3bb0 0x7f44300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 -- 172.21.15.57:0/3460581346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 msgr2=0x5567b7c58670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c58670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4440005e30 tx=0x7f4440005fa0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 -- 172.21.15.57:0/3460581346 shutdown_connections 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c60240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c5d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 --2- 172.21.15.57:0/3460581346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c58670 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 -- 172.21.15.57:0/3460581346 >> 172.21.15.57:0/3460581346 conn(0x5567b7c537c0 msgr2=0x5567b7c53bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 -- 172.21.15.57:0/3460581346 shutdown_connections 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.981+0000 7f4452b67740 1 -- 172.21.15.57:0/3460581346 wait complete. 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.982+0000 7f444d200640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44300a3bb0 0x7f44300a6070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5567b7c54730 tx=0x7f443c00b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.982+0000 7f4452b67740 1 -- start start 2024-04-13T05:19:38.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.982+0000 7f4452b67740 1 Processor -- start 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.982+0000 7f4452b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.982+0000 7f4452b67740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4452b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c69ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4452b67740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5567b7c60d00 con 0x5567b7c58c40 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4452b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567b7b77c80 con 0x5567b7c5ddb0 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4452b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567b7c60e80 con 0x5567b7c58250 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4452b67740 10 client.0 _ll_register_callbacks cb 0x5567b7b77670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c62c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59510/0 (socket says 172.21.15.57:59510) 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 -- 172.21.15.57:0/3419592683 learned_addr learned my addr 172.21.15.57:0/3419592683 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f444d200640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f444dc00640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c69ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 -- 172.21.15.57:0/3419592683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 msgr2=0x5567b7c62720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c62720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 -- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 msgr2=0x5567b7c69ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c69ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 -- 172.21.15.57:0/3419592683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b799a7d0 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f4447e00640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c62c60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f443409c260 tx=0x7f4434098630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eed824ada5668d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.983+0000 7f442ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f442ea00640 1 -- 172.21.15.57:0/3419592683 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443400de20 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f442ea00640 1 -- 172.21.15.57:0/3419592683 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f44340035f0 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4440045070 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f442ea00640 1 -- 172.21.15.57:0/3419592683 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f443409c9d0 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 10 client.15879 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567b7c6d0e0 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f442ea00640 1 -- 172.21.15.57:0/3419592683 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4440045070 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f442ea00640 1 -- 172.21.15.57:0/3419592683 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f443409b720 con 0x5567b7c58c40 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 10 client.15879 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:19:38.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 20 client.15879 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:38.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.984+0000 7f4452b67740 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7c6e7a0 0x5567b7c70bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:38.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.985+0000 7f4452b67740 4 client.15879 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:38.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.985+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b7b5d750 con 0x5567b7c6e7a0 2024-04-13T05:19:38.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.985+0000 7f444d200640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7c6e7a0 0x5567b7c70bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:38.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.986+0000 7f444d200640 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7c6e7a0 0x5567b7c70bf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=808da2eda35aa3 server_cookie=d67228d567e87aac in_seq=0 out_seq=0 2024-04-13T05:19:38.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.986+0000 7f442ea00640 10 client.15879 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:38.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f442ea00640 1 -- 172.21.15.57:0/3419592683 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567b7b5d750 con 0x5567b7c6e7a0 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f442ea00640 10 client.15879 handle_command_reply: tid=0 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 client.15879 shutdown 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 msgr2=0x5567b7c62c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c62c60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f443409c260 tx=0x7f4434098630 comp rx=0 tx=0).stop 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 shutdown_connections 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7c6e7a0 0x5567b7c70bf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c69ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c62c60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 --2- 172.21.15.57:0/3419592683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 >> 172.21.15.57:0/3419592683 conn(0x5567b7c537c0 msgr2=0x5567b7c52a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f442ea00640 0 client.15879 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 shutdown_connections 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.989+0000 7f4452b67740 1 -- 172.21.15.57:0/3419592683 wait complete. 2024-04-13T05:19:38.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:38.990+0000 7f4452b67740 20 client.15879 trim_cache size 0 max 16384 2024-04-13T05:19:39.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 Processor -- start 2024-04-13T05:19:39.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 -- start start 2024-04-13T05:19:39.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c52730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f44481b39e0 con 0x5567b7c58250 2024-04-13T05:19:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44481b12b0 con 0x5567b7c5ddb0 2024-04-13T05:19:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.009+0000 7f4452b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44481af100 con 0x5567b7c58c40 2024-04-13T05:19:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.010+0000 7f444d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.010+0000 7f444d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59520/0 (socket says 172.21.15.57:59520) 2024-04-13T05:19:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.010+0000 7f444d200640 1 -- 172.21.15.57:0/983418461 learned_addr learned my addr 172.21.15.57:0/983418461 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.010+0000 7f4447e00640 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.011+0000 7f444d200640 1 -- 172.21.15.57:0/983418461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 msgr2=0x5567b7c69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.011+0000 7f444d200640 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c69c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.011+0000 7f444d200640 1 -- 172.21.15.57:0/983418461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 msgr2=0x5567b7c52730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.011+0000 7f444d200640 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c52730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.011+0000 7f444d200640 1 -- 172.21.15.57:0/983418461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b7c5d420 con 0x5567b7c58250 2024-04-13T05:19:39.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.012+0000 7f444d200640 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c521f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f443c005fd0 tx=0x7f443c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f99b33969382af server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.013+0000 7f4439600640 1 -- 172.21.15.57:0/983418461 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c012da0 con 0x5567b7c58250 2024-04-13T05:19:39.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.013+0000 7f4439600640 1 -- 172.21.15.57:0/983418461 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f443c02e070 con 0x5567b7c58250 2024-04-13T05:19:39.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.013+0000 7f4439600640 1 -- 172.21.15.57:0/983418461 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c01fbf0 con 0x5567b7c58250 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 -- 172.21.15.57:0/983418461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 msgr2=0x5567b7c521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c521f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f443c005fd0 tx=0x7f443c014000 comp rx=0 tx=0).stop 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 -- 172.21.15.57:0/983418461 shutdown_connections 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c52730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58c40 0x5567b7c69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 --2- 172.21.15.57:0/983418461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58250 0x5567b7c521f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 -- 172.21.15.57:0/983418461 >> 172.21.15.57:0/983418461 conn(0x5567b7c537c0 msgr2=0x5567b7c62720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 -- 172.21.15.57:0/983418461 shutdown_connections 2024-04-13T05:19:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.014+0000 7f4452b67740 1 -- 172.21.15.57:0/983418461 wait complete. 2024-04-13T05:19:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.015+0000 7f4452b67740 1 -- start start 2024-04-13T05:19:39.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.015+0000 7f4452b67740 1 Processor -- start 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c67790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c68750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f44481af770 con 0x5567b7c58c40 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44481b0be0 con 0x5567b7c5ddb0 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44481afe40 con 0x5567b7c58250 2024-04-13T05:19:39.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4452b67740 10 client.0 _ll_register_callbacks cb 0x7f4434097e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4447e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c67790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4447e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c67790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59524/0 (socket says 172.21.15.57:59524) 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.016+0000 7f4447e00640 1 -- 172.21.15.57:0/1634499191 learned_addr learned my addr 172.21.15.57:0/1634499191 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4447e00640 1 -- 172.21.15.57:0/1634499191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 msgr2=0x5567b7c67250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4447e00640 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4447e00640 1 -- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 msgr2=0x5567b7c68750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4447e00640 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c68750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4447e00640 1 -- 172.21.15.57:0/1634499191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b7c68d40 con 0x5567b7c58c40 2024-04-13T05:19:39.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4447e00640 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c67790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4434098c50 tx=0x7f44340d3d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8cc16dfbaced70 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f442ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f442ea00640 1 -- 172.21.15.57:0/1634499191 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434098310 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f442ea00640 1 -- 172.21.15.57:0/1634499191 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f44340a0150 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.017+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443c02e070 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f442ea00640 1 -- 172.21.15.57:0/1634499191 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44340a4f40 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 10 client.15885 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5567b7cd4d30 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f442ea00640 1 -- 172.21.15.57:0/1634499191 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f443c02e070 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f442ea00640 1 -- 172.21.15.57:0/1634499191 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4434063030 con 0x5567b7c58c40 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 10 client.15885 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 10 client.15885 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 20 client.15885 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7cd5610 0x5567b7cd7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 4 client.15885 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:19:39.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.018+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567b7c62510 con 0x5567b7cd5610 2024-04-13T05:19:39.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.019+0000 7f444d200640 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7cd5610 0x5567b7cd7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.021+0000 7f444d200640 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7cd5610 0x5567b7cd7a50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffa02daa5baf4b21 server_cookie=c07246a09b2fe1c9 in_seq=0 out_seq=0 2024-04-13T05:19:39.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.021+0000 7f442ea00640 10 client.15885 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:39.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.023+0000 7f442ea00640 1 -- 172.21.15.57:0/1634499191 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567b7c62510 con 0x5567b7cd5610 2024-04-13T05:19:39.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.023+0000 7f442ea00640 10 client.15885 handle_command_reply: tid=0 2024-04-13T05:19:39.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.023+0000 7f4452b67740 1 client.15885 shutdown 2024-04-13T05:19:39.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 msgr2=0x5567b7c67790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c67790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4434098c50 tx=0x7f44340d3d50 comp rx=0 tx=0).stop 2024-04-13T05:19:39.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 shutdown_connections 2024-04-13T05:19:39.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5567b7cd5610 0x5567b7cd7a50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567b7c5ddb0 0x5567b7c68750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5567b7c58c40 0x5567b7c67790 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 --2- 172.21.15.57:0/1634499191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567b7c58250 0x5567b7c67250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 >> 172.21.15.57:0/1634499191 conn(0x5567b7c537c0 msgr2=0x5567b7c56a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.024+0000 7f442ea00640 0 client.15885 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:19:39.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.025+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 shutdown_connections 2024-04-13T05:19:39.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.025+0000 7f4452b67740 1 -- 172.21.15.57:0/1634499191 wait complete. 2024-04-13T05:19:39.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.025+0000 7f4452b67740 20 client.15885 trim_cache size 0 max 16384 2024-04-13T05:19:39.028 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:19:39.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44300a3bb0 msgr2=0x7f44300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44300a3bb0 0x7f44300a6070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5567b7c54730 tx=0x7f443c00b650 comp rx=0 tx=0).stop 2024-04-13T05:19:39.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 msgr2=0x7f4448196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 0x7f4448196520 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f443400c9a0 tx=0x7f443400ce60 comp rx=0 tx=0).stop 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 shutdown_connections 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f44300a3bb0 0x7f44300a6070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.027+0000 7f444fc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4448107d00 0x7f44481a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.028+0000 7f444fc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44481053d0 0x7f4448196520 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.028+0000 7f444fc00640 1 --2- 172.21.15.57:0/3252410012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4448102aa0 0x7f4448195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.028+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 >> 172.21.15.57:0/3252410012 conn(0x7f44480fc4b0 msgr2=0x7f44481021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.028+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 shutdown_connections 2024-04-13T05:19:39.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.028+0000 7f444fc00640 1 -- 172.21.15.57:0/3252410012 wait complete. 2024-04-13T05:19:39.052 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:19:39.117 INFO:tasks.workunit.client.0.smithi057.stdout:checking for inttypes.h... yes 2024-04-13T05:19:39.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.240+0000 7f886d000640 1 -- 172.21.15.57:0/2056032251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88681030d0 msgr2=0x7f8868103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.240+0000 7f886d000640 1 --2- 172.21.15.57:0/2056032251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88681030d0 0x7f8868103550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8854009900 tx=0x7f885402bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 -- 172.21.15.57:0/2056032251 shutdown_connections 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 --2- 172.21.15.57:0/2056032251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8868103a90 0x7f886810a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 --2- 172.21.15.57:0/2056032251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88681030d0 0x7f8868103550 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 --2- 172.21.15.57:0/2056032251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8868101f00 0x7f8868102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 -- 172.21.15.57:0/2056032251 >> 172.21.15.57:0/2056032251 conn(0x7f88680fd4c0 msgr2=0x7f88680ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 -- 172.21.15.57:0/2056032251 shutdown_connections 2024-04-13T05:19:39.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.241+0000 7f886d000640 1 -- 172.21.15.57:0/2056032251 wait complete. 2024-04-13T05:19:39.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 Processor -- start 2024-04-13T05:19:39.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 -- start start 2024-04-13T05:19:39.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 0x7f8868075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88681030d0 0x7f88680743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8868103a90 0x7f88680748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f886810da30 con 0x7f8868101f00 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f886810d8b0 con 0x7f88681030d0 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f886d000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f886810dbb0 con 0x7f8868103a90 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f8867400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 0x7f8868075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f8867400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 0x7f8868075d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59550/0 (socket says 172.21.15.57:59550) 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.242+0000 7f8867400640 1 -- 172.21.15.57:0/2038013064 learned_addr learned my addr 172.21.15.57:0/2038013064 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8866a00640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88681030d0 0x7f88680743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8867400640 1 -- 172.21.15.57:0/2038013064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8868103a90 msgr2=0x7f88680748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8867400640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8868103a90 0x7f88680748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8867400640 1 -- 172.21.15.57:0/2038013064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88681030d0 msgr2=0x7f88680743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8867400640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88681030d0 0x7f88680743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8867400640 1 -- 172.21.15.57:0/2038013064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8868075180 con 0x7f8868101f00 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8866a00640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88681030d0 0x7f88680743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:39.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f8867400640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 0x7f8868075d70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f885800d770 tx=0x7f885800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f885fe00640 1 -- 172.21.15.57:0/2038013064 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858017070 con 0x7f8868101f00 2024-04-13T05:19:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f885fe00640 1 -- 172.21.15.57:0/2038013064 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8858009c30 con 0x7f8868101f00 2024-04-13T05:19:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.243+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88681a2220 con 0x7f8868101f00 2024-04-13T05:19:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.244+0000 7f885fe00640 1 -- 172.21.15.57:0/2038013064 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858003d00 con 0x7f8868101f00 2024-04-13T05:19:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.244+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88681a27c0 con 0x7f8868101f00 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 Processor -- start 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 -- start start 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f885fe00640 1 -- 172.21.15.57:0/2038013064 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8858003ea0 con 0x7f8868101f00 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295fe9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295fee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559295f4f020 con 0x559295feefa0 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559295ef5950 con 0x559295fe9440 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.247+0000 7f8871629740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559295f4afe0 con 0x559295fe9e30 2024-04-13T05:19:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59566/0 (socket says 172.21.15.57:59566) 2024-04-13T05:19:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 -- 172.21.15.57:0/315080203 learned_addr learned my addr 172.21.15.57:0/315080203 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8867e00640 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295fe9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8867400640 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295fee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 -- 172.21.15.57:0/315080203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 msgr2=0x559295fee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295fee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 -- 172.21.15.57:0/315080203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 msgr2=0x559295fe9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295fe9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.248+0000 7f8866a00640 1 -- 172.21.15.57:0/315080203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559295c613f0 con 0x559295feefa0 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f885fe00640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f884c0a3c30 0x7f884c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f8866a00640 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff1430 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8854009a20 tx=0x7f8854009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d585c8d5ae74fb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f885fe00640 1 -- 172.21.15.57:0/2038013064 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f88580d0560 con 0x7f8868101f00 2024-04-13T05:19:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f885cc00640 1 -- 172.21.15.57:0/315080203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854043020 con 0x559295feefa0 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f8867e00640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f884c0a3c30 0x7f884c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f885cc00640 1 -- 172.21.15.57:0/315080203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f885402ca60 con 0x559295feefa0 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.249+0000 7f885cc00640 1 -- 172.21.15.57:0/315080203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540046c0 con 0x559295feefa0 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 -- 172.21.15.57:0/315080203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 msgr2=0x559295ff1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8867e00640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f884c0a3c30 0x7f884c0a60f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8860009270 tx=0x7f8860009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff1430 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8854009a20 tx=0x7f8854009650 comp rx=0 tx=0).stop 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 -- 172.21.15.57:0/315080203 shutdown_connections 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff1430 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295fee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 --2- 172.21.15.57:0/315080203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295fe9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 -- 172.21.15.57:0/315080203 >> 172.21.15.57:0/315080203 conn(0x559295fe49b0 msgr2=0x559295fe4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 -- 172.21.15.57:0/315080203 shutdown_connections 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.250+0000 7f8871629740 1 -- 172.21.15.57:0/315080203 wait complete. 2024-04-13T05:19:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 -- start start 2024-04-13T05:19:39.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 Processor -- start 2024-04-13T05:19:39.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295ff3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295ff3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ffaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559295ff1ef0 con 0x559295feefa0 2024-04-13T05:19:39.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559295f08c80 con 0x559295fe9440 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.251+0000 7f8871629740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559295ff2070 con 0x559295fe9e30 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8871629740 10 client.0 _ll_register_callbacks cb 0x559295f08670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ffaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ffaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59580/0 (socket says 172.21.15.57:59580) 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 -- 172.21.15.57:0/3707881703 learned_addr learned my addr 172.21.15.57:0/3707881703 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867400640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295ff3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8866a00640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295ff3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 -- 172.21.15.57:0/3707881703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 msgr2=0x559295ff3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295ff3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 -- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 msgr2=0x559295ff3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295ff3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 -- 172.21.15.57:0/3707881703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559295d2b7d0 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.252+0000 7f8867e00640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ffaf00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8860009800 tx=0x7f8860009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10af38b4fcab5616 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8851600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8851600640 1 -- 172.21.15.57:0/3707881703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860023070 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8851600640 1 -- 172.21.15.57:0/3707881703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f88600119b0 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f885402ca60 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8851600640 1 -- 172.21.15.57:0/3707881703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f886002d050 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8871629740 10 client.15897 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.253+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559295ffe340 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8851600640 1 -- 172.21.15.57:0/3707881703 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f885402ca60 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8851600640 1 -- 172.21.15.57:0/3707881703 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f886000f040 con 0x559295feefa0 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8871629740 10 client.15897 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:39.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8871629740 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:19:39.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8871629740 20 client.15897 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:39.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8871629740 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559295fffab0 0x559296001f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8871629740 4 client.15897 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:39.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.254+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559295eee750 con 0x559295fffab0 2024-04-13T05:19:39.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.255+0000 7f8867400640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559295fffab0 0x559296001f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.256+0000 7f8867400640 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559295fffab0 0x559296001f30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c76df1bee882088 server_cookie=56161db9b39c24ab in_seq=0 out_seq=0 2024-04-13T05:19:39.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.256+0000 7f8851600640 10 client.15897 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:39.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.257+0000 7f8851600640 1 -- 172.21.15.57:0/3707881703 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559295eee750 con 0x559295fffab0 2024-04-13T05:19:39.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.257+0000 7f8851600640 10 client.15897 handle_command_reply: tid=0 2024-04-13T05:19:39.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.257+0000 7f8871629740 1 client.15897 shutdown 2024-04-13T05:19:39.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 msgr2=0x559295ffaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ffaf00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8860009800 tx=0x7f8860009200 comp rx=0 tx=0).stop 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 shutdown_connections 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559295fffab0 0x559296001f30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ffaf00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9e30 0x559295ff3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 --2- 172.21.15.57:0/3707881703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9440 0x559295ff3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 >> 172.21.15.57:0/3707881703 conn(0x559295fe49b0 msgr2=0x559295fe3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8851600640 0 client.15897 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 shutdown_connections 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 1 -- 172.21.15.57:0/3707881703 wait complete. 2024-04-13T05:19:39.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.258+0000 7f8871629740 20 client.15897 trim_cache size 0 max 16384 2024-04-13T05:19:39.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.274+0000 7f8871629740 1 Processor -- start 2024-04-13T05:19:39.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.274+0000 7f8871629740 1 -- start start 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8871629740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295fe3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8871629740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ffaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8871629740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295fe3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8871629740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f88681af520 con 0x559295feefa0 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8871629740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88681acdf0 con 0x559295fe9e30 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8871629740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88681aac40 con 0x559295fe9440 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295fe3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295fe3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59582/0 (socket says 172.21.15.57:59582) 2024-04-13T05:19:39.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 -- 172.21.15.57:0/3890254523 learned_addr learned my addr 172.21.15.57:0/3890254523 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867400640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295fe3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8866a00640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ffaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 -- 172.21.15.57:0/3890254523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 msgr2=0x559295fe3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295fe3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 -- 172.21.15.57:0/3890254523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 msgr2=0x559295ffaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ffaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867e00640 1 -- 172.21.15.57:0/3890254523 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559295fee690 con 0x559295feefa0 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8866a00640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ffaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.275+0000 7f8867400640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295fe3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:39.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.276+0000 7f8867e00640 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295fe3980 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8860005d70 tx=0x7f8860028840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711715eb80e594d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.276+0000 7f8853e00640 1 -- 172.21.15.57:0/3890254523 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860023070 con 0x559295feefa0 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.276+0000 7f8853e00640 1 -- 172.21.15.57:0/3890254523 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8860028de0 con 0x559295feefa0 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.276+0000 7f8853e00640 1 -- 172.21.15.57:0/3890254523 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886001f450 con 0x559295feefa0 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 -- 172.21.15.57:0/3890254523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 msgr2=0x559295fe3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295fe3980 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8860005d70 tx=0x7f8860028840 comp rx=0 tx=0).stop 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 -- 172.21.15.57:0/3890254523 shutdown_connections 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ffaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295fe3980 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 --2- 172.21.15.57:0/3890254523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295fe3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 -- 172.21.15.57:0/3890254523 >> 172.21.15.57:0/3890254523 conn(0x559295fe49b0 msgr2=0x559295feb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 -- 172.21.15.57:0/3890254523 shutdown_connections 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.277+0000 7f8871629740 1 -- 172.21.15.57:0/3890254523 wait complete. 2024-04-13T05:19:39.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 -- start start 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 Processor -- start 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295ff82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ff8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f88681ab2b0 con 0x559295feefa0 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88681ac720 con 0x559295fe9e30 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88681ab980 con 0x559295fe9440 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.278+0000 7f8871629740 10 client.0 _ll_register_callbacks cb 0x7f8860009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59586/0 (socket says 172.21.15.57:59586) 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 -- 172.21.15.57:0/2270743988 learned_addr learned my addr 172.21.15.57:0/2270743988 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8866a00640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ff8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 -- 172.21.15.57:0/2270743988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 msgr2=0x559295ff82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867400640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295ff82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295ff82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 -- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 msgr2=0x559295ff8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ff8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 -- 172.21.15.57:0/2270743988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559295ff9e30 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8867e00640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff9840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8860012fd0 tx=0x7f88600285d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2358ff00c2379f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.279+0000 7f8851600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8851600640 1 -- 172.21.15.57:0/2270743988 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860023040 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8851600640 1 -- 172.21.15.57:0/2270743988 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8860028ab0 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8860028de0 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8851600640 1 -- 172.21.15.57:0/2270743988 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f886004e050 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8871629740 10 client.15903 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559295ffb0a0 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8851600640 1 -- 172.21.15.57:0/2270743988 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8860028de0 con 0x559295feefa0 2024-04-13T05:19:39.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.280+0000 7f8851600640 1 -- 172.21.15.57:0/2270743988 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8860004640 con 0x559295feefa0 2024-04-13T05:19:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8871629740 10 client.15903 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8871629740 10 client.15903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:19:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8871629740 20 client.15903 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8871629740 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559296066670 0x559296068a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8871629740 4 client.15903 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:19:39.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559295f5ca40 con 0x559296066670 2024-04-13T05:19:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.281+0000 7f8867400640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559296066670 0x559296068a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.282+0000 7f8867400640 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559296066670 0x559296068a90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5af329dd38b7d66b server_cookie=d555ad20025ed691 in_seq=0 out_seq=0 2024-04-13T05:19:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.282+0000 7f8851600640 10 client.15903 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:39.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8851600640 1 -- 172.21.15.57:0/2270743988 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559295f5ca40 con 0x559296066670 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8851600640 10 client.15903 handle_command_reply: tid=0 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 client.15903 shutdown 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 msgr2=0x559295ff9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff9840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8860012fd0 tx=0x7f88600285d0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 shutdown_connections 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x559296066670 0x559296068a90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559295fe9e30 0x559295ff8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559295feefa0 0x559295ff9840 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8851600640 0 client.15903 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 --2- 172.21.15.57:0/2270743988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559295fe9440 0x559295ff82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 >> 172.21.15.57:0/2270743988 conn(0x559295fe49b0 msgr2=0x559295fe7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 shutdown_connections 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 1 -- 172.21.15.57:0/2270743988 wait complete. 2024-04-13T05:19:39.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.283+0000 7f8871629740 20 client.15903 trim_cache size 0 max 16384 2024-04-13T05:19:39.289 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:19:39.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f884c0a3c30 msgr2=0x7f884c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f884c0a3c30 0x7f884c0a60f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8860009270 tx=0x7f8860009500 comp rx=0 tx=0).stop 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 msgr2=0x7f8868075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 0x7f8868075d70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f885800d770 tx=0x7f885800dc30 comp rx=0 tx=0).stop 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 shutdown_connections 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f884c0a3c30 0x7f884c0a60f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88681030d0 0x7f88680743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8868103a90 0x7f88680748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 --2- 172.21.15.57:0/2038013064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8868101f00 0x7f8868075d70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 >> 172.21.15.57:0/2038013064 conn(0x7f88680fd4c0 msgr2=0x7f8868106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 shutdown_connections 2024-04-13T05:19:39.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.284+0000 7f886d000640 1 -- 172.21.15.57:0/2038013064 wait complete. 2024-04-13T05:19:39.300 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:39 smithi057 ceph-mon[32331]: pgmap v517: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-04-13T05:19:39.300 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:39 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/432061328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:19:39.302 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:19:39.304 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:39 smithi083 ceph-mon[35133]: pgmap v517: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-04-13T05:19:39.304 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:39 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/432061328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:19:39.325 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdint.h... yes 2024-04-13T05:19:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:39 smithi189 ceph-mon[35531]: pgmap v517: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-04-13T05:19:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:39 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/432061328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:19:39.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.501+0000 7fa59be00640 1 -- 172.21.15.57:0/1582139541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c105290 msgr2=0x7fa59c107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.501+0000 7fa59be00640 1 --2- 172.21.15.57:0/1582139541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c105290 0x7fa59c107680 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009900 tx=0x7fa58c02bc40 comp rx=0 tx=0).stop 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.501+0000 7fa59be00640 1 -- 172.21.15.57:0/1582139541 shutdown_connections 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.501+0000 7fa59be00640 1 --2- 172.21.15.57:0/1582139541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59c107bc0 0x7fa59c10a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.502+0000 7fa59be00640 1 --2- 172.21.15.57:0/1582139541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c105290 0x7fa59c107680 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.502+0000 7fa59be00640 1 --2- 172.21.15.57:0/1582139541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa59c102960 0x7fa59c104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.502+0000 7fa59be00640 1 -- 172.21.15.57:0/1582139541 >> 172.21.15.57:0/1582139541 conn(0x7fa59c0fc370 msgr2=0x7fa59c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.502+0000 7fa59be00640 1 -- 172.21.15.57:0/1582139541 shutdown_connections 2024-04-13T05:19:39.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.502+0000 7fa59be00640 1 -- 172.21.15.57:0/1582139541 wait complete. 2024-04-13T05:19:39.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 Processor -- start 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 -- start start 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 0x7fa59c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59c105290 0x7fa59c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa59c107bc0 0x7fa59c1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10da40 con 0x7fa59c102960 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10d8c0 con 0x7fa59c107bc0 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.503+0000 7fa59be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10dbc0 con 0x7fa59c105290 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 0x7fa59c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 0x7fa59c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59600/0 (socket says 172.21.15.57:59600) 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 -- 172.21.15.57:0/790254699 learned_addr learned my addr 172.21.15.57:0/790254699 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59a000640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59c105290 0x7fa59c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59b400640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa59c107bc0 0x7fa59c1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 -- 172.21.15.57:0/790254699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59c105290 msgr2=0x7fa59c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59c105290 0x7fa59c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 -- 172.21.15.57:0/790254699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa59c107bc0 msgr2=0x7fa59c1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa59c107bc0 0x7fa59c1a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.504+0000 7fa59aa00640 1 -- 172.21.15.57:0/790254699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c1a1e30 con 0x7fa59c102960 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.505+0000 7fa59aa00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 0x7fa59c19a210 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c920 tx=0x7fa58400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.505+0000 7fa593400640 1 -- 172.21.15.57:0/790254699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584007c70 con 0x7fa59c102960 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.505+0000 7fa593400640 1 -- 172.21.15.57:0/790254699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa584007e10 con 0x7fa59c102960 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.505+0000 7fa593400640 1 -- 172.21.15.57:0/790254699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584018820 con 0x7fa59c102960 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.505+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c1a2120 con 0x7fa59c102960 2024-04-13T05:19:39.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.505+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1a25f0 con 0x7fa59c102960 2024-04-13T05:19:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.508+0000 7fa5a4438740 1 Processor -- start 2024-04-13T05:19:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.508+0000 7fa5a4438740 1 -- start start 2024-04-13T05:19:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.508+0000 7fa593400640 1 -- 172.21.15.57:0/790254699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa5840189c0 con 0x7fa59c102960 2024-04-13T05:19:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bbe30 0x5590b51c0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa5a4438740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5590b5121020 con 0x5590b51c0fa0 2024-04-13T05:19:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa5a4438740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590b50c7950 con 0x5590b51bb440 2024-04-13T05:19:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa5a4438740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590b511cfe0 con 0x5590b51bbe30 2024-04-13T05:19:39.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa59a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa59a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59608/0 (socket says 172.21.15.57:59608) 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa59a000640 1 -- 172.21.15.57:0/396224893 learned_addr learned my addr 172.21.15.57:0/396224893 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa59b400640 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa59aa00640 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bbe30 0x5590b51c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.509+0000 7fa59a000640 1 -- 172.21.15.57:0/396224893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bbe30 msgr2=0x5590b51c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa59a000640 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bbe30 0x5590b51c0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa59a000640 1 -- 172.21.15.57:0/396224893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 msgr2=0x5590b51bb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa59a000640 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51bb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa59a000640 1 -- 172.21.15.57:0/396224893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b4e333f0 con 0x5590b51c0fa0 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa59aa00640 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bbe30 0x5590b51c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa59a000640 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51c3430 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa58c006010 tx=0x7fa58c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dc66fd7269d1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa593400640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5800a3c30 0x7fa5800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa593400640 1 -- 172.21.15.57:0/790254699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa5840d0990 con 0x7fa59c102960 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.510+0000 7fa58b400640 1 -- 172.21.15.57:0/396224893 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c044020 con 0x5590b51c0fa0 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa58b400640 1 -- 172.21.15.57:0/396224893 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa58c005040 con 0x5590b51c0fa0 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa58b400640 1 -- 172.21.15.57:0/396224893 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c004a40 con 0x5590b51c0fa0 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa59b400640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5800a3c30 0x7fa5800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 -- 172.21.15.57:0/396224893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 msgr2=0x5590b51c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51c3430 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa58c006010 tx=0x7fa58c009680 comp rx=0 tx=0).stop 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa59b400640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5800a3c30 0x7fa5800a60f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5590b51bc690 tx=0x7fa594008390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 -- 172.21.15.57:0/396224893 shutdown_connections 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51c3430 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bbe30 0x5590b51c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 --2- 172.21.15.57:0/396224893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51bb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 -- 172.21.15.57:0/396224893 >> 172.21.15.57:0/396224893 conn(0x5590b51b69b0 msgr2=0x5590b51b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.511+0000 7fa5a4438740 1 -- 172.21.15.57:0/396224893 shutdown_connections 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 -- 172.21.15.57:0/396224893 wait complete. 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 -- start start 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 Processor -- start 2024-04-13T05:19:39.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51c0fa0 0x5590b51ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.512+0000 7fa5a4438740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5590b51c3ef0 con 0x5590b51bbe30 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa5a4438740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590b50dac80 con 0x5590b51c0fa0 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa5a4438740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590b51c4070 con 0x5590b51bb440 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa5a4438740 10 client.0 _ll_register_callbacks cb 0x5590b50da670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59610/0 (socket says 172.21.15.57:59610) 2024-04-13T05:19:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 -- 172.21.15.57:0/3442153727 learned_addr learned my addr 172.21.15.57:0/3442153727 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59aa00640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59b400640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51c0fa0 0x5590b51ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 -- 172.21.15.57:0/3442153727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 msgr2=0x5590b51c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 -- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51c0fa0 msgr2=0x5590b51ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51c0fa0 0x5590b51ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.513+0000 7fa59a000640 1 -- 172.21.15.57:0/3442153727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b4efd7d0 con 0x5590b51bbe30 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa59a000640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51c5f00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa58c036040 tx=0x7fa58c02cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a328ea271a91b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa57fe00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa57fe00640 1 -- 172.21.15.57:0/3442153727 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c044030 con 0x5590b51bbe30 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa57fe00640 1 -- 172.21.15.57:0/3442153727 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa58c004c90 con 0x5590b51bbe30 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c005040 con 0x5590b51bbe30 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa57fe00640 1 -- 172.21.15.57:0/3442153727 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa58c05a050 con 0x5590b51bbe30 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa5a4438740 10 client.15915 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590b51d0220 con 0x5590b51bbe30 2024-04-13T05:19:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.514+0000 7fa57fe00640 1 -- 172.21.15.57:0/3442153727 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa58c005040 con 0x5590b51bbe30 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa57fe00640 1 -- 172.21.15.57:0/3442153727 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa58c049570 con 0x5590b51bbe30 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa5a4438740 10 client.15915 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa5a4438740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa5a4438740 20 client.15915 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa5a4438740 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b51d1570 0x5590b51d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa5a4438740 4 client.15915 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.515+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b50c0750 con 0x5590b51d1570 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.516+0000 7fa59aa00640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b51d1570 0x5590b51d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.517+0000 7fa59aa00640 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b51d1570 0x5590b51d3960 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433ab9d3fc127ed7 server_cookie=4a4ec6fc4a99cba9 in_seq=0 out_seq=0 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.517+0000 7fa57fe00640 10 client.15915 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa57fe00640 1 -- 172.21.15.57:0/3442153727 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590b50c0750 con 0x5590b51d1570 2024-04-13T05:19:39.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa57fe00640 10 client.15915 handle_command_reply: tid=0 2024-04-13T05:19:39.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 client.15915 shutdown 2024-04-13T05:19:39.522 INFO:tasks.workunit.client.0.smithi057.stdout:checking for unistd.h... yes 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 msgr2=0x5590b51c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51c5f00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa58c036040 tx=0x7fa58c02cd20 comp rx=0 tx=0).stop 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 shutdown_connections 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b51d1570 0x5590b51d3960 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51c0fa0 0x5590b51ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51c5f00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 --2- 172.21.15.57:0/3442153727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa57fe00640 0 client.15915 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 >> 172.21.15.57:0/3442153727 conn(0x5590b51b69b0 msgr2=0x5590b51b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 shutdown_connections 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 1 -- 172.21.15.57:0/3442153727 wait complete. 2024-04-13T05:19:39.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.519+0000 7fa5a4438740 20 client.15915 trim_cache size 0 max 16384 2024-04-13T05:19:39.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.534+0000 7fa5a4438740 1 Processor -- start 2024-04-13T05:19:39.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.534+0000 7fa5a4438740 1 -- start start 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bbe30 0x5590b51ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa5a4438740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c1af370 con 0x5590b51c0fa0 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa5a4438740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c1acc40 con 0x5590b51bbe30 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa5a4438740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c1aaa90 con 0x5590b51bb440 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59626/0 (socket says 172.21.15.57:59626) 2024-04-13T05:19:39.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 -- 172.21.15.57:0/1910888657 learned_addr learned my addr 172.21.15.57:0/1910888657 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59a000640 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bbe30 0x5590b51ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59aa00640 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 -- 172.21.15.57:0/1910888657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 msgr2=0x5590b51b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 -- 172.21.15.57:0/1910888657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bbe30 msgr2=0x5590b51ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bbe30 0x5590b51ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.535+0000 7fa59b400640 1 -- 172.21.15.57:0/1910888657 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b51c41b0 con 0x5590b51c0fa0 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.536+0000 7fa59b400640 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51b5980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa594008fd0 tx=0x7fa594012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d184ff7da4741 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.536+0000 7fa58aa00640 1 -- 172.21.15.57:0/1910888657 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59401c900 con 0x5590b51c0fa0 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.536+0000 7fa58aa00640 1 -- 172.21.15.57:0/1910888657 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa59401d420 con 0x5590b51c0fa0 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.536+0000 7fa58aa00640 1 -- 172.21.15.57:0/1910888657 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59401a670 con 0x5590b51c0fa0 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 -- 172.21.15.57:0/1910888657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 msgr2=0x5590b51b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51b5980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa594008fd0 tx=0x7fa594012000 comp rx=0 tx=0).stop 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 -- 172.21.15.57:0/1910888657 shutdown_connections 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bbe30 0x5590b51ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51c0fa0 0x5590b51b5980 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1910888657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51bb440 0x5590b51b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 -- 172.21.15.57:0/1910888657 >> 172.21.15.57:0/1910888657 conn(0x5590b51b69b0 msgr2=0x5590b51bd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 -- 172.21.15.57:0/1910888657 shutdown_connections 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 -- 172.21.15.57:0/1910888657 wait complete. 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.537+0000 7fa5a4438740 1 -- start start 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 Processor -- start 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51c0fa0 0x5590b51cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c1ab100 con 0x5590b51bbe30 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c1ac570 con 0x5590b51bb440 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c1ab7d0 con 0x5590b51c0fa0 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa5a4438740 10 client.0 _ll_register_callbacks cb 0x7fa58c0055d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa59a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa59a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59630/0 (socket says 172.21.15.57:59630) 2024-04-13T05:19:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.538+0000 7fa59a000640 1 -- 172.21.15.57:0/1134838052 learned_addr learned my addr 172.21.15.57:0/1134838052 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59aa00640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59a000640 1 -- 172.21.15.57:0/1134838052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51c0fa0 msgr2=0x5590b51cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59b400640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51c0fa0 0x5590b51cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59a000640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51c0fa0 0x5590b51cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59a000640 1 -- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 msgr2=0x5590b51ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59a000640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59a000640 1 -- 172.21.15.57:0/1134838052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b51cc110 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa59a000640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51cab10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x5590b51b78f0 tx=0x7fa58c049660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc744a11fb88744b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa57fe00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa57fe00640 1 -- 172.21.15.57:0/1134838052 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c044040 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59401d420 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.539+0000 7fa57fe00640 1 -- 172.21.15.57:0/1134838052 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa58c004030 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa57fe00640 1 -- 172.21.15.57:0/1134838052 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa58c066050 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 10 client.15921 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590b5237a70 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa57fe00640 1 -- 172.21.15.57:0/1134838052 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa59401d420 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa57fe00640 1 -- 172.21.15.57:0/1134838052 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa58c0035f0 con 0x5590b51bbe30 2024-04-13T05:19:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 10 client.15921 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 20 client.15921 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b52382b0 0x5590b523a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 4 client.15921 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.540+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590b512ea40 con 0x5590b52382b0 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.541+0000 7fa59aa00640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b52382b0 0x5590b523a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.542+0000 7fa59aa00640 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b52382b0 0x5590b523a6d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc69d27f9d512775 server_cookie=62310b81fb12868b in_seq=0 out_seq=0 2024-04-13T05:19:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.542+0000 7fa57fe00640 10 client.15921 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.545+0000 7fa57fe00640 1 -- 172.21.15.57:0/1134838052 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590b512ea40 con 0x5590b52382b0 2024-04-13T05:19:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.545+0000 7fa57fe00640 10 client.15921 handle_command_reply: tid=0 2024-04-13T05:19:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.545+0000 7fa5a4438740 1 client.15921 shutdown 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 msgr2=0x5590b51cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51cab10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x5590b51b78f0 tx=0x7fa58c049660 comp rx=0 tx=0).stop 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 shutdown_connections 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5590b52382b0 0x5590b523a6d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590b51c0fa0 0x5590b51cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5590b51bbe30 0x5590b51cab10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 --2- 172.21.15.57:0/1134838052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590b51bb440 0x5590b51ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 >> 172.21.15.57:0/1134838052 conn(0x5590b51b69b0 msgr2=0x5590b51b9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa57fe00640 0 client.15921 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 shutdown_connections 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 1 -- 172.21.15.57:0/1134838052 wait complete. 2024-04-13T05:19:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.546+0000 7fa5a4438740 20 client.15921 trim_cache size 0 max 16384 2024-04-13T05:19:39.550 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5800a3c30 msgr2=0x7fa5800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5800a3c30 0x7fa5800a60f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5590b51bc690 tx=0x7fa594008390 comp rx=0 tx=0).stop 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 msgr2=0x7fa59c19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 0x7fa59c19a210 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c920 tx=0x7fa58400cde0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 shutdown_connections 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5800a3c30 0x7fa5800a60f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa59c107bc0 0x7fa59c1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59c105290 0x7fa59c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 --2- 172.21.15.57:0/790254699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa59c102960 0x7fa59c19a210 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 >> 172.21.15.57:0/790254699 conn(0x7fa59c0fc370 msgr2=0x7fa59c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 shutdown_connections 2024-04-13T05:19:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:39.547+0000 7fa59be00640 1 -- 172.21.15.57:0/790254699 wait complete. 2024-04-13T05:19:39.575 INFO:tasks.workunit.client.0.smithi057.stdout:checking for memory.h... (cached) yes 2024-04-13T05:19:39.607 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strings.h... (cached) yes 2024-04-13T05:19:39.646 INFO:tasks.workunit.client.0.smithi057.stdout:checking for unistd.h... (cached) yes 2024-04-13T05:19:39.859 INFO:tasks.workunit.client.0.smithi057.stdout:checking locale.h usability... yes 2024-04-13T05:19:39.982 INFO:tasks.workunit.client.0.smithi057.stdout:checking locale.h presence... yes 2024-04-13T05:19:39.987 INFO:tasks.workunit.client.0.smithi057.stdout:checking for locale.h... yes 2024-04-13T05:19:40.080 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdlib.h... (cached) yes 2024-04-13T05:19:40.117 INFO:tasks.workunit.client.0.smithi057.stdout:checking for string.h... (cached) yes 2024-04-13T05:19:40.355 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/socket.h usability... yes 2024-04-13T05:19:40.468 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/socket.h presence... yes 2024-04-13T05:19:40.475 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/socket.h... yes 2024-04-13T05:19:40.567 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:19:40.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:19:40.567 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:19:40.736 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/time.h usability... yes 2024-04-13T05:19:40.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 -- 172.21.15.57:0/1200252827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda84104100 msgr2=0x7fda84104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 --2- 172.21.15.57:0/1200252827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda84104100 0x7fda84104520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402bc30 comp rx=0 tx=0).stop 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 -- 172.21.15.57:0/1200252827 shutdown_connections 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 --2- 172.21.15.57:0/1200252827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda841052f0 0x7fda84105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 --2- 172.21.15.57:0/1200252827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda84105cb0 0x7fda8410c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 --2- 172.21.15.57:0/1200252827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda84104100 0x7fda84104520 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 -- 172.21.15.57:0/1200252827 >> 172.21.15.57:0/1200252827 conn(0x7fda840ff670 msgr2=0x7fda84101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 -- 172.21.15.57:0/1200252827 shutdown_connections 2024-04-13T05:19:40.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.805+0000 7fda89000640 1 -- 172.21.15.57:0/1200252827 wait complete. 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.806+0000 7fda89000640 1 Processor -- start 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.806+0000 7fda89000640 1 -- start start 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda89000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda84104100 0x7fda8419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda89000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 0x7fda8419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda89000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda84105cb0 0x7fda841a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda89000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410fc40 con 0x7fda841052f0 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda89000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410fac0 con 0x7fda84104100 2024-04-13T05:19:40.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda89000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410fdc0 con 0x7fda84105cb0 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda82a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 0x7fda8419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda82a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 0x7fda8419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59650/0 (socket says 172.21.15.57:59650) 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda82a00640 1 -- 172.21.15.57:0/1100537655 learned_addr learned my addr 172.21.15.57:0/1100537655 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.807+0000 7fda83400640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda84104100 0x7fda8419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda83e00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda84105cb0 0x7fda841a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda82a00640 1 -- 172.21.15.57:0/1100537655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda84105cb0 msgr2=0x7fda841a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda82a00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda84105cb0 0x7fda841a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda82a00640 1 -- 172.21.15.57:0/1100537655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda84104100 msgr2=0x7fda8419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda82a00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda84104100 0x7fda8419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda82a00640 1 -- 172.21.15.57:0/1100537655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda841a1ec0 con 0x7fda841052f0 2024-04-13T05:19:40.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda82a00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 0x7fda8419a7e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fda7000c920 tx=0x7fda7000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda7be00640 1 -- 172.21.15.57:0/1100537655 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70007c70 con 0x7fda841052f0 2024-04-13T05:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda7be00640 1 -- 172.21.15.57:0/1100537655 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda70007e10 con 0x7fda841052f0 2024-04-13T05:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda7be00640 1 -- 172.21.15.57:0/1100537655 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70018820 con 0x7fda841052f0 2024-04-13T05:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.808+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda841a21b0 con 0x7fda841052f0 2024-04-13T05:19:40.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.809+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841a26b0 con 0x7fda841052f0 2024-04-13T05:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.813+0000 7fda8d5a9740 1 Processor -- start 2024-04-13T05:19:40.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.813+0000 7fda8d5a9740 1 -- start start 2024-04-13T05:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.813+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353defc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353deff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56353de5dbc0 con 0x56353def7e10 2024-04-13T05:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56353de04660 con 0x56353def7420 2024-04-13T05:19:40.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56353de59b80 con 0x56353defcf80 2024-04-13T05:19:40.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda83400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353defc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda7be00640 1 -- 172.21.15.57:0/1100537655 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda700189c0 con 0x7fda841052f0 2024-04-13T05:19:40.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda83400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353defc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59652/0 (socket says 172.21.15.57:59652) 2024-04-13T05:19:40.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda83400640 1 -- 172.21.15.57:0/1350804860 learned_addr learned my addr 172.21.15.57:0/1350804860 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:40.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.814+0000 7fda83e00640 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83400640 1 -- 172.21.15.57:0/1350804860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 msgr2=0x56353deff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83400640 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353deff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83400640 1 -- 172.21.15.57:0/1350804860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 msgr2=0x56353def7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83400640 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83400640 1 -- 172.21.15.57:0/1350804860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56353de27220 con 0x56353def7e10 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83e00640 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda83400640 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353defc910 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda74005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686f047aff63efec server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.815+0000 7fda78c00640 1 -- 172.21.15.57:0/1350804860 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74044020 con 0x56353def7e10 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda78c00640 1 -- 172.21.15.57:0/1350804860 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda74005040 con 0x56353def7e10 2024-04-13T05:19:40.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda78c00640 1 -- 172.21.15.57:0/1350804860 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740048c0 con 0x56353def7e10 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda7be00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fda680a3bb0 0x7fda680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda7be00640 1 -- 172.21.15.57:0/1100537655 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fda700d09e0 con 0x7fda841052f0 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1350804860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 msgr2=0x56353defc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353defc910 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda74005e80 comp rx=0 tx=0).stop 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.816+0000 7fda83e00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fda680a3bb0 0x7fda680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1350804860 shutdown_connections 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353deff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353defc910 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1350804860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1350804860 >> 172.21.15.57:0/1350804860 conn(0x56353def2990 msgr2=0x56353def2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1350804860 shutdown_connections 2024-04-13T05:19:40.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1350804860 wait complete. 2024-04-13T05:19:40.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda83e00640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fda680a3bb0 0x7fda680a6070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56353def8670 tx=0x7fda7c009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 -- start start 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.817+0000 7fda8d5a9740 1 Processor -- start 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353df01960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353def7e10 0x56353df01ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 0x56353df08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56353deffed0 con 0x56353defcf80 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56353de17a30 con 0x56353def7420 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56353df00050 con 0x56353def7e10 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda8d5a9740 10 client.0 _ll_register_callbacks cb 0x56353db6fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda83e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 0x56353df08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda83e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 0x56353df08ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59666/0 (socket says 172.21.15.57:59666) 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.818+0000 7fda83e00640 1 -- 172.21.15.57:0/4143107604 learned_addr learned my addr 172.21.15.57:0/4143107604 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:40.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83400640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353df01960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda82a00640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353def7e10 0x56353df01ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83e00640 1 -- 172.21.15.57:0/4143107604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353def7e10 msgr2=0x56353df01ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83e00640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353def7e10 0x56353df01ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83e00640 1 -- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 msgr2=0x56353df01960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83e00640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353df01960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83e00640 1 -- 172.21.15.57:0/4143107604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56353dc387e0 con 0x56353defcf80 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda83e00640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 0x56353df08ee0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fda7c012930 tx=0x7fda7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0841c6140b92d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda6d600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda6d600640 1 -- 172.21.15.57:0/4143107604 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00ece0 con 0x56353defcf80 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda6d600640 1 -- 172.21.15.57:0/4143107604 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fda7c01f420 con 0x56353defcf80 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.819+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda74005040 con 0x56353defcf80 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda6d600640 1 -- 172.21.15.57:0/4143107604 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda7c01f620 con 0x56353defcf80 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 10 client.15933 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56353dc98f70 con 0x56353defcf80 2024-04-13T05:19:40.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda6d600640 1 -- 172.21.15.57:0/4143107604 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda74005040 con 0x56353defcf80 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda6d600640 1 -- 172.21.15.57:0/4143107604 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fda7c00f040 con 0x56353defcf80 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 10 client.15933 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 20 client.15933 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df0d830 0x56353df0fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 4 client.15933 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.820+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56353de6b5e0 con 0x56353df0d830 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.821+0000 7fda83400640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df0d830 0x56353df0fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.822+0000 7fda83400640 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df0d830 0x56353df0fcd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bebf797d214922 server_cookie=2538f8aa69b7889e in_seq=0 out_seq=0 2024-04-13T05:19:40.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.822+0000 7fda6d600640 10 client.15933 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:40.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda6d600640 1 -- 172.21.15.57:0/4143107604 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56353de6b5e0 con 0x56353df0d830 2024-04-13T05:19:40.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda6d600640 10 client.15933 handle_command_reply: tid=0 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 client.15933 shutdown 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 msgr2=0x56353df08ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 0x56353df08ee0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fda7c012930 tx=0x7fda7c004a40 comp rx=0 tx=0).stop 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 shutdown_connections 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df0d830 0x56353df0fcd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353defcf80 0x56353df08ee0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353def7e10 0x56353df01ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/4143107604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353df01960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 >> 172.21.15.57:0/4143107604 conn(0x56353def2990 msgr2=0x56353def1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda6d600640 0 client.15933 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 shutdown_connections 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 1 -- 172.21.15.57:0/4143107604 wait complete. 2024-04-13T05:19:40.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.828+0000 7fda8d5a9740 20 client.15933 trim_cache size 0 max 16384 2024-04-13T05:19:40.835 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/time.h presence... yes 2024-04-13T05:19:40.839 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/time.h... yes 2024-04-13T05:19:40.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 Processor -- start 2024-04-13T05:19:40.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 -- start start 2024-04-13T05:19:40.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353df08ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fda841af430 con 0x56353def7e10 2024-04-13T05:19:40.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda841acd00 con 0x56353def7420 2024-04-13T05:19:40.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.849+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda841aab50 con 0x56353defcf80 2024-04-13T05:19:40.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353df08ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353df08ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59678/0 (socket says 172.21.15.57:59678) 2024-04-13T05:19:40.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 -- 172.21.15.57:0/1545919546 learned_addr learned my addr 172.21.15.57:0/1545919546 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:40.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda83400640 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda83e00640 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 -- 172.21.15.57:0/1545919546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 msgr2=0x56353df14590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df14590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 -- 172.21.15.57:0/1545919546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 msgr2=0x56353def6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.850+0000 7fda82a00640 1 -- 172.21.15.57:0/1545919546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56353db30170 con 0x56353def7e10 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.851+0000 7fda82a00640 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353df08ea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fda7009c5c0 tx=0x7fda70018cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5451121bdbf278d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.851+0000 7fda6fe00640 1 -- 172.21.15.57:0/1545919546 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7009c8c0 con 0x56353def7e10 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.851+0000 7fda6fe00640 1 -- 172.21.15.57:0/1545919546 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda70085e50 con 0x56353def7e10 2024-04-13T05:19:40.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.851+0000 7fda6fe00640 1 -- 172.21.15.57:0/1545919546 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700870c0 con 0x56353def7e10 2024-04-13T05:19:40.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1545919546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 msgr2=0x56353df08ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353df08ea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fda7009c5c0 tx=0x7fda70018cb0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1545919546 shutdown_connections 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7e10 0x56353df08ea0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/1545919546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7420 0x56353def6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1545919546 >> 172.21.15.57:0/1545919546 conn(0x56353def2990 msgr2=0x56353def5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1545919546 shutdown_connections 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 -- 172.21.15.57:0/1545919546 wait complete. 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 -- start start 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.852+0000 7fda8d5a9740 1 Processor -- start 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 0x56353df07010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7e10 0x56353df07550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fda841ab1c0 con 0x56353def7420 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda841ac630 con 0x56353def7e10 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda841ab890 con 0x56353defcf80 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda8d5a9740 10 client.0 _ll_register_callbacks cb 0x7fda7c011f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda83400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 0x56353df07010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda83400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 0x56353df07010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59686/0 (socket says 172.21.15.57:59686) 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.853+0000 7fda83400640 1 -- 172.21.15.57:0/663249589 learned_addr learned my addr 172.21.15.57:0/663249589 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda82a00640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7e10 0x56353df07550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83e00640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83400640 1 -- 172.21.15.57:0/663249589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 msgr2=0x56353df083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83400640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83400640 1 -- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7e10 msgr2=0x56353df07550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83400640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7e10 0x56353df07550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83400640 1 -- 172.21.15.57:0/663249589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56353de18be0 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83e00640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda83400640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 0x56353df07010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da66a71ea9349c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda6d600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda6d600640 1 -- 172.21.15.57:0/663249589 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74044040 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda6d600640 1 -- 172.21.15.57:0/663249589 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fda740054f0 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.854+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda70085e50 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda6d600640 1 -- 172.21.15.57:0/663249589 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda7402ca60 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 10 client.15939 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56353df7ce60 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda6d600640 1 -- 172.21.15.57:0/663249589 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda70085e50 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda6d600640 1 -- 172.21.15.57:0/663249589 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fda74003b10 con 0x56353def7420 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 10 client.15939 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:40.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 20 client.15939 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:40.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df7d3d0 0x56353df7f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:40.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 4 client.15939 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:19:40.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.855+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56353de65450 con 0x56353df7d3d0 2024-04-13T05:19:40.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.856+0000 7fda82a00640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df7d3d0 0x56353df7f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:40.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.857+0000 7fda82a00640 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df7d3d0 0x56353df7f7c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9019c58657894bbc server_cookie=12bad12a03094901 in_seq=0 out_seq=0 2024-04-13T05:19:40.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.857+0000 7fda6d600640 10 client.15939 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.881+0000 7fda6d600640 1 -- 172.21.15.57:0/663249589 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56353de65450 con 0x56353df7d3d0 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.881+0000 7fda6d600640 10 client.15939 handle_command_reply: tid=0 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.881+0000 7fda8d5a9740 1 client.15939 shutdown 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 msgr2=0x56353df07010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 0x56353df07010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402c8e0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 shutdown_connections 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56353df7d3d0 0x56353df7f7c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56353def7e10 0x56353df07550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56353defcf80 0x56353df083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 --2- 172.21.15.57:0/663249589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353def7420 0x56353df07010 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 >> 172.21.15.57:0/663249589 conn(0x56353def2990 msgr2=0x56353def6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:40.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda6d600640 0 client.15939 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:40.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 shutdown_connections 2024-04-13T05:19:40.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 1 -- 172.21.15.57:0/663249589 wait complete. 2024-04-13T05:19:40.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.882+0000 7fda8d5a9740 20 client.15939 trim_cache size 0 max 16384 2024-04-13T05:19:40.886 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:19:40.886 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:19:40.887 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "be4f2aaf-0aec-469a-b194-74d17d0a8283", 2024-04-13T05:19:40.887 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:19:40.887 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fda680a3bb0 msgr2=0x7fda680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fda680a3bb0 0x7fda680a6070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56353def8670 tx=0x7fda7c009580 comp rx=0 tx=0).stop 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 msgr2=0x7fda8419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 0x7fda8419a7e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fda7000c920 tx=0x7fda7000cde0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 shutdown_connections 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fda680a3bb0 0x7fda680a6070 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda84105cb0 0x7fda841a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda841052f0 0x7fda8419a7e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 --2- 172.21.15.57:0/1100537655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda84104100 0x7fda8419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.883+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 >> 172.21.15.57:0/1100537655 conn(0x7fda840ff670 msgr2=0x7fda841085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.884+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 shutdown_connections 2024-04-13T05:19:40.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:40.884+0000 7fda89000640 1 -- 172.21.15.57:0/1100537655 wait complete. 2024-04-13T05:19:40.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:19:40.908 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:19:41.072 INFO:tasks.workunit.client.0.smithi057.stdout:checking fcntl.h usability... yes 2024-04-13T05:19:41.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 -- 172.21.15.57:0/1319110876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 msgr2=0x7f297810c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 --2- 172.21.15.57:0/1319110876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f297810c680 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2970009ca0 tx=0x7f297002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 -- 172.21.15.57:0/1319110876 shutdown_connections 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 --2- 172.21.15.57:0/1319110876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f297810c680 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 --2- 172.21.15.57:0/1319110876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2978105430 0x7f29781058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 --2- 172.21.15.57:0/1319110876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2978104240 0x7f2978104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 -- 172.21.15.57:0/1319110876 >> 172.21.15.57:0/1319110876 conn(0x7f29780ff7b0 msgr2=0x7f2978101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 -- 172.21.15.57:0/1319110876 shutdown_connections 2024-04-13T05:19:41.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.139+0000 7f297d200640 1 -- 172.21.15.57:0/1319110876 wait complete. 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.140+0000 7f297d200640 1 Processor -- start 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.140+0000 7f297d200640 1 -- start start 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f297d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2978104240 0x7f297819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f297d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2978105430 0x7f297819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f297d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f29781a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f297d200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f297810fd80 con 0x7f2978105df0 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f297d200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f297810fc00 con 0x7f2978105430 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f297d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f297810ff00 con 0x7f2978104240 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f2976800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f29781a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f2976800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f29781a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59704/0 (socket says 172.21.15.57:59704) 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f2976800640 1 -- 172.21.15.57:0/3258797541 learned_addr learned my addr 172.21.15.57:0/3258797541 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:41.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f2975e00640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2978104240 0x7f297819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.141+0000 7f296fe00640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2978105430 0x7f297819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f2976800640 1 -- 172.21.15.57:0/3258797541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2978104240 msgr2=0x7f297819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f2976800640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2978104240 0x7f297819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f2976800640 1 -- 172.21.15.57:0/3258797541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2978105430 msgr2=0x7f297819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f2976800640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2978105430 0x7f297819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f2976800640 1 -- 172.21.15.57:0/3258797541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29781a1ff0 con 0x7f2978105df0 2024-04-13T05:19:41.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f2976800640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f29781a1980 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f297002c700 tx=0x7f29700328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:41.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f296ea00640 1 -- 172.21.15.57:0/3258797541 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970041020 con 0x7f2978105df0 2024-04-13T05:19:41.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29781a2280 con 0x7f2978105df0 2024-04-13T05:19:41.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.142+0000 7f296ea00640 1 -- 172.21.15.57:0/3258797541 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2970003e20 con 0x7f2978105df0 2024-04-13T05:19:41.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.143+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29781a2770 con 0x7f2978105df0 2024-04-13T05:19:41.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.143+0000 7f296ea00640 1 -- 172.21.15.57:0/3258797541 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970039040 con 0x7f2978105df0 2024-04-13T05:19:41.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.146+0000 7f2980232740 1 Processor -- start 2024-04-13T05:19:41.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.146+0000 7f296ea00640 1 -- 172.21.15.57:0/3258797541 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2970044020 con 0x7f2978105df0 2024-04-13T05:19:41.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 -- start start 2024-04-13T05:19:41.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43faeec0 0x55eb43fb39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fb4030 0x55eb43fb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55eb43f15020 con 0x55eb43fae4d0 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb43ebb6d0 con 0x55eb43fb4030 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f2980232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb43f10fe0 con 0x55eb43faeec0 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f296fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f296fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59712/0 (socket says 172.21.15.57:59712) 2024-04-13T05:19:41.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.147+0000 7f296fe00640 1 -- 172.21.15.57:0/2068652641 learned_addr learned my addr 172.21.15.57:0/2068652641 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f2975e00640 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43faeec0 0x55eb43fb39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fb4030 0x55eb43fb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 -- 172.21.15.57:0/2068652641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43faeec0 msgr2=0x55eb43fb39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43faeec0 0x55eb43fb39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 -- 172.21.15.57:0/2068652641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fb4030 msgr2=0x55eb43fb64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fb4030 0x55eb43fb64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 -- 172.21.15.57:0/2068652641 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb43cf17d0 con 0x55eb43fae4d0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296ea00640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f295c0a3bb0 0x7f295c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296fe00640 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fae8f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f29600135b0 tx=0x7f296000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38d31be674a94f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f296ea00640 1 -- 172.21.15.57:0/3258797541 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f29700f2c90 con 0x7f2978105df0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.148+0000 7f2975e00640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f295c0a3bb0 0x7f295c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.149+0000 7f2967400640 1 -- 172.21.15.57:0/2068652641 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960019030 con 0x55eb43fae4d0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.149+0000 7f2967400640 1 -- 172.21.15.57:0/2068652641 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2960011970 con 0x55eb43fae4d0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.149+0000 7f2967400640 1 -- 172.21.15.57:0/2068652641 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296001d780 con 0x55eb43fae4d0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.149+0000 7f2980232740 1 -- 172.21.15.57:0/2068652641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 msgr2=0x55eb43fae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.149+0000 7f2980232740 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fae8f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f29600135b0 tx=0x7f296000b690 comp rx=0 tx=0).stop 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.149+0000 7f2975e00640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f295c0a3bb0 0x7f295c0a6070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2968005fd0 tx=0x7f2968005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 -- 172.21.15.57:0/2068652641 shutdown_connections 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fb4030 0x55eb43fb64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43faeec0 0x55eb43fb39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 --2- 172.21.15.57:0/2068652641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fae8f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 -- 172.21.15.57:0/2068652641 >> 172.21.15.57:0/2068652641 conn(0x55eb43fa9a40 msgr2=0x55eb43fa9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:41.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 -- 172.21.15.57:0/2068652641 shutdown_connections 2024-04-13T05:19:41.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 -- 172.21.15.57:0/2068652641 wait complete. 2024-04-13T05:19:41.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 -- start start 2024-04-13T05:19:41.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.150+0000 7f2980232740 1 Processor -- start 2024-04-13T05:19:41.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fb87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55eb43fb7030 con 0x55eb43faeec0 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb43ecea90 con 0x55eb43fae4d0 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb43fb71b0 con 0x55eb43fb4030 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2980232740 10 client.0 _ll_register_callbacks cb 0x55eb43ece480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f296fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f296fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fb8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59720/0 (socket says 172.21.15.57:59720) 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f296fe00640 1 -- 172.21.15.57:0/2107942467 learned_addr learned my addr 172.21.15.57:0/2107942467 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.151+0000 7f2975e00640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fb87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f2976800640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f296fe00640 1 -- 172.21.15.57:0/2107942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 msgr2=0x55eb43fbfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f296fe00640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f296fe00640 1 -- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 msgr2=0x55eb43fb87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f296fe00640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fb87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f296fe00640 1 -- 172.21.15.57:0/2107942467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb43c273f0 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f296fe00640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fb8cf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f29600058c0 tx=0x7f29600118b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbb56b34185fd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f295be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.152+0000 7f295be00640 1 -- 172.21.15.57:0/2107942467 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960019040 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f295be00640 1 -- 172.21.15.57:0/2107942467 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2960011e20 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2960011970 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f295be00640 1 -- 172.21.15.57:0/2107942467 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2960030050 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f2980232740 10 client.15951 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb43be70f0 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f295be00640 1 -- 172.21.15.57:0/2107942467 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2960011970 con 0x55eb43faeec0 2024-04-13T05:19:41.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.153+0000 7f295be00640 1 -- 172.21.15.57:0/2107942467 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2960025020 con 0x55eb43faeec0 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2980232740 10 client.15951 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2980232740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2980232740 20 client.15951 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2980232740 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb43fc4380 0x55eb43fc67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2980232740 4 client.15951 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb43eb4600 con 0x55eb43fc4380 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.154+0000 7f2975e00640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb43fc4380 0x55eb43fc67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.156+0000 7f2975e00640 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb43fc4380 0x55eb43fc67b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53058dd9d76d62a server_cookie=ab809e8c64527739 in_seq=0 out_seq=0 2024-04-13T05:19:41.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.156+0000 7f295be00640 10 client.15951 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:41.160 INFO:tasks.workunit.client.0.smithi057.stdout:checking fcntl.h presence... yes 2024-04-13T05:19:41.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.158+0000 7f295be00640 1 -- 172.21.15.57:0/2107942467 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb43eb4600 con 0x55eb43fc4380 2024-04-13T05:19:41.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.158+0000 7f295be00640 10 client.15951 handle_command_reply: tid=0 2024-04-13T05:19:41.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.158+0000 7f2980232740 1 client.15951 shutdown 2024-04-13T05:19:41.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 msgr2=0x55eb43fb8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fb8cf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f29600058c0 tx=0x7f29600118b0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 shutdown_connections 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb43fc4380 0x55eb43fc67b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fb8cf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 --2- 172.21.15.57:0/2107942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fb87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f295be00640 0 client.15951 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 >> 172.21.15.57:0/2107942467 conn(0x55eb43fa9a40 msgr2=0x55eb43fa8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 shutdown_connections 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 1 -- 172.21.15.57:0/2107942467 wait complete. 2024-04-13T05:19:41.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.159+0000 7f2980232740 20 client.15951 trim_cache size 0 max 16384 2024-04-13T05:19:41.164 INFO:tasks.workunit.client.0.smithi057.stdout:checking for fcntl.h... yes 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.179+0000 7f2980232740 1 Processor -- start 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.179+0000 7f2980232740 1 -- start start 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2980232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fa82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2980232740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fbfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2980232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fa8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2980232740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f29781af5f0 con 0x55eb43faeec0 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2980232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29781acec0 con 0x55eb43fae4d0 2024-04-13T05:19:41.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2980232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29781aad10 con 0x55eb43fb4030 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fbfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fbfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59736/0 (socket says 172.21.15.57:59736) 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 -- 172.21.15.57:0/363098620 learned_addr learned my addr 172.21.15.57:0/363098620 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 -- 172.21.15.57:0/363098620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 msgr2=0x55eb43fa8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fa8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f2975e00640 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fa82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 -- 172.21.15.57:0/363098620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 msgr2=0x55eb43fa82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fa82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.180+0000 7f296fe00640 1 -- 172.21.15.57:0/363098620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb43fa4030 con 0x55eb43faeec0 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.181+0000 7f296fe00640 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fbfcf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2960015fa0 tx=0x7f296000bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ec5e9df6c40d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.181+0000 7f2966a00640 1 -- 172.21.15.57:0/363098620 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960019070 con 0x55eb43faeec0 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.181+0000 7f2966a00640 1 -- 172.21.15.57:0/363098620 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f296000b5f0 con 0x55eb43faeec0 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.181+0000 7f2966a00640 1 -- 172.21.15.57:0/363098620 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960011e00 con 0x55eb43faeec0 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 -- 172.21.15.57:0/363098620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 msgr2=0x55eb43fbfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fbfcf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2960015fa0 tx=0x7f296000bb80 comp rx=0 tx=0).stop 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 -- 172.21.15.57:0/363098620 shutdown_connections 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fa8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43faeec0 0x55eb43fbfcf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 --2- 172.21.15.57:0/363098620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43fae4d0 0x55eb43fa82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 -- 172.21.15.57:0/363098620 >> 172.21.15.57:0/363098620 conn(0x55eb43fa9a40 msgr2=0x55eb43fb01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 -- 172.21.15.57:0/363098620 shutdown_connections 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 -- 172.21.15.57:0/363098620 wait complete. 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.182+0000 7f2980232740 1 -- start start 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 Processor -- start 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fbd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43faeec0 0x55eb43fbdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f29781ab380 con 0x55eb43fae4d0 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29781ac7f0 con 0x55eb43faeec0 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29781aba50 con 0x55eb43fb4030 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2980232740 10 client.0 _ll_register_callbacks cb 0x7f29600114c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2975e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fbd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2975e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fbd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59744/0 (socket says 172.21.15.57:59744) 2024-04-13T05:19:41.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.183+0000 7f2975e00640 1 -- 172.21.15.57:0/4290815382 learned_addr learned my addr 172.21.15.57:0/4290815382 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f296fe00640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43faeec0 0x55eb43fbdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2975e00640 1 -- 172.21.15.57:0/4290815382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 msgr2=0x55eb43fbeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2976800640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2975e00640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2975e00640 1 -- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43faeec0 msgr2=0x55eb43fbdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2975e00640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43faeec0 0x55eb43fbdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2975e00640 1 -- 172.21.15.57:0/4290815382 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb43fbf100 con 0x55eb43fae4d0 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2975e00640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fbd5c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55eb43faf2c0 tx=0x7f29680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9878741eb63fd5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:19:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f295be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:19:41.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f295be00640 1 -- 172.21.15.57:0/4290815382 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968004020 con 0x55eb43fae4d0 2024-04-13T05:19:41.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f295be00640 1 -- 172.21.15.57:0/4290815382 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f29680041c0 con 0x55eb43fae4d0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.184+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296000b5f0 con 0x55eb43fae4d0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f295be00640 1 -- 172.21.15.57:0/4290815382 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2968037050 con 0x55eb43fae4d0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 10 client.15957 fetch_fsmap learned FSMap version 25 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb4402abe0 con 0x55eb43fae4d0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f295be00640 1 -- 172.21.15.57:0/4290815382 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f296000b5f0 con 0x55eb43fae4d0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f295be00640 1 -- 172.21.15.57:0/4290815382 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2968003990 con 0x55eb43fae4d0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 10 client.15957 fetch_fsmap finished waiting for FSMap version 25 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 20 client.15957 populate_metadata read hostname 'smithi057' 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb4402b610 0x55eb4402da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 4 client.15957 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.185+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb43f22a40 con 0x55eb4402b610 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.186+0000 7f296fe00640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb4402b610 0x55eb4402da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:19:41.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.187+0000 7f296fe00640 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb4402b610 0x55eb4402da30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e850091d9bd07b75 server_cookie=15718ee34ef20fe9 in_seq=0 out_seq=0 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.187+0000 7f295be00640 10 client.15957 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f295be00640 1 -- 172.21.15.57:0/4290815382 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eb43f22a40 con 0x55eb4402b610 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f295be00640 10 client.15957 handle_command_reply: tid=0 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 client.15957 shutdown 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 msgr2=0x55eb43fbd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fbd5c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55eb43faf2c0 tx=0x7f29680046c0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 shutdown_connections 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55eb4402b610 0x55eb4402da30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb43faeec0 0x55eb43fbdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb43fb4030 0x55eb43fbeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 --2- 172.21.15.57:0/4290815382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb43fae4d0 0x55eb43fbd5c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f295be00640 0 client.15957 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.188+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 >> 172.21.15.57:0/4290815382 conn(0x55eb43fa9a40 msgr2=0x55eb43facca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:41.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.189+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 shutdown_connections 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.189+0000 7f2980232740 1 -- 172.21.15.57:0/4290815382 wait complete. 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.189+0000 7f2980232740 20 client.15957 trim_cache size 0 max 16384 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout: "be4f2aaf-0aec-469a-b194-74d17d0a8283": { 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "be4f2aaf-0aec-469a-b194-74d17d0a8283", 2024-04-13T05:19:41.194 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:19:41.195 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:19:41.195 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:19:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f295c0a3bb0 msgr2=0x7f295c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f295c0a3bb0 0x7f295c0a6070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2968005fd0 tx=0x7f2968005c50 comp rx=0 tx=0).stop 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 msgr2=0x7f29781a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f29781a1980 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f297002c700 tx=0x7f29700328a0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 shutdown_connections 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f295c0a3bb0 0x7f295c0a6070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2978105430 0x7f297819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2978105df0 0x7f29781a1980 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 --2- 172.21.15.57:0/3258797541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2978104240 0x7f297819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 >> 172.21.15.57:0/3258797541 conn(0x7f29780ff7b0 msgr2=0x7f2978108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 shutdown_connections 2024-04-13T05:19:41.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:19:41.190+0000 7f297d200640 1 -- 172.21.15.57:0/3258797541 wait complete. 2024-04-13T05:19:41.211 INFO:tasks.cephfs.filesystem:scrub status for tag:be4f2aaf-0aec-469a-b194-74d17d0a8283 - {'path': '/', 'tag': 'be4f2aaf-0aec-469a-b194-74d17d0a8283', 'options': 'recursive,force'} 2024-04-13T05:19:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:41 smithi057 ceph-mon[32331]: pgmap v518: 129 pgs: 129 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 187 KiB/s wr, 33 op/s 2024-04-13T05:19:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:41 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:19:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:41 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:41 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:19:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:41 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:41 smithi189 ceph-mon[35531]: pgmap v518: 129 pgs: 129 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 187 KiB/s wr, 33 op/s 2024-04-13T05:19:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:41 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:19:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:41 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:41 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:19:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:41 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:41.432 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/fcntl.h usability... yes 2024-04-13T05:19:41.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:41 smithi083 ceph-mon[35133]: pgmap v518: 129 pgs: 129 active+clean; 2.6 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 187 KiB/s wr, 33 op/s 2024-04-13T05:19:41.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:41 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:19:41.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:41 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:41.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:41 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:19:41.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:41 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:19:41.536 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/fcntl.h presence... yes 2024-04-13T05:19:41.543 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/fcntl.h... yes 2024-04-13T05:19:41.866 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/param.h usability... yes 2024-04-13T05:19:41.983 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/param.h presence... yes 2024-04-13T05:19:41.988 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/param.h... yes 2024-04-13T05:19:42.193 INFO:tasks.workunit.client.0.smithi057.stdout:checking ioctl.h usability... no 2024-04-13T05:19:42.332 INFO:tasks.workunit.client.0.smithi057.stdout:checking ioctl.h presence... no 2024-04-13T05:19:42.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:42 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:19:42.338 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ioctl.h... no 2024-04-13T05:19:42.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:42 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:19:42.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:42 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:19:42.578 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/ioctl.h usability... yes 2024-04-13T05:19:42.683 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/ioctl.h presence... yes 2024-04-13T05:19:42.689 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/ioctl.h... yes 2024-04-13T05:19:42.939 INFO:tasks.workunit.client.0.smithi057.stdout:checking return type of signal handlers... void 2024-04-13T05:19:43.139 INFO:tasks.workunit.client.0.smithi057.stdout:checking for size_t... yes 2024-04-13T05:19:43.324 INFO:tasks.workunit.client.0.smithi057.stdout:checking for dev_t... yes 2024-04-13T05:19:43.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:43 smithi083 ceph-mon[35133]: pgmap v519: 129 pgs: 129 active+clean; 2.6 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 192 KiB/s wr, 36 op/s 2024-04-13T05:19:43.519 INFO:tasks.workunit.client.0.smithi057.stdout:checking for short... yes 2024-04-13T05:19:43.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:43 smithi057 ceph-mon[32331]: pgmap v519: 129 pgs: 129 active+clean; 2.6 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 192 KiB/s wr, 36 op/s 2024-04-13T05:19:43.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:43 smithi189 ceph-mon[35531]: pgmap v519: 129 pgs: 129 active+clean; 2.6 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 192 KiB/s wr, 36 op/s 2024-04-13T05:19:43.877 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of short... 2 2024-04-13T05:19:44.093 INFO:tasks.workunit.client.0.smithi057.stdout:checking for int... yes 2024-04-13T05:19:44.406 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of int... 4 2024-04-13T05:19:44.618 INFO:tasks.workunit.client.0.smithi057.stdout:checking for long... yes 2024-04-13T05:19:44.942 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of long... 8 2024-04-13T05:19:45.148 INFO:tasks.workunit.client.0.smithi057.stdout:checking for long long... yes 2024-04-13T05:19:45.474 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of long long... 8 2024-04-13T05:19:45.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:45 smithi083 ceph-mon[35133]: pgmap v520: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 198 KiB/s wr, 46 op/s 2024-04-13T05:19:45.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:45 smithi057 ceph-mon[32331]: pgmap v520: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 198 KiB/s wr, 46 op/s 2024-04-13T05:19:45.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:45 smithi189 ceph-mon[35531]: pgmap v520: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 198 KiB/s wr, 46 op/s 2024-04-13T05:19:45.685 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ssize_t... yes 2024-04-13T05:19:45.872 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sig_atomic_t... yes 2024-04-13T05:19:46.100 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether gcc needs -traditional... no 2024-04-13T05:19:46.282 INFO:tasks.workunit.client.0.smithi057.stdout:checking for an ANSI C-conforming const... yes 2024-04-13T05:19:46.420 INFO:tasks.workunit.client.0.smithi057.stdout:checking for inline... inline 2024-04-13T05:19:46.727 INFO:tasks.workunit.client.0.smithi057.stdout:checking for working alloca.h... yes 2024-04-13T05:19:47.029 INFO:tasks.workunit.client.0.smithi057.stdout:checking for alloca... yes 2024-04-13T05:19:47.287 INFO:tasks.workunit.client.0.smithi057.stdout:checking for working memcmp... yes 2024-04-13T05:19:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:47 smithi083 ceph-mon[35133]: pgmap v521: 129 pgs: 129 active+clean; 3.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 277 KiB/s wr, 60 op/s 2024-04-13T05:19:47.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:47 smithi057 ceph-mon[32331]: pgmap v521: 129 pgs: 129 active+clean; 3.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 277 KiB/s wr, 60 op/s 2024-04-13T05:19:47.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:47 smithi189 ceph-mon[35531]: pgmap v521: 129 pgs: 129 active+clean; 3.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 277 KiB/s wr, 60 op/s 2024-04-13T05:19:47.615 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strftime... yes 2024-04-13T05:19:47.972 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-04-13T05:19:48.284 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether stat accepts an empty string... no 2024-04-13T05:19:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:48 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:19:48.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:48 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:19:48.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:48 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:19:48.605 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vprintf... yes 2024-04-13T05:19:48.905 INFO:tasks.workunit.client.0.smithi057.stdout:checking for _doprnt... no 2024-04-13T05:19:49.016 INFO:tasks.workunit.client.0.smithi057.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-13T05:19:49.075 INFO:tasks.workunit.client.0.smithi057.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-13T05:19:49.288 INFO:tasks.workunit.client.0.smithi057.stdout:checking for the pthreads library -lpthreads... no 2024-04-13T05:19:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:49 smithi083 ceph-mon[35133]: pgmap v522: 129 pgs: 129 active+clean; 3.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 52 op/s 2024-04-13T05:19:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:49 smithi057 ceph-mon[32331]: pgmap v522: 129 pgs: 129 active+clean; 3.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 52 op/s 2024-04-13T05:19:49.576 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work without any flags... yes 2024-04-13T05:19:49.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:49 smithi189 ceph-mon[35531]: pgmap v522: 129 pgs: 129 active+clean; 3.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 52 op/s 2024-04-13T05:19:49.854 INFO:tasks.workunit.client.0.smithi057.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-04-13T05:19:49.856 INFO:tasks.workunit.client.0.smithi057.stdout:checking if more special flags are required for pthreads... no 2024-04-13T05:19:49.858 INFO:tasks.workunit.client.0.smithi057.stdout:checking for cc_r... gcc 2024-04-13T05:19:50.173 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setrlimit... yes 2024-04-13T05:19:50.500 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setproctitle... no 2024-04-13T05:19:50.828 INFO:tasks.workunit.client.0.smithi057.stdout:checking for getopt_long... yes 2024-04-13T05:19:51.144 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strtoull... yes 2024-04-13T05:19:51.470 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strtoq... yes 2024-04-13T05:19:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:51 smithi083 ceph-mon[35133]: pgmap v523: 129 pgs: 129 active+clean; 4.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 291 KiB/s wr, 79 op/s 2024-04-13T05:19:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:51 smithi057 ceph-mon[32331]: pgmap v523: 129 pgs: 129 active+clean; 4.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 291 KiB/s wr, 79 op/s 2024-04-13T05:19:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:51 smithi189 ceph-mon[35531]: pgmap v523: 129 pgs: 129 active+clean; 4.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 291 KiB/s wr, 79 op/s 2024-04-13T05:19:51.820 INFO:tasks.workunit.client.0.smithi057.stdout:checking for snprintf... yes 2024-04-13T05:19:52.143 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vsnprintf... yes 2024-04-13T05:19:52.486 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vfprintf... yes 2024-04-13T05:19:52.829 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gethostname... yes 2024-04-13T05:19:53.155 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setlocale... yes 2024-04-13T05:19:53.421 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sync... yes 2024-04-13T05:19:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:53 smithi083 ceph-mon[35133]: pgmap v524: 129 pgs: 129 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 191 KiB/s wr, 65 op/s 2024-04-13T05:19:53.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:53 smithi057 ceph-mon[32331]: pgmap v524: 129 pgs: 129 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 191 KiB/s wr, 65 op/s 2024-04-13T05:19:53.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:53 smithi189 ceph-mon[35531]: pgmap v524: 129 pgs: 129 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 191 KiB/s wr, 65 op/s 2024-04-13T05:19:53.678 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether snprintf is C99 conformant... done 2024-04-13T05:19:53.738 INFO:tasks.workunit.client.0.smithi057.stdout:configure: creating ./config.status 2024-04-13T05:19:54.445 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating Makefile 2024-04-13T05:19:54.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:54 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:19:54.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:54 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:19:54.538 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating src/Makefile 2024-04-13T05:19:54.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:54 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:19:55.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:55 smithi083 ceph-mon[35133]: pgmap v525: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 213 KiB/s wr, 77 op/s 2024-04-13T05:19:55.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:19:55.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:19:55.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:19:55.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:55 smithi057 ceph-mon[32331]: pgmap v525: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 213 KiB/s wr, 77 op/s 2024-04-13T05:19:55.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:19:55.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:19:55.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:19:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:55 smithi189 ceph-mon[35531]: pgmap v525: 129 pgs: 129 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 213 KiB/s wr, 77 op/s 2024-04-13T05:19:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:19:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:19:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:19:57.045 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating man/Makefile 2024-04-13T05:19:57.168 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating config.h 2024-04-13T05:19:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:57 smithi083 ceph-mon[35133]: pgmap v526: 129 pgs: 129 active+clean; 5.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 303 KiB/s wr, 87 op/s 2024-04-13T05:19:57.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:57 smithi057 ceph-mon[32331]: pgmap v526: 129 pgs: 129 active+clean; 5.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 303 KiB/s wr, 87 op/s 2024-04-13T05:19:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:57 smithi189 ceph-mon[35531]: pgmap v526: 129 pgs: 129 active+clean; 5.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 303 KiB/s wr, 87 op/s 2024-04-13T05:19:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:19:59 smithi083 ceph-mon[35133]: pgmap v527: 129 pgs: 129 active+clean; 5.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 213 KiB/s wr, 70 op/s 2024-04-13T05:19:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:19:59 smithi057 ceph-mon[32331]: pgmap v527: 129 pgs: 129 active+clean; 5.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 213 KiB/s wr, 70 op/s 2024-04-13T05:19:59.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:19:59 smithi189 ceph-mon[35531]: pgmap v527: 129 pgs: 129 active+clean; 5.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 213 KiB/s wr, 70 op/s 2024-04-13T05:20:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:00 smithi083 ceph-mon[35133]: overall HEALTH_OK 2024-04-13T05:20:00.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:00 smithi057 ceph-mon[32331]: overall HEALTH_OK 2024-04-13T05:20:00.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:00 smithi189 ceph-mon[35531]: overall HEALTH_OK 2024-04-13T05:20:00.703 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: executing depfiles commands 2024-04-13T05:20:01.486 INFO:tasks.workunit.client.0.smithi057.stderr:+ make 2024-04-13T05:20:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:01 smithi083 ceph-mon[35133]: pgmap v528: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 241 KiB/s wr, 85 op/s 2024-04-13T05:20:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:01 smithi057 ceph-mon[32331]: pgmap v528: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 241 KiB/s wr, 85 op/s 2024-04-13T05:20:01.572 INFO:tasks.workunit.client.0.smithi057.stdout:make all-recursive 2024-04-13T05:20:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:01 smithi189 ceph-mon[35531]: pgmap v528: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 241 KiB/s wr, 85 op/s 2024-04-13T05:20:01.609 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-13T05:20:01.629 INFO:tasks.workunit.client.0.smithi057.stdout:Making all in src 2024-04-13T05:20:01.749 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-04-13T05:20:01.749 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-04-13T05:20:01.749 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-04-13T05:20:02.261 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-04-13T05:20:02.261 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-04-13T05:20:03.040 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-04-13T05:20:03.041 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-04-13T05:20:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:03 smithi083 ceph-mon[35133]: pgmap v529: 129 pgs: 129 active+clean; 5.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 156 KiB/s wr, 59 op/s 2024-04-13T05:20:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:03 smithi057 ceph-mon[32331]: pgmap v529: 129 pgs: 129 active+clean; 5.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 156 KiB/s wr, 59 op/s 2024-04-13T05:20:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:03 smithi189 ceph-mon[35531]: pgmap v529: 129 pgs: 129 active+clean; 5.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 156 KiB/s wr, 59 op/s 2024-04-13T05:20:03.726 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-04-13T05:20:03.727 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-04-13T05:20:04.482 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-04-13T05:20:04.482 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-04-13T05:20:05.209 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-04-13T05:20:05.209 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-04-13T05:20:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:05 smithi083 ceph-mon[35133]: pgmap v530: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 163 KiB/s wr, 57 op/s 2024-04-13T05:20:05.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:05 smithi057 ceph-mon[32331]: pgmap v530: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 163 KiB/s wr, 57 op/s 2024-04-13T05:20:05.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:05 smithi189 ceph-mon[35531]: pgmap v530: 129 pgs: 129 active+clean; 5.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 163 KiB/s wr, 57 op/s 2024-04-13T05:20:05.946 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-04-13T05:20:05.946 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-04-13T05:20:06.749 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-04-13T05:20:06.749 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-04-13T05:20:07.438 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-04-13T05:20:07.439 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-04-13T05:20:07.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:07 smithi083 ceph-mon[35133]: pgmap v531: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 150 KiB/s wr, 47 op/s 2024-04-13T05:20:07.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:07 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:20:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:07 smithi057 ceph-mon[32331]: pgmap v531: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 150 KiB/s wr, 47 op/s 2024-04-13T05:20:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:07 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:20:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:07 smithi189 ceph-mon[35531]: pgmap v531: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 150 KiB/s wr, 47 op/s 2024-04-13T05:20:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:07 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:20:08.125 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-04-13T05:20:08.126 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-04-13T05:20:08.580 INFO:tasks.workunit.client.0.smithi057.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-04-13T05:20:08.936 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-04-13T05:20:08.961 INFO:tasks.workunit.client.0.smithi057.stdout:Making all in man 2024-04-13T05:20:09.022 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-04-13T05:20:09.023 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Nothing to be done for 'all'. 2024-04-13T05:20:09.023 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-04-13T05:20:09.070 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-13T05:20:09.086 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-13T05:20:09.089 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-13T05:20:09.095 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd src 2024-04-13T05:20:09.104 INFO:tasks.workunit.client.0.smithi057.stderr:+ mkdir blogtest_in 2024-04-13T05:20:09.119 INFO:tasks.workunit.client.0.smithi057.stdout:running blogbench 2024-04-13T05:20:09.119 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'running blogbench' 2024-04-13T05:20:09.119 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./blogbench -d blogtest_in 2024-04-13T05:20:09.122 INFO:tasks.workunit.client.0.smithi057.stdout: 2024-04-13T05:20:09.123 INFO:tasks.workunit.client.0.smithi057.stdout:Frequency = 10 secs 2024-04-13T05:20:09.123 INFO:tasks.workunit.client.0.smithi057.stdout:Scratch dir = [blogtest_in] 2024-04-13T05:20:09.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:09 smithi057 ceph-mon[32331]: pgmap v532: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 55 KiB/s wr, 28 op/s 2024-04-13T05:20:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:09 smithi189 ceph-mon[35531]: pgmap v532: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 55 KiB/s wr, 28 op/s 2024-04-13T05:20:09.626 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 3 writers... 2024-04-13T05:20:09.627 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 1 rewriters... 2024-04-13T05:20:09.627 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 5 commenters... 2024-04-13T05:20:09.627 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 100 readers... 2024-04-13T05:20:09.634 INFO:tasks.workunit.client.0.smithi057.stdout:Benchmarking for 30 iterations. 2024-04-13T05:20:09.634 INFO:tasks.workunit.client.0.smithi057.stdout:The test will run during 5 minutes. 2024-04-13T05:20:09.634 INFO:tasks.workunit.client.0.smithi057.stdout: 2024-04-13T05:20:09.634 INFO:tasks.workunit.client.0.smithi057.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-04-13T05:20:09.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:09 smithi083 ceph-mon[35133]: pgmap v532: 129 pgs: 129 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 55 KiB/s wr, 28 op/s 2024-04-13T05:20:11.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:20:11.213 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:20:11.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.452+0000 7f27b0c00640 1 -- 172.21.15.57:0/3268114943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 msgr2=0x7f27ac1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.452+0000 7f27b0c00640 1 --2- 172.21.15.57:0/3268114943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1022c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f279c009900 tx=0x7f279c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 -- 172.21.15.57:0/3268114943 shutdown_connections 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 --2- 172.21.15.57:0/3268114943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27ac103080 0x7f27ac103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 --2- 172.21.15.57:0/3268114943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 0x7f27ac10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 --2- 172.21.15.57:0/3268114943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1022c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 -- 172.21.15.57:0/3268114943 >> 172.21.15.57:0/3268114943 conn(0x7f27ac0fd460 msgr2=0x7f27ac0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 -- 172.21.15.57:0/3268114943 shutdown_connections 2024-04-13T05:20:11.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.453+0000 7f27b0c00640 1 -- 172.21.15.57:0/3268114943 wait complete. 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.454+0000 7f27b0c00640 1 Processor -- start 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.454+0000 7f27b0c00640 1 -- start start 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27b0c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27b0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27ac103080 0x7f27ac1a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27b0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 0x7f27ac1aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27b0c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac06c2e0 con 0x7f27ac101ea0 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27b0c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac06c160 con 0x7f27ac103080 2024-04-13T05:20:11.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27b0c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac06c460 con 0x7f27ac103a40 2024-04-13T05:20:11.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1a3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34800/0 (socket says 172.21.15.57:34800) 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27ab400640 1 -- 172.21.15.57:0/973876030 learned_addr learned my addr 172.21.15.57:0/973876030 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.455+0000 7f27aaa00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27ac103080 0x7f27ac1a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27abe00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 0x7f27ac1aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27ab400640 1 -- 172.21.15.57:0/973876030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 msgr2=0x7f27ac1aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27ab400640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 0x7f27ac1aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27ab400640 1 -- 172.21.15.57:0/973876030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27ac103080 msgr2=0x7f27ac1a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27ab400640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27ac103080 0x7f27ac1a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27ab400640 1 -- 172.21.15.57:0/973876030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac1aa7c0 con 0x7f27ac101ea0 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27abe00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 0x7f27ac1aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:11.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.456+0000 7f27ab400640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1a3130 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f279c005ab0 tx=0x7f279c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.457+0000 7f27a3e00640 1 -- 172.21.15.57:0/973876030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c043020 con 0x7f27ac101ea0 2024-04-13T05:20:11.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.457+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ac1aaa50 con 0x7f27ac101ea0 2024-04-13T05:20:11.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.458+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ac1aaff0 con 0x7f27ac101ea0 2024-04-13T05:20:11.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.458+0000 7f27a3e00640 1 -- 172.21.15.57:0/973876030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279c0054c0 con 0x7f27ac101ea0 2024-04-13T05:20:11.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.458+0000 7f27a3e00640 1 -- 172.21.15.57:0/973876030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c003d10 con 0x7f27ac101ea0 2024-04-13T05:20:11.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.462+0000 7f27b51f9740 1 Processor -- start 2024-04-13T05:20:11.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 -- start start 2024-04-13T05:20:11.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27a3e00640 1 -- 172.21.15.57:0/973876030 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f279c046020 con 0x7f27ac101ea0 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2649320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 0x55f0c264e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c264ea60 0x55f0c2650ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55f0c25a7020 con 0x55f0c26498f0 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0c254d6d0 con 0x55f0c264ea60 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27b51f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0c25a2fe0 con 0x55f0c2648f00 2024-04-13T05:20:11.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.463+0000 7f27ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 0x55f0c264e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 0x55f0c264e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34810/0 (socket says 172.21.15.57:34810) 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 -- 172.21.15.57:0/1759400061 learned_addr learned my addr 172.21.15.57:0/1759400061 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27abe00640 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2649320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 -- 172.21.15.57:0/1759400061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 msgr2=0x55f0c2649320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2649320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 -- 172.21.15.57:0/1759400061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c264ea60 msgr2=0x55f0c2650ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c264ea60 0x55f0c2650ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 -- 172.21.15.57:0/1759400061 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c23837d0 con 0x55f0c26498f0 2024-04-13T05:20:11.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.464+0000 7f27ab400640 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 0x55f0c264e3f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f279c0f8a90 tx=0x7f279c0f8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c2259e91604457 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.465+0000 7f2797e00640 1 -- 172.21.15.57:0/1759400061 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c101070 con 0x55f0c26498f0 2024-04-13T05:20:11.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.465+0000 7f2797e00640 1 -- 172.21.15.57:0/1759400061 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279c0fad30 con 0x55f0c26498f0 2024-04-13T05:20:11.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.465+0000 7f2797e00640 1 -- 172.21.15.57:0/1759400061 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c0f4900 con 0x55f0c26498f0 2024-04-13T05:20:11.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.465+0000 7f27a3e00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27900a3cb0 0x7f27900a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.466+0000 7f27a3e00640 1 -- 172.21.15.57:0/973876030 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f279c0f2720 con 0x7f27ac101ea0 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.466+0000 7f27abe00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27900a3cb0 0x7f27900a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.466+0000 7f27b51f9740 1 -- 172.21.15.57:0/1759400061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 msgr2=0x55f0c264e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.466+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 0x55f0c264e3f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f279c0f8a90 tx=0x7f279c0f8b80 comp rx=0 tx=0).stop 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 -- 172.21.15.57:0/1759400061 shutdown_connections 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c264ea60 0x55f0c2650ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c26498f0 0x55f0c264e3f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1759400061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2649320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 -- 172.21.15.57:0/1759400061 >> 172.21.15.57:0/1759400061 conn(0x55f0c263bc60 msgr2=0x55f0c263c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27abe00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27900a3cb0 0x7f27900a6170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55f0c264a150 tx=0x7f27a4009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 -- 172.21.15.57:0/1759400061 shutdown_connections 2024-04-13T05:20:11.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.467+0000 7f27b51f9740 1 -- 172.21.15.57:0/1759400061 wait complete. 2024-04-13T05:20:11.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.468+0000 7f27b51f9740 1 -- start start 2024-04-13T05:20:11.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.468+0000 7f27b51f9740 1 Processor -- start 2024-04-13T05:20:11.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2652f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2653440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c265a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55f0c26517d0 con 0x55f0c264ea60 2024-04-13T05:20:11.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0c2560a90 con 0x55f0c26498f0 2024-04-13T05:20:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0c2651950 con 0x55f0c2648f00 2024-04-13T05:20:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.469+0000 7f27b51f9740 10 client.0 _ll_register_callbacks cb 0x55f0c2560480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2653440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2653440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59288/0 (socket says 172.21.15.57:59288) 2024-04-13T05:20:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 -- 172.21.15.57:0/3067109561 learned_addr learned my addr 172.21.15.57:0/3067109561 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:11.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27abe00640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c265a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27ab400640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2652f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 -- 172.21.15.57:0/3067109561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 msgr2=0x55f0c2652f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2652f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 -- 172.21.15.57:0/3067109561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 msgr2=0x55f0c265a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c265a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.470+0000 7f27aaa00640 1 -- 172.21.15.57:0/3067109561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c22b93f0 con 0x55f0c26498f0 2024-04-13T05:20:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.471+0000 7f27aaa00640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2653440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f279800bea0 tx=0x7f279800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e4fafda2a1a167 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.471+0000 7f2794c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:11.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.471+0000 7f2794c00640 1 -- 172.21.15.57:0/3067109561 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798011850 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.471+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279c0fad30 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.471+0000 7f2794c00640 1 -- 172.21.15.57:0/3067109561 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f27980119f0 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.472+0000 7f2794c00640 1 -- 172.21.15.57:0/3067109561 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2798025050 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.472+0000 7f27b51f9740 10 client.35441 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.472+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0c22790f0 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f2794c00640 1 -- 172.21.15.57:0/3067109561 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279c0fad30 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f2794c00640 1 -- 172.21.15.57:0/3067109561 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f279801a380 con 0x55f0c26498f0 2024-04-13T05:20:11.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f27b51f9740 10 client.35441 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:11.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f27b51f9740 10 client.35441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:11.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f27b51f9740 20 client.35441 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:11.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c265edd0 0x55f0c2661250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f27b51f9740 4 client.35441 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:11.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.473+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c2546600 con 0x55f0c265edd0 2024-04-13T05:20:11.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.474+0000 7f27ab400640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c265edd0 0x55f0c2661250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.474+0000 7f27ab400640 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c265edd0 0x55f0c2661250 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b5a225447b7c37 server_cookie=ae733fb8e6eed69c in_seq=0 out_seq=0 2024-04-13T05:20:11.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.474+0000 7f2794c00640 10 client.35441 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:11.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f2794c00640 1 -- 172.21.15.57:0/3067109561 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0c2546600 con 0x55f0c265edd0 2024-04-13T05:20:11.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f2794c00640 10 client.35441 handle_command_reply: tid=0 2024-04-13T05:20:11.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 client.35441 shutdown 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 msgr2=0x55f0c2653440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2653440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f279800bea0 tx=0x7f279800bf90 comp rx=0 tx=0).stop 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 shutdown_connections 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c265edd0 0x55f0c2661250 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2653440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c265a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3067109561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c2648f00 0x55f0c2652f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 >> 172.21.15.57:0/3067109561 conn(0x55f0c263bc60 msgr2=0x55f0c263ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f2794c00640 0 client.35441 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:11.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.476+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 shutdown_connections 2024-04-13T05:20:11.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.477+0000 7f27b51f9740 1 -- 172.21.15.57:0/3067109561 wait complete. 2024-04-13T05:20:11.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.477+0000 7f27b51f9740 20 client.35441 trim_cache size 0 max 16384 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 Processor -- start 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 -- start start 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c2648f00 0x55f0c263f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c26498f0 0x55f0c265a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c2651cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f278c0124c0 con 0x55f0c264ea60 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f278c00fd60 con 0x55f0c2648f00 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27b51f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f278c00dbb0 con 0x55f0c26498f0 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.505+0000 7f27abe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c2651cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c2651cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34820/0 (socket says 172.21.15.57:34820) 2024-04-13T05:20:11.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 -- 172.21.15.57:0/3153359426 learned_addr learned my addr 172.21.15.57:0/3153359426 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:11.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27ab400640 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c2648f00 0x55f0c263f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27aaa00640 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c26498f0 0x55f0c265a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 -- 172.21.15.57:0/3153359426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c26498f0 msgr2=0x55f0c265a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c26498f0 0x55f0c265a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 -- 172.21.15.57:0/3153359426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c2648f00 msgr2=0x55f0c263f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c2648f00 0x55f0c263f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 -- 172.21.15.57:0/3153359426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c264a180 con 0x55f0c264ea60 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.506+0000 7f27abe00640 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c2651cf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f27a4018880 tx=0x7f27a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28416ac71c8d5f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.507+0000 7f2797400640 1 -- 172.21.15.57:0/3153359426 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4029070 con 0x55f0c264ea60 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.507+0000 7f2797400640 1 -- 172.21.15.57:0/3153359426 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27a4017c60 con 0x55f0c264ea60 2024-04-13T05:20:11.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.507+0000 7f2797400640 1 -- 172.21.15.57:0/3153359426 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4024cc0 con 0x55f0c264ea60 2024-04-13T05:20:11.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 -- 172.21.15.57:0/3153359426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 msgr2=0x55f0c2651cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c2651cf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f27a4018880 tx=0x7f27a4013000 comp rx=0 tx=0).stop 2024-04-13T05:20:11.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 -- 172.21.15.57:0/3153359426 shutdown_connections 2024-04-13T05:20:11.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c264ea60 0x55f0c2651cf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c26498f0 0x55f0c265a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 --2- 172.21.15.57:0/3153359426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c2648f00 0x55f0c263f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 -- 172.21.15.57:0/3153359426 >> 172.21.15.57:0/3153359426 conn(0x55f0c263bc60 msgr2=0x55f0c263eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 -- 172.21.15.57:0/3153359426 shutdown_connections 2024-04-13T05:20:11.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.508+0000 7f27b51f9740 1 -- 172.21.15.57:0/3153359426 wait complete. 2024-04-13T05:20:11.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.509+0000 7f27b51f9740 1 -- start start 2024-04-13T05:20:11.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.509+0000 7f27b51f9740 1 Processor -- start 2024-04-13T05:20:11.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 0x55f0c26581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2658730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c264ea60 0x55f0c26596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f278c00e220 con 0x55f0c2648f00 2024-04-13T05:20:11.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f278c00f690 con 0x55f0c26498f0 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f278c00e8f0 con 0x55f0c264ea60 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27b51f9740 10 client.0 _ll_register_callbacks cb 0x7f279800bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 0x55f0c26581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 0x55f0c26581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34832/0 (socket says 172.21.15.57:34832) 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 -- 172.21.15.57:0/1697022729 learned_addr learned my addr 172.21.15.57:0/1697022729 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27aaa00640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2658730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27abe00640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c264ea60 0x55f0c26596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 -- 172.21.15.57:0/1697022729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c264ea60 msgr2=0x55f0c26596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c264ea60 0x55f0c26596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 -- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 msgr2=0x55f0c2658730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2658730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.510+0000 7f27ab400640 1 -- 172.21.15.57:0/1697022729 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c2659cc0 con 0x55f0c2648f00 2024-04-13T05:20:11.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f27ab400640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 0x55f0c26581f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f279c0bafd0 tx=0x7f279c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2b04af25d7a874 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f2794c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f2794c00640 1 -- 172.21.15.57:0/1697022729 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c101040 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f2794c00640 1 -- 172.21.15.57:0/1697022729 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f279c0035f0 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a4017c60 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f2794c00640 1 -- 172.21.15.57:0/1697022729 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f279c0ba040 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f27b51f9740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.511+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0c26c59c0 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f2794c00640 1 -- 172.21.15.57:0/1697022729 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27a4017c60 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f2794c00640 1 -- 172.21.15.57:0/1697022729 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f279c0bac70 con 0x55f0c2648f00 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f27b51f9740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f27b51f9740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f27b51f9740 20 client.15975 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c26c6200 0x55f0c26c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f27b51f9740 4 client.15975 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:20:11.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.512+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0c25b4a40 con 0x55f0c26c6200 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.513+0000 7f27aaa00640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c26c6200 0x55f0c26c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.513+0000 7f27aaa00640 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c26c6200 0x55f0c26c8620 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc57213da749b8a server_cookie=26fc75b67537574 in_seq=0 out_seq=0 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.513+0000 7f2794c00640 10 client.15975 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f2794c00640 1 -- 172.21.15.57:0/1697022729 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f0c25b4a40 con 0x55f0c26c6200 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f2794c00640 10 client.15975 handle_command_reply: tid=0 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 client.15975 shutdown 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 msgr2=0x55f0c26581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 0x55f0c26581f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f279c0bafd0 tx=0x7f279c009260 comp rx=0 tx=0).stop 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 shutdown_connections 2024-04-13T05:20:11.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55f0c26c6200 0x55f0c26c8620 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c26498f0 0x55f0c2658730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c264ea60 0x55f0c26596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 --2- 172.21.15.57:0/1697022729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f0c2648f00 0x55f0c26581f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 >> 172.21.15.57:0/1697022729 conn(0x55f0c263bc60 msgr2=0x55f0c263f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f2794c00640 0 client.15975 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 shutdown_connections 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 1 -- 172.21.15.57:0/1697022729 wait complete. 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.514+0000 7f27b51f9740 20 client.15975 trim_cache size 0 max 16384 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:20:11.519 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:20:11.520 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:20:11.520 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:20:11.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27900a3cb0 msgr2=0x7f27900a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27900a3cb0 0x7f27900a6170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55f0c264a150 tx=0x7f27a4009630 comp rx=0 tx=0).stop 2024-04-13T05:20:11.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 msgr2=0x7f27ac1a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1a3130 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f279c005ab0 tx=0x7f279c0042f0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 shutdown_connections 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f27900a3cb0 0x7f27900a6170 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27ac103080 0x7f27ac1a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27ac103a40 0x7f27ac1aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 --2- 172.21.15.57:0/973876030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27ac101ea0 0x7f27ac1a3130 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 >> 172.21.15.57:0/973876030 conn(0x7f27ac0fd460 msgr2=0x7f27ac06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 shutdown_connections 2024-04-13T05:20:11.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.515+0000 7f27b0c00640 1 -- 172.21.15.57:0/973876030 wait complete. 2024-04-13T05:20:11.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:11 smithi057 ceph-mon[32331]: pgmap v533: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-04-13T05:20:11.538 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:20:11.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:20:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:11 smithi189 ceph-mon[35531]: pgmap v533: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-04-13T05:20:11.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:11 smithi083 ceph-mon[35133]: pgmap v533: 129 pgs: 129 active+clean; 5.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-04-13T05:20:11.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 -- 172.21.15.57:0/2523332914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228103020 msgr2=0x7f9228105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 --2- 172.21.15.57:0/2523332914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228103020 0x7f9228105490 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9214009860 tx=0x7f921402bb70 comp rx=0 tx=0).stop 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 -- 172.21.15.57:0/2523332914 shutdown_connections 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 --2- 172.21.15.57:0/2523332914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92281059d0 0x7f9228105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 --2- 172.21.15.57:0/2523332914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9228103020 0x7f9228105490 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 --2- 172.21.15.57:0/2523332914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92281006b0 0x7f9228102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 -- 172.21.15.57:0/2523332914 >> 172.21.15.57:0/2523332914 conn(0x7f92280fc290 msgr2=0x7f92280fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 -- 172.21.15.57:0/2523332914 shutdown_connections 2024-04-13T05:20:11.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.797+0000 7f9227200640 1 -- 172.21.15.57:0/2523332914 wait complete. 2024-04-13T05:20:11.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.799+0000 7f9227200640 1 Processor -- start 2024-04-13T05:20:11.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.799+0000 7f9227200640 1 -- start start 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9227200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 0x7f922819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9227200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9228103020 0x7f922819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9227200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92281059d0 0x7f92281a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9227200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9228113f60 con 0x7f92281006b0 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9227200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9228113de0 con 0x7f92281059d0 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9227200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92281140e0 con 0x7f9228103020 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9225e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 0x7f922819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9225e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 0x7f922819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34858/0 (socket says 172.21.15.57:34858) 2024-04-13T05:20:11.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9225e00640 1 -- 172.21.15.57:0/2526822735 learned_addr learned my addr 172.21.15.57:0/2526822735 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:11.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9225400640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9228103020 0x7f922819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9226800640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92281059d0 0x7f92281a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.800+0000 7f9225e00640 1 -- 172.21.15.57:0/2526822735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9228103020 msgr2=0x7f922819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f9225e00640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9228103020 0x7f922819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f9225e00640 1 -- 172.21.15.57:0/2526822735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92281059d0 msgr2=0x7f92281a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f9225e00640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92281059d0 0x7f92281a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f9225e00640 1 -- 172.21.15.57:0/2526822735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92281a6290 con 0x7f92281006b0 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f9225e00640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 0x7f922819e670 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9c0 tx=0x7f921800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921800ccc0 con 0x7f92281006b0 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92180054d0 con 0x7f92281006b0 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.801+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92281a6580 con 0x7f92281006b0 2024-04-13T05:20:11.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.802+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92281a6aa0 con 0x7f92281006b0 2024-04-13T05:20:11.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.802+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218010430 con 0x7f92281006b0 2024-04-13T05:20:11.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.803+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92280fdd80 con 0x7f92281006b0 2024-04-13T05:20:11.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.807+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92180046c0 con 0x7f92281006b0 2024-04-13T05:20:11.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.809+0000 7f921ea00640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f920c0a3bb0 0x7f920c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:11.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.809+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f92180d0b20 con 0x7f92281006b0 2024-04-13T05:20:11.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.810+0000 7f9225400640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f920c0a3bb0 0x7f920c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:11.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.812+0000 7f9225400640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f920c0a3bb0 0x7f920c0a6070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f921402c180 tx=0x7f92140095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:11.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:11.818+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f921809d160 con 0x7f92281006b0 2024-04-13T05:20:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.229+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f922802d5e0 con 0x7f92281006b0 2024-04-13T05:20:12.233 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:20:12.233 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:20:05.635994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:20:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.230+0000 7f921ea00640 1 -- 172.21.15.57:0/2526822735 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11857 (secure 0 0 0) 0x7f921809ca60 con 0x7f92281006b0 2024-04-13T05:20:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2024-04-13T05:20:12.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f920c0a3bb0 msgr2=0x7f920c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f920c0a3bb0 0x7f920c0a6070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f921402c180 tx=0x7f92140095b0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 msgr2=0x7f922819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 0x7f922819e670 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9c0 tx=0x7f921800ee80 comp rx=0 tx=0).stop 2024-04-13T05:20:12.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 shutdown_connections 2024-04-13T05:20:12.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f920c0a3bb0 0x7f920c0a6070 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92281059d0 0x7f92281a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9228103020 0x7f922819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 --2- 172.21.15.57:0/2526822735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92281006b0 0x7f922819e670 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 >> 172.21.15.57:0/2526822735 conn(0x7f92280fc290 msgr2=0x7f9228101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.234+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 shutdown_connections 2024-04-13T05:20:12.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.235+0000 7f9227200640 1 -- 172.21.15.57:0/2526822735 wait complete. 2024-04-13T05:20:12.260 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:20:12.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 -- 172.21.15.57:0/2746121563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614103080 msgr2=0x7fd614103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 --2- 172.21.15.57:0/2746121563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614103080 0x7fd614103500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 -- 172.21.15.57:0/2746121563 shutdown_connections 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 --2- 172.21.15.57:0/2746121563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd614103a40 0x7fd61410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 --2- 172.21.15.57:0/2746121563 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614103080 0x7fd614103500 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 --2- 172.21.15.57:0/2746121563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd614101ea0 0x7fd6141022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 -- 172.21.15.57:0/2746121563 >> 172.21.15.57:0/2746121563 conn(0x7fd6140fd460 msgr2=0x7fd6140ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.525+0000 7fd613200640 1 -- 172.21.15.57:0/2746121563 shutdown_connections 2024-04-13T05:20:12.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.526+0000 7fd613200640 1 -- 172.21.15.57:0/2746121563 wait complete. 2024-04-13T05:20:12.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 Processor -- start 2024-04-13T05:20:12.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 -- start start 2024-04-13T05:20:12.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 0x7fd61419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd614103080 0x7fd61419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd614103a40 0x7fd6141a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd61410daa0 con 0x7fd614101ea0 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd61410d920 con 0x7fd614103080 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.527+0000 7fd613200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd61410dc20 con 0x7fd614103a40 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.528+0000 7fd611e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 0x7fd61419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.528+0000 7fd611e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 0x7fd61419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34876/0 (socket says 172.21.15.57:34876) 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.528+0000 7fd611e00640 1 -- 172.21.15.57:0/805138763 learned_addr learned my addr 172.21.15.57:0/805138763 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.528+0000 7fd612800640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd614103a40 0x7fd6141a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.528+0000 7fd611400640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd614103080 0x7fd61419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.529+0000 7fd611e00640 1 -- 172.21.15.57:0/805138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd614103a40 msgr2=0x7fd6141a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.529+0000 7fd611e00640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd614103a40 0x7fd6141a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.529+0000 7fd611e00640 1 -- 172.21.15.57:0/805138763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd614103080 msgr2=0x7fd61419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.529+0000 7fd611e00640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd614103080 0x7fd61419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.529+0000 7fd611e00640 1 -- 172.21.15.57:0/805138763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6141a6290 con 0x7fd614101ea0 2024-04-13T05:20:12.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.529+0000 7fd611e00640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 0x7fd61419e670 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd60400c970 tx=0x7fd60400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:12 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2526822735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:20:12.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.530+0000 7fd60aa00640 1 -- 172.21.15.57:0/805138763 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604007c10 con 0x7fd614101ea0 2024-04-13T05:20:12.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.530+0000 7fd60aa00640 1 -- 172.21.15.57:0/805138763 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd604007db0 con 0x7fd614101ea0 2024-04-13T05:20:12.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.530+0000 7fd60aa00640 1 -- 172.21.15.57:0/805138763 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60400f870 con 0x7fd614101ea0 2024-04-13T05:20:12.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.530+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141a6580 con 0x7fd614101ea0 2024-04-13T05:20:12.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.530+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a6a80 con 0x7fd614101ea0 2024-04-13T05:20:12.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.534+0000 7fd60aa00640 1 -- 172.21.15.57:0/805138763 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd60400fa10 con 0x7fd614101ea0 2024-04-13T05:20:12.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.535+0000 7fd61be23740 1 Processor -- start 2024-04-13T05:20:12.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.535+0000 7fd61be23740 1 -- start start 2024-04-13T05:20:12.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.535+0000 7fd61be23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046ed3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.535+0000 7fd61be23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3e30 0x561046ed8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd61be23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed8fa0 0x561046edb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd61be23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561046e39020 con 0x561046ed3440 2024-04-13T05:20:12.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd61be23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561046ddf950 con 0x561046ed3e30 2024-04-13T05:20:12.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd61be23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561046e34fe0 con 0x561046ed8fa0 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd612800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046ed3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd612800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046ed3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34884/0 (socket says 172.21.15.57:34884) 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd612800640 1 -- 172.21.15.57:0/4280469307 learned_addr learned my addr 172.21.15.57:0/4280469307 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd60aa00640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd5f80a3cb0 0x7fd5f80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd611e00640 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3e30 0x561046ed8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.536+0000 7fd611400640 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed8fa0 0x561046edb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd611400640 1 -- 172.21.15.57:0/4280469307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3e30 msgr2=0x561046ed8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd611400640 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3e30 0x561046ed8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd611400640 1 -- 172.21.15.57:0/4280469307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 msgr2=0x561046ed3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd611400640 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046ed3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd611400640 1 -- 172.21.15.57:0/4280469307 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561046b4b3f0 con 0x561046ed8fa0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd611e00640 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3e30 0x561046ed8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.537+0000 7fd612800640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd5f80a3cb0 0x7fd5f80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.538+0000 7fd611400640 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed8fa0 0x561046edb430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd60002c170 tx=0x7fd600005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393665fa5fe967e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.538+0000 7fd60aa00640 1 -- 172.21.15.57:0/805138763 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd6040d0a90 con 0x7fd614101ea0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.538+0000 7fd612800640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd5f80a3cb0 0x7fd5f80a6170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x561046ed4690 tx=0x7fd60c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.538+0000 7fd5ff400640 1 -- 172.21.15.57:0/4280469307 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600043020 con 0x561046ed8fa0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.538+0000 7fd5ff400640 1 -- 172.21.15.57:0/4280469307 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd600004040 con 0x561046ed8fa0 2024-04-13T05:20:12.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.538+0000 7fd5ff400640 1 -- 172.21.15.57:0/4280469307 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6000046c0 con 0x561046ed8fa0 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.539+0000 7fd61be23740 1 -- 172.21.15.57:0/4280469307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed8fa0 msgr2=0x561046edb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.539+0000 7fd61be23740 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed8fa0 0x561046edb430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd60002c170 tx=0x7fd600005d50 comp rx=0 tx=0).stop 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 -- 172.21.15.57:0/4280469307 shutdown_connections 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3e30 0x561046ed8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed8fa0 0x561046edb430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 --2- 172.21.15.57:0/4280469307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046ed3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 -- 172.21.15.57:0/4280469307 >> 172.21.15.57:0/4280469307 conn(0x561046ece9b0 msgr2=0x561046ecedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 -- 172.21.15.57:0/4280469307 shutdown_connections 2024-04-13T05:20:12.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 -- 172.21.15.57:0/4280469307 wait complete. 2024-04-13T05:20:12.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 -- start start 2024-04-13T05:20:12.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.540+0000 7fd61be23740 1 Processor -- start 2024-04-13T05:20:12.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.541+0000 7fd61be23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3440 0x561046eddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.541+0000 7fd61be23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046ede070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.541+0000 7fd61be23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ee30a0 0x561046ee5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd61be23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561046edbef0 con 0x561046ed3e30 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd611400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046ede070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd611e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3440 0x561046eddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd611400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046ede070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34900/0 (socket says 172.21.15.57:34900) 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd611400640 1 -- 172.21.15.57:0/1802306387 learned_addr learned my addr 172.21.15.57:0/1802306387 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd612800640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ee30a0 0x561046ee5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561046df2c80 con 0x561046ed3440 2024-04-13T05:20:12.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.542+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561046edc070 con 0x561046ee30a0 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd61be23740 10 client.0 _ll_register_callbacks cb 0x561046df2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd612800640 1 -- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3440 msgr2=0x561046eddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd612800640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3440 0x561046eddb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd612800640 1 -- 172.21.15.57:0/1802306387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 msgr2=0x561046ede070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd612800640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046ede070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd612800640 1 -- 172.21.15.57:0/1802306387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561046c157d0 con 0x561046ee30a0 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd611400640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046ede070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.543+0000 7fd612800640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ee30a0 0x561046ee5490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005fd0 tx=0x7fd60c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bb92c30652447ab server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.544+0000 7fd5f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.544+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1802306387 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01c750 con 0x561046ee30a0 2024-04-13T05:20:12.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.544+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1802306387 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd60c01c8f0 con 0x561046ee30a0 2024-04-13T05:20:12.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.544+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd600004040 con 0x561046ee30a0 2024-04-13T05:20:12.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.545+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1802306387 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd60c01caf0 con 0x561046ee30a0 2024-04-13T05:20:12.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.545+0000 7fd61be23740 10 client.25411 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:12.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.545+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561046ee88b0 con 0x561046ee30a0 2024-04-13T05:20:12.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.545+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1802306387 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd600004040 con 0x561046ee30a0 2024-04-13T05:20:12.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1802306387 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd60c025070 con 0x561046ee30a0 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd61be23740 10 client.25411 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd61be23740 10 client.25411 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd61be23740 20 client.25411 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd61be23740 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046eea070 0x561046eec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd61be23740 4 client.25411 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.546+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561046dd8750 con 0x561046eea070 2024-04-13T05:20:12.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.547+0000 7fd611e00640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046eea070 0x561046eec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.549+0000 7fd5f3e00640 10 client.25411 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:12.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.549+0000 7fd611e00640 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046eea070 0x561046eec4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba962bb2c7c80753 server_cookie=4a0d70854877e800 in_seq=0 out_seq=0 2024-04-13T05:20:12.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.550+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1802306387 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561046dd8750 con 0x561046eea070 2024-04-13T05:20:12.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.550+0000 7fd5f3e00640 10 client.25411 handle_command_reply: tid=0 2024-04-13T05:20:12.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.550+0000 7fd61be23740 1 client.25411 shutdown 2024-04-13T05:20:12.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ee30a0 msgr2=0x561046ee5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ee30a0 0x561046ee5490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005fd0 tx=0x7fd60c004ab0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 shutdown_connections 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046eea070 0x561046eec4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ee30a0 0x561046ee5490 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd5f3e00640 0 client.25411 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046ede070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 --2- 172.21.15.57:0/1802306387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ed3440 0x561046eddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 >> 172.21.15.57:0/1802306387 conn(0x561046ece9b0 msgr2=0x561046ecdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 shutdown_connections 2024-04-13T05:20:12.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.551+0000 7fd61be23740 1 -- 172.21.15.57:0/1802306387 wait complete. 2024-04-13T05:20:12.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.552+0000 7fd61be23740 20 client.25411 trim_cache size 0 max 16384 2024-04-13T05:20:12.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.577+0000 7fd61be23740 1 Processor -- start 2024-04-13T05:20:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.577+0000 7fd61be23740 1 -- start start 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd61be23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046edcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd61be23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3e30 0x561046ecd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd61be23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046ecd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd61be23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141b38c0 con 0x561046ed3440 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd61be23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141b1160 con 0x561046ee30a0 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd61be23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141aefb0 con 0x561046ed3e30 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd612800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046ecd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd612800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046ecd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59370/0 (socket says 172.21.15.57:59370) 2024-04-13T05:20:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.578+0000 7fd612800640 1 -- 172.21.15.57:0/1475853901 learned_addr learned my addr 172.21.15.57:0/1475853901 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.579+0000 7fd611e00640 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046edcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.579+0000 7fd611400640 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3e30 0x561046ecd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.580+0000 7fd612800640 1 -- 172.21.15.57:0/1475853901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3e30 msgr2=0x561046ecd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.580+0000 7fd612800640 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3e30 0x561046ecd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.580+0000 7fd612800640 1 -- 172.21.15.57:0/1475853901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 msgr2=0x561046edcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.580+0000 7fd612800640 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046edcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.580+0000 7fd612800640 1 -- 172.21.15.57:0/1475853901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561046ed4210 con 0x561046ee30a0 2024-04-13T05:20:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.580+0000 7fd611e00640 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046edcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:12 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2526822735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:20:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.581+0000 7fd612800640 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046ecd980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd60c01cf50 tx=0x7fd60c01cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150886bff16b8aba server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.582+0000 7fd5fea00640 1 -- 172.21.15.57:0/1475853901 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c03a070 con 0x561046ee30a0 2024-04-13T05:20:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.582+0000 7fd5fea00640 1 -- 172.21.15.57:0/1475853901 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd60c01d5d0 con 0x561046ee30a0 2024-04-13T05:20:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.582+0000 7fd5fea00640 1 -- 172.21.15.57:0/1475853901 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01c950 con 0x561046ee30a0 2024-04-13T05:20:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.583+0000 7fd61be23740 1 -- 172.21.15.57:0/1475853901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 msgr2=0x561046ecd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.583+0000 7fd61be23740 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046ecd980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd60c01cf50 tx=0x7fd60c01cf80 comp rx=0 tx=0).stop 2024-04-13T05:20:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 -- 172.21.15.57:0/1475853901 shutdown_connections 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046ecd980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3e30 0x561046ecd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 --2- 172.21.15.57:0/1475853901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3440 0x561046edcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 -- 172.21.15.57:0/1475853901 >> 172.21.15.57:0/1475853901 conn(0x561046ece9b0 msgr2=0x561046edc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 -- 172.21.15.57:0/1475853901 shutdown_connections 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.584+0000 7fd61be23740 1 -- 172.21.15.57:0/1475853901 wait complete. 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.585+0000 7fd61be23740 1 -- start start 2024-04-13T05:20:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.585+0000 7fd61be23740 1 Processor -- start 2024-04-13T05:20:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 0x561046f4cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046f4d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046efc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141af620 con 0x561046ed3e30 2024-04-13T05:20:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141b0a90 con 0x561046ee30a0 2024-04-13T05:20:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141afcf0 con 0x561046ed3440 2024-04-13T05:20:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd61be23740 10 client.0 _ll_register_callbacks cb 0x7fd60c010680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd611e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 0x561046f4cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd611e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 0x561046f4cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:34384/0 (socket says 172.21.15.57:34384) 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd611e00640 1 -- 172.21.15.57:0/1764906111 learned_addr learned my addr 172.21.15.57:0/1764906111 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.586+0000 7fd611400640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046f4d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.587+0000 7fd612800640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046efc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.587+0000 7fd611e00640 1 -- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 msgr2=0x561046efc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.587+0000 7fd611e00640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046efc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.587+0000 7fd611e00640 1 -- 172.21.15.57:0/1764906111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 msgr2=0x561046f4d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.587+0000 7fd611e00640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046f4d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.587+0000 7fd611e00640 1 -- 172.21.15.57:0/1764906111 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561046efcdc0 con 0x561046ed3440 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.588+0000 7fd611e00640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 0x561046f4cff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6040993d0 tx=0x7fd6040a3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb9ce689857d0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.588+0000 7fd5f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.588+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1764906111 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604099070 con 0x561046ed3440 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.588+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c01d5d0 con 0x561046ed3440 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.588+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1764906111 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd6040a5150 con 0x561046ed3440 2024-04-13T05:20:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.589+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1764906111 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd604099210 con 0x561046ed3440 2024-04-13T05:20:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.589+0000 7fd61be23740 10 client.25417 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.589+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561046f61410 con 0x561046ed3440 2024-04-13T05:20:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1764906111 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6040b8020 con 0x561046ed3440 2024-04-13T05:20:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1764906111 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd604020030 con 0x561046ed3440 2024-04-13T05:20:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd61be23740 10 client.25417 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd61be23740 10 client.25417 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:20:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd61be23740 20 client.25417 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd61be23740 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046f61dc0 0x561046f641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd61be23740 4 client.25417 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:20:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.590+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561046e46a40 con 0x561046f61dc0 2024-04-13T05:20:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.591+0000 7fd611400640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046f61dc0 0x561046f641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.592+0000 7fd611400640 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046f61dc0 0x561046f641e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=198062c111a5aa99 server_cookie=15746e0a47f78632 in_seq=0 out_seq=0 2024-04-13T05:20:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.592+0000 7fd5f3e00640 10 client.25417 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:12.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.593+0000 7fd5f3e00640 1 -- 172.21.15.57:0/1764906111 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561046e46a40 con 0x561046f61dc0 2024-04-13T05:20:12.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.593+0000 7fd5f3e00640 10 client.25417 handle_command_reply: tid=0 2024-04-13T05:20:12.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.593+0000 7fd61be23740 1 client.25417 shutdown 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.593+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 msgr2=0x561046f4cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.593+0000 7fd61be23740 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 0x561046f4cff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6040993d0 tx=0x7fd6040a3040 comp rx=0 tx=0).stop 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 shutdown_connections 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x561046f61dc0 0x561046f641e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561046ee30a0 0x561046efc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561046ed3e30 0x561046f4d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 --2- 172.21.15.57:0/1764906111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561046ed3440 0x561046f4cff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd5f3e00640 0 client.25417 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 >> 172.21.15.57:0/1764906111 conn(0x561046ece9b0 msgr2=0x561046ed1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 shutdown_connections 2024-04-13T05:20:12.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 1 -- 172.21.15.57:0/1764906111 wait complete. 2024-04-13T05:20:12.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.594+0000 7fd61be23740 20 client.25417 trim_cache size 0 max 16384 2024-04-13T05:20:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.595+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd5f80a3cb0 msgr2=0x7fd5f80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.595+0000 7fd613200640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd5f80a3cb0 0x7fd5f80a6170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x561046ed4690 tx=0x7fd60c004a40 comp rx=0 tx=0).stop 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.595+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 msgr2=0x7fd61419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.595+0000 7fd613200640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 0x7fd61419e670 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd60400c970 tx=0x7fd60400ce30 comp rx=0 tx=0).stop 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 shutdown_connections 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd5f80a3cb0 0x7fd5f80a6170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd614103080 0x7fd61419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd614103a40 0x7fd6141a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 --2- 172.21.15.57:0/805138763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd614101ea0 0x7fd61419e670 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 >> 172.21.15.57:0/805138763 conn(0x7fd6140fd460 msgr2=0x7fd614104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 shutdown_connections 2024-04-13T05:20:12.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.596+0000 7fd613200640 1 -- 172.21.15.57:0/805138763 wait complete. 2024-04-13T05:20:12.623 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:20:12.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:12 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2526822735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:20:12.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 -- 172.21.15.57:0/2122661559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 msgr2=0x7f92a0069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 --2- 172.21.15.57:0/2122661559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a0069150 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9298009ca0 tx=0x7f929802c100 comp rx=0 tx=0).stop 2024-04-13T05:20:12.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 -- 172.21.15.57:0/2122661559 shutdown_connections 2024-04-13T05:20:12.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 --2- 172.21.15.57:0/2122661559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 0x7f92a0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 --2- 172.21.15.57:0/2122661559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a0069150 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 --2- 172.21.15.57:0/2122661559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 0x7f92a01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.937+0000 7f92a5000640 1 -- 172.21.15.57:0/2122661559 >> 172.21.15.57:0/2122661559 conn(0x7f92a00fc360 msgr2=0x7f92a00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.938+0000 7f92a5000640 1 -- 172.21.15.57:0/2122661559 shutdown_connections 2024-04-13T05:20:12.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.938+0000 7f92a5000640 1 -- 172.21.15.57:0/2122661559 wait complete. 2024-04-13T05:20:12.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.939+0000 7f92a5000640 1 Processor -- start 2024-04-13T05:20:12.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.939+0000 7f92a5000640 1 -- start start 2024-04-13T05:20:12.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f92a5000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 0x7f92a019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f92a5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 0x7f92a019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f92a5000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a01a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f92a5000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f92a010d9e0 con 0x7f92a0102980 2024-04-13T05:20:12.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f92a5000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a010d860 con 0x7f92a0101fc0 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f92a5000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92a010db60 con 0x7f92a0100dd0 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f929ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 0x7f92a019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f929fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a01a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f929ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 0x7f92a019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59386/0 (socket says 172.21.15.57:59386) 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.940+0000 7f929ea00640 1 -- 172.21.15.57:0/4005329090 learned_addr learned my addr 172.21.15.57:0/4005329090 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929f400640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 0x7f92a019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929fe00640 1 -- 172.21.15.57:0/4005329090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 msgr2=0x7f92a019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929fe00640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 0x7f92a019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929fe00640 1 -- 172.21.15.57:0/4005329090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 msgr2=0x7f92a019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929fe00640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 0x7f92a019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929fe00640 1 -- 172.21.15.57:0/4005329090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a01a1ed0 con 0x7f92a0102980 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929f400640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 0x7f92a019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.941+0000 7f929fe00640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a01a1860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9298009bf0 tx=0x7f9298003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.942+0000 7f9297e00640 1 -- 172.21.15.57:0/4005329090 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298039040 con 0x7f92a0102980 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.942+0000 7f9297e00640 1 -- 172.21.15.57:0/4005329090 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9298003d10 con 0x7f92a0102980 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.942+0000 7f9297e00640 1 -- 172.21.15.57:0/4005329090 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298032a80 con 0x7f92a0102980 2024-04-13T05:20:12.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.942+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a01a2160 con 0x7f92a0102980 2024-04-13T05:20:12.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.942+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01a25d0 con 0x7f92a0102980 2024-04-13T05:20:12.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.947+0000 7f92a95de740 1 Processor -- start 2024-04-13T05:20:12.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f9297e00640 1 -- 172.21.15.57:0/4005329090 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9298004610 con 0x7f92a0102980 2024-04-13T05:20:12.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 -- start start 2024-04-13T05:20:12.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 0x56080bf76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf79430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56080bed7020 con 0x56080bf71e30 2024-04-13T05:20:12.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56080be7d950 con 0x56080bf76fa0 2024-04-13T05:20:12.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.948+0000 7f92a95de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56080bed2fe0 con 0x56080bf71440 2024-04-13T05:20:12.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 0x56080bf76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 0x56080bf76930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34952/0 (socket says 172.21.15.57:34952) 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 -- 172.21.15.57:0/4219517602 learned_addr learned my addr 172.21.15.57:0/4219517602 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929ea00640 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf79430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 -- 172.21.15.57:0/4219517602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 msgr2=0x56080bf71860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf71860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 -- 172.21.15.57:0/4219517602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 msgr2=0x56080bf79430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf79430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.949+0000 7f929f400640 1 -- 172.21.15.57:0/4219517602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080bbe93f0 con 0x56080bf71e30 2024-04-13T05:20:12.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f929f400640 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 0x56080bf76930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f929000bd30 tx=0x7f929000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85c862e636c035d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f9297e00640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92840a3c30 0x7f92840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f9297e00640 1 -- 172.21.15.57:0/4005329090 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f92980f2440 con 0x7f92a0102980 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f929ea00640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92840a3c30 0x7f92840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f9294c00640 1 -- 172.21.15.57:0/4219517602 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290015070 con 0x56080bf71e30 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f9294c00640 1 -- 172.21.15.57:0/4219517602 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929001a910 con 0x56080bf71e30 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.950+0000 7f9294c00640 1 -- 172.21.15.57:0/4219517602 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290011880 con 0x56080bf71e30 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 -- 172.21.15.57:0/4219517602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 msgr2=0x56080bf76930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 0x56080bf76930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f929000bd30 tx=0x7f929000f730 comp rx=0 tx=0).stop 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 -- 172.21.15.57:0/4219517602 shutdown_connections 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf79430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71e30 0x56080bf76930 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 --2- 172.21.15.57:0/4219517602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 -- 172.21.15.57:0/4219517602 >> 172.21.15.57:0/4219517602 conn(0x56080bf6c9b0 msgr2=0x56080bf6cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f929ea00640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92840a3c30 0x7f92840a60f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x56080bf72690 tx=0x7f928c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.951+0000 7f92a95de740 1 -- 172.21.15.57:0/4219517602 shutdown_connections 2024-04-13T05:20:12.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.952+0000 7f92a95de740 1 -- 172.21.15.57:0/4219517602 wait complete. 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.952+0000 7f92a95de740 1 -- start start 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.952+0000 7f92a95de740 1 Processor -- start 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf7bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf810c0 0x56080bf834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56080bf79fa0 con 0x56080bf71440 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56080be90c80 con 0x56080bf76fa0 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56080bf7a120 con 0x56080bf810c0 2024-04-13T05:20:12.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.953+0000 7f92a95de740 10 client.0 _ll_register_callbacks cb 0x56080be90670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:12.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.954+0000 7f929f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf7bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.954+0000 7f929ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.954+0000 7f929ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf7c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59406/0 (socket says 172.21.15.57:59406) 2024-04-13T05:20:12.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.954+0000 7f929ea00640 1 -- 172.21.15.57:0/310102429 learned_addr learned my addr 172.21.15.57:0/310102429 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.954+0000 7f929fe00640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf810c0 0x56080bf834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929ea00640 1 -- 172.21.15.57:0/310102429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf810c0 msgr2=0x56080bf834b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929ea00640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf810c0 0x56080bf834b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929ea00640 1 -- 172.21.15.57:0/310102429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 msgr2=0x56080bf7bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929ea00640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf7bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929ea00640 1 -- 172.21.15.57:0/310102429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080bcb37d0 con 0x56080bf76fa0 2024-04-13T05:20:12.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929f400640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf7bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f929ea00640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf7c090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f928c010cd0 tx=0x7f928c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760169fe55a7a05b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.955+0000 7f9289600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.956+0000 7f9289600640 1 -- 172.21.15.57:0/310102429 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c019040 con 0x56080bf76fa0 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.956+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929001a910 con 0x56080bf76fa0 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.956+0000 7f9289600640 1 -- 172.21.15.57:0/310102429 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f928c0200d0 con 0x56080bf76fa0 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.956+0000 7f9289600640 1 -- 172.21.15.57:0/310102429 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f928c034050 con 0x56080bf76fa0 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.956+0000 7f92a95de740 10 client.35477 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:12.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.956+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56080bf868e0 con 0x56080bf76fa0 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.957+0000 7f9289600640 1 -- 172.21.15.57:0/310102429 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929001a910 con 0x56080bf76fa0 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f9289600640 1 -- 172.21.15.57:0/310102429 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f928c005450 con 0x56080bf76fa0 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f92a95de740 10 client.35477 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f92a95de740 10 client.35477 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f92a95de740 20 client.35477 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f92a95de740 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bf87f10 0x56080bf8a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f92a95de740 4 client.35477 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:12.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.958+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56080be76750 con 0x56080bf87f10 2024-04-13T05:20:12.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.959+0000 7f929f400640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bf87f10 0x56080bf8a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.961+0000 7f929f400640 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bf87f10 0x56080bf8a3d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=492586bb58cbf4ae server_cookie=4e1aff16645796b8 in_seq=0 out_seq=0 2024-04-13T05:20:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.961+0000 7f9289600640 10 client.35477 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.963+0000 7f9289600640 1 -- 172.21.15.57:0/310102429 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56080be76750 con 0x56080bf87f10 2024-04-13T05:20:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.963+0000 7f9289600640 10 client.35477 handle_command_reply: tid=0 2024-04-13T05:20:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.963+0000 7f92a95de740 1 client.35477 shutdown 2024-04-13T05:20:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.963+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 msgr2=0x56080bf7c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.963+0000 7f92a95de740 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf7c090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f928c010cd0 tx=0x7f928c004730 comp rx=0 tx=0).stop 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 shutdown_connections 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bf87f10 0x56080bf8a3d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf76fa0 0x56080bf7c090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f9289600640 0 client.35477 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf810c0 0x56080bf834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 --2- 172.21.15.57:0/310102429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf7bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 >> 172.21.15.57:0/310102429 conn(0x56080bf6c9b0 msgr2=0x56080bf6bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.964+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 shutdown_connections 2024-04-13T05:20:12.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.965+0000 7f92a95de740 1 -- 172.21.15.57:0/310102429 wait complete. 2024-04-13T05:20:12.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.965+0000 7f92a95de740 20 client.35477 trim_cache size 0 max 16384 2024-04-13T05:20:12.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.986+0000 7f92a95de740 1 Processor -- start 2024-04-13T05:20:12.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.986+0000 7f92a95de740 1 -- start start 2024-04-13T05:20:12.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.987+0000 7f92a95de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f92a95de740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf76fa0 0x56080bf7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f92a95de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f92a95de740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01af2e0 con 0x56080bf76fa0 2024-04-13T05:20:12.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f92a95de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01acb80 con 0x56080bf810c0 2024-04-13T05:20:12.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f92a95de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01aaa80 con 0x56080bf71440 2024-04-13T05:20:12.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f929f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f929f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf67b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:34426/0 (socket says 172.21.15.57:34426) 2024-04-13T05:20:12.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f929f400640 1 -- 172.21.15.57:0/4198627382 learned_addr learned my addr 172.21.15.57:0/4198627382 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.988+0000 7f929ea00640 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf76fa0 0x56080bf7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929fe00640 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929f400640 1 -- 172.21.15.57:0/4198627382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 msgr2=0x56080bf7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929f400640 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf7aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929f400640 1 -- 172.21.15.57:0/4198627382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf76fa0 msgr2=0x56080bf7a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929f400640 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf76fa0 0x56080bf7a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929f400640 1 -- 172.21.15.57:0/4198627382 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080bf7af40 con 0x56080bf71440 2024-04-13T05:20:12.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.989+0000 7f929ea00640 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf76fa0 0x56080bf7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:12.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.990+0000 7f929f400640 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf67b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f929000b410 tx=0x7f9290000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8055e43d0f9226 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.991+0000 7f928be00640 1 -- 172.21.15.57:0/4198627382 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290015070 con 0x56080bf71440 2024-04-13T05:20:12.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.991+0000 7f928be00640 1 -- 172.21.15.57:0/4198627382 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929000fde0 con 0x56080bf71440 2024-04-13T05:20:12.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.991+0000 7f928be00640 1 -- 172.21.15.57:0/4198627382 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92900048c0 con 0x56080bf71440 2024-04-13T05:20:12.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 -- 172.21.15.57:0/4198627382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 msgr2=0x56080bf67b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:12.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf67b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f929000b410 tx=0x7f9290000bc0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 -- 172.21.15.57:0/4198627382 shutdown_connections 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf76fa0 0x56080bf7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 --2- 172.21.15.57:0/4198627382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf71440 0x56080bf67b60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 -- 172.21.15.57:0/4198627382 >> 172.21.15.57:0/4198627382 conn(0x56080bf6c9b0 msgr2=0x56080bf72930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.992+0000 7f92a95de740 1 -- 172.21.15.57:0/4198627382 shutdown_connections 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.993+0000 7f92a95de740 1 -- 172.21.15.57:0/4198627382 wait complete. 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.993+0000 7f92a95de740 1 -- start start 2024-04-13T05:20:12.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.993+0000 7f92a95de740 1 Processor -- start 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf98f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf76fa0 0x56080bf99480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf9a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ab040 con 0x56080bf71440 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ac4b0 con 0x56080bf810c0 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ab710 con 0x56080bf76fa0 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f92a95de740 10 client.0 _ll_register_callbacks cb 0x7f928c010980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f929f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf98f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f929ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf76fa0 0x56080bf99480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.994+0000 7f929fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf9a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:12.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf9a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59424/0 (socket says 172.21.15.57:59424) 2024-04-13T05:20:12.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 -- 172.21.15.57:0/3947121462 learned_addr learned my addr 172.21.15.57:0/3947121462 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:12.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf76fa0 0x56080bf99480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:34438/0 (socket says 172.21.15.57:34438) 2024-04-13T05:20:12.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 -- 172.21.15.57:0/3947121462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf76fa0 msgr2=0x56080bf99480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf76fa0 0x56080bf99480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 -- 172.21.15.57:0/3947121462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 msgr2=0x56080bf98f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:12.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf98f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:12.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929fe00640 1 -- 172.21.15.57:0/3947121462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080bf9aa80 con 0x56080bf810c0 2024-04-13T05:20:12.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.995+0000 7f929f400640 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf98f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:12.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.996+0000 7f929fe00640 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf9a490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9298006010 tx=0x7f92980bdf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69a7a05751b724c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:12.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.996+0000 7f9289600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:12.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.997+0000 7f9289600640 1 -- 172.21.15.57:0/3947121462 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92980d7060 con 0x56080bf810c0 2024-04-13T05:20:13.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.997+0000 7f9289600640 1 -- 172.21.15.57:0/3947121462 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9298032c20 con 0x56080bf810c0 2024-04-13T05:20:13.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.997+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929000fde0 con 0x56080bf810c0 2024-04-13T05:20:13.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.997+0000 7f9289600640 1 -- 172.21.15.57:0/3947121462 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92980c26d0 con 0x56080bf810c0 2024-04-13T05:20:13.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.997+0000 7f92a95de740 10 client.35480 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:13.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.997+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56080bff68d0 con 0x56080bf810c0 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f9289600640 1 -- 172.21.15.57:0/3947121462 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929000fde0 con 0x56080bf810c0 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f9289600640 1 -- 172.21.15.57:0/3947121462 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f929803c030 con 0x56080bf810c0 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f92a95de740 10 client.35480 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f92a95de740 10 client.35480 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f92a95de740 20 client.35480 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f92a95de740 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bff71d0 0x56080bff95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f92a95de740 4 client.35480 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.998+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56080bee4a40 con 0x56080bff71d0 2024-04-13T05:20:13.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:12.999+0000 7f929f400640 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bff71d0 0x56080bff95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.000+0000 7f929f400640 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bff71d0 0x56080bff95f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7894610bc88ef566 server_cookie=b55d3bc0a2898e5e in_seq=0 out_seq=0 2024-04-13T05:20:13.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.000+0000 7f9289600640 10 client.35480 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:13.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.000+0000 7f9289600640 1 -- 172.21.15.57:0/3947121462 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56080bee4a40 con 0x56080bff71d0 2024-04-13T05:20:13.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.000+0000 7f9289600640 10 client.35480 handle_command_reply: tid=0 2024-04-13T05:20:13.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.000+0000 7f92a95de740 1 client.35480 shutdown 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 msgr2=0x56080bf9a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf9a490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9298006010 tx=0x7f92980bdf20 comp rx=0 tx=0).stop 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 shutdown_connections 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56080bff71d0 0x56080bff95f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56080bf810c0 0x56080bf9a490 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56080bf76fa0 0x56080bf99480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 --2- 172.21.15.57:0/3947121462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56080bf71440 0x56080bf98f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f9289600640 0 client.35480 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.001+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 >> 172.21.15.57:0/3947121462 conn(0x56080bf6c9b0 msgr2=0x56080bf67b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.002+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 shutdown_connections 2024-04-13T05:20:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.002+0000 7f92a95de740 1 -- 172.21.15.57:0/3947121462 wait complete. 2024-04-13T05:20:13.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.002+0000 7f92a95de740 20 client.35480 trim_cache size 0 max 16384 2024-04-13T05:20:13.005 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:20:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.004+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92840a3c30 msgr2=0x7f92840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.004+0000 7f92a5000640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92840a3c30 0x7f92840a60f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x56080bf72690 tx=0x7f928c0046c0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.004+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 msgr2=0x7f92a01a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.004+0000 7f92a5000640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a01a1860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9298009bf0 tx=0x7f9298003ba0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 shutdown_connections 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92840a3c30 0x7f92840a60f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0101fc0 0x7f92a019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92a0102980 0x7f92a01a1860 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 --2- 172.21.15.57:0/4005329090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a0100dd0 0x7f92a019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 >> 172.21.15.57:0/4005329090 conn(0x7f92a00fc360 msgr2=0x7f92a01031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 shutdown_connections 2024-04-13T05:20:13.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.005+0000 7f92a5000640 1 -- 172.21.15.57:0/4005329090 wait complete. 2024-04-13T05:20:13.028 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 -- 172.21.15.57:0/1050750443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01030b0 msgr2=0x7f53f0105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 --2- 172.21.15.57:0/1050750443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01030b0 0x7f53f0105520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009900 tx=0x7f53dc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 -- 172.21.15.57:0/1050750443 shutdown_connections 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 --2- 172.21.15.57:0/1050750443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0105a60 0x7f53f0105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 --2- 172.21.15.57:0/1050750443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01030b0 0x7f53f0105520 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 --2- 172.21.15.57:0/1050750443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0100740 0x7f53f0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 -- 172.21.15.57:0/1050750443 >> 172.21.15.57:0/1050750443 conn(0x7f53f00fc360 msgr2=0x7f53f00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 -- 172.21.15.57:0/1050750443 shutdown_connections 2024-04-13T05:20:13.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.286+0000 7f53efe00640 1 -- 172.21.15.57:0/1050750443 wait complete. 2024-04-13T05:20:13.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.287+0000 7f53efe00640 1 Processor -- start 2024-04-13T05:20:13.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.288+0000 7f53efe00640 1 -- start start 2024-04-13T05:20:13.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.289+0000 7f53efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0100740 0x7f53f019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.289+0000 7f53efe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 0x7f53f019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.289+0000 7f53efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01a3fa0 0x7f53f01a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53ee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 0x7f53f019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53ee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 0x7f53f019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:34450/0 (socket says 172.21.15.57:34450) 2024-04-13T05:20:13.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53efe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0114110 con 0x7f53f01a3fa0 2024-04-13T05:20:13.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53efe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0113f90 con 0x7f53f0100740 2024-04-13T05:20:13.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53efe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0114290 con 0x7f53f0105a60 2024-04-13T05:20:13.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53ee000640 1 -- 172.21.15.57:0/968760767 learned_addr learned my addr 172.21.15.57:0/968760767 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:13.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53eea00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0100740 0x7f53f019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.290+0000 7f53ef400640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01a3fa0 0x7f53f01a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.291+0000 7f53ee000640 1 -- 172.21.15.57:0/968760767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0100740 msgr2=0x7f53f019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.291+0000 7f53ee000640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0100740 0x7f53f019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.291+0000 7f53ee000640 1 -- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01a3fa0 msgr2=0x7f53f01a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.291+0000 7f53ee000640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01a3fa0 0x7f53f01a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.291+0000 7f53ee000640 1 -- 172.21.15.57:0/968760767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f01a6a00 con 0x7f53f0105a60 2024-04-13T05:20:13.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.291+0000 7f53eea00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0100740 0x7f53f019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:13.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.292+0000 7f53ef400640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01a3fa0 0x7f53f01a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:13.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.292+0000 7f53ee000640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 0x7f53f019ef40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53dc02c240 tx=0x7f53dc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:13.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.293+0000 7f53f4a00640 1 -- 172.21.15.57:0/968760767 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc043020 con 0x7f53f0105a60 2024-04-13T05:20:13.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.293+0000 7f53f4a00640 1 -- 172.21.15.57:0/968760767 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53dc0054c0 con 0x7f53f0105a60 2024-04-13T05:20:13.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.293+0000 7f53f4a00640 1 -- 172.21.15.57:0/968760767 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc003db0 con 0x7f53f0105a60 2024-04-13T05:20:13.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.293+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f01a6c90 con 0x7f53f0105a60 2024-04-13T05:20:13.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.293+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01a7190 con 0x7f53f0105a60 2024-04-13T05:20:13.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.299+0000 7f53f8ddc740 1 Processor -- start 2024-04-13T05:20:13.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.299+0000 7f53f8ddc740 1 -- start start 2024-04-13T05:20:13.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.300+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 0x559e77e5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e5de30 0x559e77e62930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e62fa0 0x559e77e65430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559e77dc3020 con 0x559e77e62fa0 2024-04-13T05:20:13.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e77d69950 con 0x559e77e5de30 2024-04-13T05:20:13.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e77dbefe0 con 0x559e77e5d440 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53ef400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 0x559e77e5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53ef400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 0x559e77e5d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:34466/0 (socket says 172.21.15.57:34466) 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53ef400640 1 -- 172.21.15.57:0/2218994445 learned_addr learned my addr 172.21.15.57:0/2218994445 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.301+0000 7f53eea00640 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e5de30 0x559e77e62930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.302+0000 7f53ef400640 1 -- 172.21.15.57:0/2218994445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e5de30 msgr2=0x559e77e62930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.302+0000 7f53ef400640 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e5de30 0x559e77e62930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.302+0000 7f53ef400640 1 -- 172.21.15.57:0/2218994445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e62fa0 msgr2=0x559e77e65430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:20:13.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.302+0000 7f53ef400640 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e62fa0 0x559e77e65430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.302+0000 7f53ef400640 1 -- 172.21.15.57:0/2218994445 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e77ad53f0 con 0x559e77e5d440 2024-04-13T05:20:13.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.302+0000 7f53ef400640 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 0x559e77e5d860 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53e800a870 tx=0x7f53e8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971e5f7639a3a24f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:13.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.303+0000 7f53f4a00640 1 -- 172.21.15.57:0/968760767 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f53dc046020 con 0x7f53f0105a60 2024-04-13T05:20:13.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.303+0000 7f53dbe00640 1 -- 172.21.15.57:0/2218994445 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8018020 con 0x559e77e5d440 2024-04-13T05:20:13.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.303+0000 7f53dbe00640 1 -- 172.21.15.57:0/2218994445 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8014400 con 0x559e77e5d440 2024-04-13T05:20:13.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.303+0000 7f53dbe00640 1 -- 172.21.15.57:0/2218994445 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e80135c0 con 0x559e77e5d440 2024-04-13T05:20:13.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2218994445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 msgr2=0x559e77e5d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 0x559e77e5d860 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53e800a870 tx=0x7f53e8009650 comp rx=0 tx=0).stop 2024-04-13T05:20:13.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2218994445 shutdown_connections 2024-04-13T05:20:13.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e5de30 0x559e77e62930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e62fa0 0x559e77e65430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2218994445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5d440 0x559e77e5d860 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.305+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2218994445 >> 172.21.15.57:0/2218994445 conn(0x559e77e589b0 msgr2=0x559e77e58db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.306+0000 7f53f4a00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53d40a3cb0 0x7f53d40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.306+0000 7f53f4a00640 1 -- 172.21.15.57:0/968760767 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f53dc0f27b0 con 0x7f53f0105a60 2024-04-13T05:20:13.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.307+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2218994445 shutdown_connections 2024-04-13T05:20:13.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.308+0000 7f53eea00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53d40a3cb0 0x7f53d40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.309+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2218994445 wait complete. 2024-04-13T05:20:13.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.310+0000 7f53eea00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53d40a3cb0 0x7f53d40a6170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x559e77e59920 tx=0x7f53e0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:13.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.310+0000 7f53f8ddc740 1 -- start start 2024-04-13T05:20:13.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.310+0000 7f53f8ddc740 1 Processor -- start 2024-04-13T05:20:13.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559e77e65f50 con 0x559e77e5d440 2024-04-13T05:20:13.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e77d7cc80 con 0x559e77e62fa0 2024-04-13T05:20:13.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e77e660d0 con 0x559e77e5de30 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.311+0000 7f53f8ddc740 10 client.0 _ll_register_callbacks cb 0x559e77d7c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35014/0 (socket says 172.21.15.57:35014) 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 -- 172.21.15.57:0/2259159871 learned_addr learned my addr 172.21.15.57:0/2259159871 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53ee000640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 -- 172.21.15.57:0/2259159871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 msgr2=0x559e77e67fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e67fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53ef400640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 -- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 msgr2=0x559e77e6eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e6eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.312+0000 7f53eea00640 1 -- 172.21.15.57:0/2259159871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e77b9f7d0 con 0x559e77e5d440 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.313+0000 7f53eea00640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67a70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f53e0014e10 tx=0x7f53e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43de6b52ef37898f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.313+0000 7f53d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.313+0000 7f53d8c00640 1 -- 172.21.15.57:0/2259159871 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0025070 con 0x559e77e5d440 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.314+0000 7f53d8c00640 1 -- 172.21.15.57:0/2259159871 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f53e0013920 con 0x559e77e5d440 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.314+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e8014400 con 0x559e77e5d440 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.314+0000 7f53d8c00640 1 -- 172.21.15.57:0/2259159871 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53e0031050 con 0x559e77e5d440 2024-04-13T05:20:13.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.314+0000 7f53f8ddc740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.314+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e77e723d0 con 0x559e77e5d440 2024-04-13T05:20:13.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53d8c00640 1 -- 172.21.15.57:0/2259159871 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e8014400 con 0x559e77e5d440 2024-04-13T05:20:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53d8c00640 1 -- 172.21.15.57:0/2259159871 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53e0020070 con 0x559e77e5d440 2024-04-13T05:20:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53f8ddc740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53f8ddc740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:13.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53f8ddc740 20 client.16026 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:13.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77e73a60 0x559e77e75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53f8ddc740 4 client.16026 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:13.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.315+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e77d62750 con 0x559e77e73a60 2024-04-13T05:20:13.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.316+0000 7f53ee000640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77e73a60 0x559e77e75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.317+0000 7f53ee000640 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77e73a60 0x559e77e75f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e874dc20fe96fe1 server_cookie=5c1579251b349a65 in_seq=0 out_seq=0 2024-04-13T05:20:13.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.317+0000 7f53d8c00640 10 client.16026 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:13.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.319+0000 7f53d8c00640 1 -- 172.21.15.57:0/2259159871 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e77d62750 con 0x559e77e73a60 2024-04-13T05:20:13.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.319+0000 7f53d8c00640 10 client.16026 handle_command_reply: tid=0 2024-04-13T05:20:13.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.319+0000 7f53f8ddc740 1 client.16026 shutdown 2024-04-13T05:20:13.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.319+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 msgr2=0x559e77e67a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.319+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67a70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f53e0014e10 tx=0x7f53e00046c0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 shutdown_connections 2024-04-13T05:20:13.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77e73a60 0x559e77e75f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/2259159871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67a70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 >> 172.21.15.57:0/2259159871 conn(0x559e77e589b0 msgr2=0x559e77e57cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53d8c00640 0 client.16026 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 shutdown_connections 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 1 -- 172.21.15.57:0/2259159871 wait complete. 2024-04-13T05:20:13.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.320+0000 7f53f8ddc740 20 client.16026 trim_cache size 0 max 16384 2024-04-13T05:20:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.357+0000 7f53f8ddc740 1 Processor -- start 2024-04-13T05:20:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.357+0000 7f53f8ddc740 1 -- start start 2024-04-13T05:20:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.357+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e67bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01b3ec0 con 0x559e77e5d440 2024-04-13T05:20:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01b1760 con 0x559e77e62fa0 2024-04-13T05:20:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01af5b0 con 0x559e77e5de30 2024-04-13T05:20:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35028/0 (socket says 172.21.15.57:35028) 2024-04-13T05:20:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53eea00640 1 -- 172.21.15.57:0/1515767104 learned_addr learned my addr 172.21.15.57:0/1515767104 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53ef400640 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e67bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.358+0000 7f53ee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53eea00640 1 -- 172.21.15.57:0/1515767104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 msgr2=0x559e77e6efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53eea00640 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53eea00640 1 -- 172.21.15.57:0/1515767104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 msgr2=0x559e77e67bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53eea00640 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e67bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.359+0000 7f53eea00640 1 -- 172.21.15.57:0/1515767104 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e77e54410 con 0x559e77e5d440 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.360+0000 7f53ef400640 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:13.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.360+0000 7f53eea00640 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67680 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f53e0016040 tx=0x7f53e0020640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0118e8fc1237d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:13.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.361+0000 7f53db400640 1 -- 172.21.15.57:0/1515767104 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0025040 con 0x559e77e5d440 2024-04-13T05:20:13.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.361+0000 7f53db400640 1 -- 172.21.15.57:0/1515767104 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e001b070 con 0x559e77e5d440 2024-04-13T05:20:13.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.361+0000 7f53db400640 1 -- 172.21.15.57:0/1515767104 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0020910 con 0x559e77e5d440 2024-04-13T05:20:13.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.362+0000 7f53f8ddc740 1 -- 172.21.15.57:0/1515767104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 msgr2=0x559e77e67680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.362+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67680 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f53e0016040 tx=0x7f53e0020640 comp rx=0 tx=0).stop 2024-04-13T05:20:13.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.362+0000 7f53f8ddc740 1 -- 172.21.15.57:0/1515767104 shutdown_connections 2024-04-13T05:20:13.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.362+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77e67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.362+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.363+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/1515767104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e67680 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.363+0000 7f53f8ddc740 1 -- 172.21.15.57:0/1515767104 >> 172.21.15.57:0/1515767104 conn(0x559e77e589b0 msgr2=0x559e77e53b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.363+0000 7f53f8ddc740 1 -- 172.21.15.57:0/1515767104 shutdown_connections 2024-04-13T05:20:13.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.363+0000 7f53f8ddc740 1 -- 172.21.15.57:0/1515767104 wait complete. 2024-04-13T05:20:13.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.364+0000 7f53f8ddc740 1 -- start start 2024-04-13T05:20:13.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.364+0000 7f53f8ddc740 1 Processor -- start 2024-04-13T05:20:13.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.365+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e6be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.365+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.365+0000 7f53f8ddc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77ed8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.365+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01afc20 con 0x559e77e5d440 2024-04-13T05:20:13.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.365+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01b1090 con 0x559e77e62fa0 2024-04-13T05:20:13.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.365+0000 7f53f8ddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01b02f0 con 0x559e77e5de30 2024-04-13T05:20:13.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.366+0000 7f53f8ddc740 10 client.0 _ll_register_callbacks cb 0x7f53e0014ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:13.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.366+0000 7f53ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77ed8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.366+0000 7f53ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77ed8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59488/0 (socket says 172.21.15.57:59488) 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.366+0000 7f53ef400640 1 -- 172.21.15.57:0/3653546541 learned_addr learned my addr 172.21.15.57:0/3653546541 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.366+0000 7f53ee000640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.367+0000 7f53eea00640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e6be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.367+0000 7f53ef400640 1 -- 172.21.15.57:0/3653546541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 msgr2=0x559e77e6c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.367+0000 7f53ef400640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.368+0000 7f53ef400640 1 -- 172.21.15.57:0/3653546541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 msgr2=0x559e77e6be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.368+0000 7f53ef400640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e6be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.368+0000 7f53ef400640 1 -- 172.21.15.57:0/3653546541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e77e6c9e0 con 0x559e77e62fa0 2024-04-13T05:20:13.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.368+0000 7f53eea00640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e6be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:13.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.369+0000 7f53ef400640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77ed8050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f53e800a870 tx=0x7f53e8009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23efe689265166c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:13.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.369+0000 7f53d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:13.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.370+0000 7f53d8c00640 1 -- 172.21.15.57:0/3653546541 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8018040 con 0x559e77e62fa0 2024-04-13T05:20:13.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.370+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e001b070 con 0x559e77e62fa0 2024-04-13T05:20:13.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.370+0000 7f53d8c00640 1 -- 172.21.15.57:0/3653546541 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f53e8023050 con 0x559e77e62fa0 2024-04-13T05:20:13.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.371+0000 7f53d8c00640 1 -- 172.21.15.57:0/3653546541 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53e8018040 con 0x559e77e62fa0 2024-04-13T05:20:13.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.371+0000 7f53f8ddc740 10 client.35492 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:13.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.371+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e77ed9780 con 0x559e77e62fa0 2024-04-13T05:20:13.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53d8c00640 1 -- 172.21.15.57:0/3653546541 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53e001b070 con 0x559e77e62fa0 2024-04-13T05:20:13.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53d8c00640 1 -- 172.21.15.57:0/3653546541 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f53e802e030 con 0x559e77e62fa0 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53f8ddc740 10 client.35492 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53f8ddc740 10 client.35492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53f8ddc740 20 client.35492 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77ed9e40 0x559e77edc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53f8ddc740 4 client.35492 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.372+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e77dd0a40 con 0x559e77ed9e40 2024-04-13T05:20:13.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.373+0000 7f53eea00640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77ed9e40 0x559e77edc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:13.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.374+0000 7f53eea00640 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77ed9e40 0x559e77edc260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db94065eb80fe62f server_cookie=6c6afa62950e7f6d in_seq=0 out_seq=0 2024-04-13T05:20:13.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.374+0000 7f53d8c00640 10 client.35492 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:13.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.375+0000 7f53d8c00640 1 -- 172.21.15.57:0/3653546541 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e77dd0a40 con 0x559e77ed9e40 2024-04-13T05:20:13.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.375+0000 7f53d8c00640 10 client.35492 handle_command_reply: tid=0 2024-04-13T05:20:13.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.375+0000 7f53f8ddc740 1 client.35492 shutdown 2024-04-13T05:20:13.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.375+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 msgr2=0x559e77ed8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77ed8050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f53e800a870 tx=0x7f53e8009e00 comp rx=0 tx=0).stop 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 shutdown_connections 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e77ed9e40 0x559e77edc260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e77e62fa0 0x559e77ed8050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77e5de30 0x559e77e6c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 --2- 172.21.15.57:0/3653546541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77e5d440 0x559e77e6be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 >> 172.21.15.57:0/3653546541 conn(0x559e77e589b0 msgr2=0x559e77e67680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53d8c00640 0 client.35492 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:13.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 shutdown_connections 2024-04-13T05:20:13.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.376+0000 7f53f8ddc740 1 -- 172.21.15.57:0/3653546541 wait complete. 2024-04-13T05:20:13.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.377+0000 7f53f8ddc740 20 client.35492 trim_cache size 0 max 16384 2024-04-13T05:20:13.383 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53d40a3cb0 msgr2=0x7f53d40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53d40a3cb0 0x7f53d40a6170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x559e77e59920 tx=0x7f53e0009110 comp rx=0 tx=0).stop 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 msgr2=0x7f53f019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 0x7f53f019ef40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53dc02c240 tx=0x7f53dc0042f0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 shutdown_connections 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53d40a3cb0 0x7f53d40a6170 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53f01a3fa0 0x7f53f01a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f0105a60 0x7f53f019ef40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 --2- 172.21.15.57:0/968760767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53f0100740 0x7f53f019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.378+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 >> 172.21.15.57:0/968760767 conn(0x7f53f00fc360 msgr2=0x7f53f01012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.379+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 shutdown_connections 2024-04-13T05:20:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:13.379+0000 7f53efe00640 1 -- 172.21.15.57:0/968760767 wait complete. 2024-04-13T05:20:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:13 smithi057 ceph-mon[32331]: pgmap v534: 129 pgs: 129 active+clean; 6.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 80 KiB/s wr, 27 op/s 2024-04-13T05:20:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:13 smithi189 ceph-mon[35531]: pgmap v534: 129 pgs: 129 active+clean; 6.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 80 KiB/s wr, 27 op/s 2024-04-13T05:20:13.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:13 smithi083 ceph-mon[35133]: pgmap v534: 129 pgs: 129 active+clean; 6.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 80 KiB/s wr, 27 op/s 2024-04-13T05:20:14.408 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:20:14.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:20:14.408 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:20:14.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.671+0000 7fe4dca00640 1 -- 172.21.15.57:0/3115136507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d810e4c0 msgr2=0x7fe4d81108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.671+0000 7fe4dca00640 1 --2- 172.21.15.57:0/3115136507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81108b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009cd0 tx=0x7fe4d002c100 comp rx=0 tx=0).stop 2024-04-13T05:20:14.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 -- 172.21.15.57:0/3115136507 shutdown_connections 2024-04-13T05:20:14.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 --2- 172.21.15.57:0/3115136507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8074a20 0x7fe4d8074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 --2- 172.21.15.57:0/3115136507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81108b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 --2- 172.21.15.57:0/3115136507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4d8076000 0x7fe4d8074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 -- 172.21.15.57:0/3115136507 >> 172.21.15.57:0/3115136507 conn(0x7fe4d80fd460 msgr2=0x7fe4d80ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:14.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 -- 172.21.15.57:0/3115136507 shutdown_connections 2024-04-13T05:20:14.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.672+0000 7fe4dca00640 1 -- 172.21.15.57:0/3115136507 wait complete. 2024-04-13T05:20:14.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.673+0000 7fe4dca00640 1 Processor -- start 2024-04-13T05:20:14.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.673+0000 7fe4dca00640 1 -- start start 2024-04-13T05:20:14.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.674+0000 7fe4dca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4d8074a20 0x7fe4d80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.674+0000 7fe4dca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 0x7fe4d8070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.674+0000 7fe4d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 0x7fe4d8070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.674+0000 7fe4d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 0x7fe4d8070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35058/0 (socket says 172.21.15.57:35058) 2024-04-13T05:20:14.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.674+0000 7fe4d5e00640 1 -- 172.21.15.57:0/2069229837 learned_addr learned my addr 172.21.15.57:0/2069229837 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:14.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.675+0000 7fe4d6800640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4d8074a20 0x7fe4d80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.675+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.675+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8114030 con 0x7fe4d8076000 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.675+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8113eb0 con 0x7fe4d810e4c0 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.675+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81141b0 con 0x7fe4d8074a20 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.675+0000 7fe4d7200640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d5e00640 1 -- 172.21.15.57:0/2069229837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4d8074a20 msgr2=0x7fe4d80709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4d8074a20 0x7fe4d80709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d5e00640 1 -- 172.21.15.57:0/2069229837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d810e4c0 msgr2=0x7fe4d81ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d5e00640 1 -- 172.21.15.57:0/2069229837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d81aec40 con 0x7fe4d8076000 2024-04-13T05:20:14.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d7200640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:14.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.676+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 0x7fe4d8070f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400d950 tx=0x7fe4c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:14.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.677+0000 7fe4cf400640 1 -- 172.21.15.57:0/2069229837 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4017070 con 0x7fe4d8076000 2024-04-13T05:20:14.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.677+0000 7fe4cf400640 1 -- 172.21.15.57:0/2069229837 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c4009c30 con 0x7fe4d8076000 2024-04-13T05:20:14.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.677+0000 7fe4cf400640 1 -- 172.21.15.57:0/2069229837 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400f620 con 0x7fe4d8076000 2024-04-13T05:20:14.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.677+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d81aee10 con 0x7fe4d8076000 2024-04-13T05:20:14.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.678+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81af280 con 0x7fe4d8076000 2024-04-13T05:20:14.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 Processor -- start 2024-04-13T05:20:14.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 -- start start 2024-04-13T05:20:14.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd6083a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd6083f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd60842410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55cd607a0bc0 con 0x55cd6083ff80 2024-04-13T05:20:14.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd60747660 con 0x55cd6083a420 2024-04-13T05:20:14.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.682+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd6079cb80 con 0x55cd6083ae10 2024-04-13T05:20:14.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.683+0000 7fe4cf400640 1 -- 172.21.15.57:0/2069229837 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe4c40109a0 con 0x7fe4d8076000 2024-04-13T05:20:14.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.683+0000 7fe4d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd6083a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.683+0000 7fe4d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd6083a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59496/0 (socket says 172.21.15.57:59496) 2024-04-13T05:20:14.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.683+0000 7fe4d7200640 1 -- 172.21.15.57:0/1037871545 learned_addr learned my addr 172.21.15.57:0/1037871545 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:14.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.683+0000 7fe4d6800640 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd6083f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.684+0000 7fe4d7200640 1 -- 172.21.15.57:0/1037871545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 msgr2=0x55cd6083f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:14.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.684+0000 7fe4d7200640 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd6083f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.684+0000 7fe4d7200640 1 -- 172.21.15.57:0/1037871545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 msgr2=0x55cd60842410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:14.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.684+0000 7fe4d7200640 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd60842410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.684+0000 7fe4d7200640 1 -- 172.21.15.57:0/1037871545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6076a220 con 0x55cd6083a420 2024-04-13T05:20:14.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.684+0000 7fe4d6800640 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd6083f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:14.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.685+0000 7fe4d7200640 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd6083a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0037040 tx=0x7fe4d0005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dea0a7fc908e61d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:14.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.685+0000 7fe4cf400640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4bc0a3c80 0x7fe4bc0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.685+0000 7fe4cf400640 1 -- 172.21.15.57:0/2069229837 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe4c40cfc80 con 0x7fe4d8076000 2024-04-13T05:20:14.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.685+0000 7fe4d6800640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4bc0a3c80 0x7fe4bc0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.686+0000 7fe4c3400640 1 -- 172.21.15.57:0/1037871545 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0039050 con 0x55cd6083a420 2024-04-13T05:20:14.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.686+0000 7fe4c3400640 1 -- 172.21.15.57:0/1037871545 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d003a040 con 0x55cd6083a420 2024-04-13T05:20:14.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.686+0000 7fe4c3400640 1 -- 172.21.15.57:0/1037871545 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0040b30 con 0x55cd6083a420 2024-04-13T05:20:14.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.687+0000 7fe4e03ca740 1 -- 172.21.15.57:0/1037871545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 msgr2=0x55cd6083a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.687+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd6083a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0037040 tx=0x7fe4d0005f80 comp rx=0 tx=0).stop 2024-04-13T05:20:14.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.687+0000 7fe4d6800640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4bc0a3c80 0x7fe4bc0a6140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55cd6083b210 tx=0x7fe4c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:14.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.688+0000 7fe4e03ca740 1 -- 172.21.15.57:0/1037871545 shutdown_connections 2024-04-13T05:20:14.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.688+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd60842410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.688+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd6083f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.688+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/1037871545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd6083a840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.688+0000 7fe4e03ca740 1 -- 172.21.15.57:0/1037871545 >> 172.21.15.57:0/1037871545 conn(0x55cd60835990 msgr2=0x55cd60835d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.689+0000 7fe4e03ca740 1 -- 172.21.15.57:0/1037871545 shutdown_connections 2024-04-13T05:20:14.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.689+0000 7fe4e03ca740 1 -- 172.21.15.57:0/1037871545 wait complete. 2024-04-13T05:20:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.690+0000 7fe4e03ca740 1 -- start start 2024-04-13T05:20:14.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.690+0000 7fe4e03ca740 1 Processor -- start 2024-04-13T05:20:14.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd608449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ae10 0x55cd60844f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ff80 0x55cd6084bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55cd60842ed0 con 0x55cd6083ae10 2024-04-13T05:20:14.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd6075aa30 con 0x55cd6083a420 2024-04-13T05:20:14.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd60843050 con 0x55cd6083ff80 2024-04-13T05:20:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.691+0000 7fe4e03ca740 10 client.0 _ll_register_callbacks cb 0x55cd604b2c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.692+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd608449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.692+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd608449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59510/0 (socket says 172.21.15.57:59510) 2024-04-13T05:20:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.692+0000 7fe4d6800640 1 -- 172.21.15.57:0/765221498 learned_addr learned my addr 172.21.15.57:0/765221498 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:14.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.692+0000 7fe4d7200640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ff80 0x55cd6084bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.693+0000 7fe4d6800640 1 -- 172.21.15.57:0/765221498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ff80 msgr2=0x55cd6084bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.693+0000 7fe4d6800640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ff80 0x55cd6084bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.693+0000 7fe4d6800640 1 -- 172.21.15.57:0/765221498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ae10 msgr2=0x55cd60844f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.693+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ae10 0x55cd60844f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.693+0000 7fe4d6800640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ae10 0x55cd60844f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.693+0000 7fe4d6800640 1 -- 172.21.15.57:0/765221498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6057b7e0 con 0x55cd6083a420 2024-04-13T05:20:14.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.694+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ae10 0x55cd60844f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.694+0000 7fe4d6800640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd608449c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009fd0 tx=0x7fe4c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8073b62057a9a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:14.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.694+0000 7fe4b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.695+0000 7fe4b7e00640 1 -- 172.21.15.57:0/765221498 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c802d060 con 0x55cd6083a420 2024-04-13T05:20:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.696+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d003a040 con 0x55cd6083a420 2024-04-13T05:20:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.696+0000 7fe4b7e00640 1 -- 172.21.15.57:0/765221498 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe4c8018030 con 0x55cd6083a420 2024-04-13T05:20:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.696+0000 7fe4b7e00640 1 -- 172.21.15.57:0/765221498 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4c8034050 con 0x55cd6083a420 2024-04-13T05:20:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.696+0000 7fe4e03ca740 10 client.35501 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:14.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.696+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd605dbf70 con 0x55cd6083a420 2024-04-13T05:20:14.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.697+0000 7fe4b7e00640 1 -- 172.21.15.57:0/765221498 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d003a040 con 0x55cd6083a420 2024-04-13T05:20:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4b7e00640 1 -- 172.21.15.57:0/765221498 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4c801d780 con 0x55cd6083a420 2024-04-13T05:20:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4e03ca740 10 client.35501 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4e03ca740 10 client.35501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4e03ca740 20 client.35501 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd60850820 0x55cd60852cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4e03ca740 4 client.35501 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.698+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd607ae5e0 con 0x55cd60850820 2024-04-13T05:20:14.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.699+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd60850820 0x55cd60852cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.706+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd60850820 0x55cd60852cc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6a989bb8ebc51f server_cookie=c2694bac942c7d45 in_seq=0 out_seq=0 2024-04-13T05:20:14.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.706+0000 7fe4b7e00640 10 client.35501 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:14.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4b7e00640 1 -- 172.21.15.57:0/765221498 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd607ae5e0 con 0x55cd60850820 2024-04-13T05:20:14.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4b7e00640 10 client.35501 handle_command_reply: tid=0 2024-04-13T05:20:14.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 client.35501 shutdown 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 msgr2=0x55cd608449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd608449c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009fd0 tx=0x7fe4c8004730 comp rx=0 tx=0).stop 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 shutdown_connections 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd60850820 0x55cd60852cc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ff80 0x55cd6084bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4b7e00640 0 client.35501 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ae10 0x55cd60844f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.711+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/765221498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083a420 0x55cd608449c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.712+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 >> 172.21.15.57:0/765221498 conn(0x55cd60835990 msgr2=0x55cd60834b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:14.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.712+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 shutdown_connections 2024-04-13T05:20:14.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.712+0000 7fe4e03ca740 1 -- 172.21.15.57:0/765221498 wait complete. 2024-04-13T05:20:14.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.712+0000 7fe4e03ca740 20 client.35501 trim_cache size 0 max 16384 2024-04-13T05:20:14.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.746+0000 7fe4e03ca740 1 Processor -- start 2024-04-13T05:20:14.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.747+0000 7fe4e03ca740 1 -- start start 2024-04-13T05:20:14.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.747+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083a420 0x55cd608445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.748+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd60844b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.748+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ff80 0x55cd60834300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81bc030 con 0x55cd6083a420 2024-04-13T05:20:14.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083a420 0x55cd608445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b98d0 con 0x55cd6083ff80 2024-04-13T05:20:14.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b7720 con 0x55cd6083ae10 2024-04-13T05:20:14.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4d5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd60844b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4d5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd60844b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:34520/0 (socket says 172.21.15.57:34520) 2024-04-13T05:20:14.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.749+0000 7fe4d5e00640 1 -- 172.21.15.57:0/76547764 learned_addr learned my addr 172.21.15.57:0/76547764 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:14.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d7200640 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ff80 0x55cd60834300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d5e00640 1 -- 172.21.15.57:0/76547764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ff80 msgr2=0x55cd60834300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:14.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ff80 0x55cd60834300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d5e00640 1 -- 172.21.15.57:0/76547764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083a420 msgr2=0x55cd608445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:14.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083a420 0x55cd608445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d5e00640 1 -- 172.21.15.57:0/76547764 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd60473170 con 0x55cd6083ae10 2024-04-13T05:20:14.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.750+0000 7fe4d6800640 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083a420 0x55cd608445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:14.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.751+0000 7fe4d5e00640 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd60844b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4c409bb50 tx=0x7fe4c400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26168d480cd1592e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:14.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.751+0000 7fe4c2a00640 1 -- 172.21.15.57:0/76547764 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40b5020 con 0x55cd6083ae10 2024-04-13T05:20:14.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.752+0000 7fe4c2a00640 1 -- 172.21.15.57:0/76547764 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c40a4830 con 0x55cd6083ae10 2024-04-13T05:20:14.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.752+0000 7fe4c2a00640 1 -- 172.21.15.57:0/76547764 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4020040 con 0x55cd6083ae10 2024-04-13T05:20:14.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.752+0000 7fe4e03ca740 1 -- 172.21.15.57:0/76547764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 msgr2=0x55cd60844b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.752+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd60844b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4c409bb50 tx=0x7fe4c400bf60 comp rx=0 tx=0).stop 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 -- 172.21.15.57:0/76547764 shutdown_connections 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ff80 0x55cd60834300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083ae10 0x55cd60844b10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/76547764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083a420 0x55cd608445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 -- 172.21.15.57:0/76547764 >> 172.21.15.57:0/76547764 conn(0x55cd60835990 msgr2=0x55cd60843450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 -- 172.21.15.57:0/76547764 shutdown_connections 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.753+0000 7fe4e03ca740 1 -- 172.21.15.57:0/76547764 wait complete. 2024-04-13T05:20:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.754+0000 7fe4e03ca740 1 -- start start 2024-04-13T05:20:14.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.754+0000 7fe4e03ca740 1 Processor -- start 2024-04-13T05:20:14.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083a420 0x55cd60848b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ae10 0x55cd60849080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd6084a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b7d90 con 0x55cd6083ff80 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b9200 con 0x55cd6083ae10 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b8460 con 0x55cd6083a420 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.755+0000 7fe4e03ca740 10 client.0 _ll_register_callbacks cb 0x7fe4c8009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.756+0000 7fe4d7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd6084a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.756+0000 7fe4d7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd6084a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35088/0 (socket says 172.21.15.57:35088) 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.756+0000 7fe4d7200640 1 -- 172.21.15.57:0/268272342 learned_addr learned my addr 172.21.15.57:0/268272342 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.756+0000 7fe4d7200640 1 -- 172.21.15.57:0/268272342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083a420 msgr2=0x55cd60848b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.757+0000 7fe4d7200640 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083a420 0x55cd60848b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.757+0000 7fe4d7200640 1 -- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ae10 msgr2=0x55cd60849080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:20:14.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.757+0000 7fe4d7200640 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ae10 0x55cd60849080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.757+0000 7fe4d7200640 1 -- 172.21.15.57:0/268272342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6075bbe0 con 0x55cd6083ff80 2024-04-13T05:20:14.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.757+0000 7fe4d7200640 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd6084a090 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0037040 tx=0x7fe4d00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bfc31c9649c7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:14.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.757+0000 7fe4b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:14.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.758+0000 7fe4b7e00640 1 -- 172.21.15.57:0/268272342 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0040650 con 0x55cd6083ff80 2024-04-13T05:20:14.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.758+0000 7fe4b7e00640 1 -- 172.21.15.57:0/268272342 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe4d0040450 con 0x55cd6083ff80 2024-04-13T05:20:14.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.758+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c40a4830 con 0x55cd6083ff80 2024-04-13T05:20:14.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.759+0000 7fe4b7e00640 1 -- 172.21.15.57:0/268272342 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0039020 con 0x55cd6083ff80 2024-04-13T05:20:14.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.759+0000 7fe4e03ca740 10 client.16044 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:14.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.759+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd608b6e00 con 0x55cd6083ff80 2024-04-13T05:20:14.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4b7e00640 1 -- 172.21.15.57:0/268272342 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c40a4830 con 0x55cd6083ff80 2024-04-13T05:20:14.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4b7e00640 1 -- 172.21.15.57:0/268272342 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4d0034810 con 0x55cd6083ff80 2024-04-13T05:20:14.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4e03ca740 10 client.16044 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:14.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4e03ca740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:14.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4e03ca740 20 client.16044 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:14.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd608b7250 0x55cd608b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:14.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.760+0000 7fe4e03ca740 4 client.16044 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:20:14.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.761+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd607a8450 con 0x55cd608b7250 2024-04-13T05:20:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.761+0000 7fe4d6800640 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd608b7250 0x55cd608b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.762+0000 7fe4d6800640 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd608b7250 0x55cd608b9640 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8157ee1e62467ee3 server_cookie=48e4dd4aaf666867 in_seq=0 out_seq=0 2024-04-13T05:20:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.762+0000 7fe4b7e00640 10 client.16044 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.765+0000 7fe4b7e00640 1 -- 172.21.15.57:0/268272342 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd607a8450 con 0x55cd608b7250 2024-04-13T05:20:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4b7e00640 10 client.16044 handle_command_reply: tid=0 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 client.16044 shutdown 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 msgr2=0x55cd6084a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd6084a090 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0037040 tx=0x7fe4d00327b0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 shutdown_connections 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55cd608b7250 0x55cd608b9640 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6083ae10 0x55cd60849080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6083ff80 0x55cd6084a090 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 --2- 172.21.15.57:0/268272342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6083a420 0x55cd60848b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 >> 172.21.15.57:0/268272342 conn(0x55cd60835990 msgr2=0x55cd608445d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.766+0000 7fe4b7e00640 0 client.16044 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:14.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.767+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 shutdown_connections 2024-04-13T05:20:14.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.767+0000 7fe4e03ca740 1 -- 172.21.15.57:0/268272342 wait complete. 2024-04-13T05:20:14.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.767+0000 7fe4e03ca740 20 client.16044 trim_cache size 0 max 16384 2024-04-13T05:20:14.770 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:20:14.771 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:20:14.771 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "9e8ccdc7-14ac-4a70-bc9a-34a383cee567", 2024-04-13T05:20:14.771 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:20:14.771 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4bc0a3c80 msgr2=0x7fe4bc0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4bc0a3c80 0x7fe4bc0a6140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55cd6083b210 tx=0x7fe4c8005f90 comp rx=0 tx=0).stop 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 msgr2=0x7fe4d8070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 0x7fe4d8070f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400d950 tx=0x7fe4c400de10 comp rx=0 tx=0).stop 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 shutdown_connections 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe4bc0a3c80 0x7fe4bc0a6140 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d810e4c0 0x7fe4d81ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d8076000 0x7fe4d8070f10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 --2- 172.21.15.57:0/2069229837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4d8074a20 0x7fe4d80709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:14.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 >> 172.21.15.57:0/2069229837 conn(0x7fe4d80fd460 msgr2=0x7fe4d80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:14.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.768+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 shutdown_connections 2024-04-13T05:20:14.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:14.769+0000 7fe4dca00640 1 -- 172.21.15.57:0/2069229837 wait complete. 2024-04-13T05:20:14.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:20:14.793 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.064+0000 7fdb42a00640 1 -- 172.21.15.57:0/4194601476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c0ffd70 msgr2=0x7fdb3c100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.064+0000 7fdb42a00640 1 --2- 172.21.15.57:0/4194601476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c100190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009900 tx=0x7fdb2c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 -- 172.21.15.57:0/4194601476 shutdown_connections 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 --2- 172.21.15.57:0/4194601476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c1006d0 0x7fdb3c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 --2- 172.21.15.57:0/4194601476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c107c60 0x7fdb3c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 --2- 172.21.15.57:0/4194601476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c100190 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 -- 172.21.15.57:0/4194601476 >> 172.21.15.57:0/4194601476 conn(0x7fdb3c0fb7c0 msgr2=0x7fdb3c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 -- 172.21.15.57:0/4194601476 shutdown_connections 2024-04-13T05:20:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.065+0000 7fdb42a00640 1 -- 172.21.15.57:0/4194601476 wait complete. 2024-04-13T05:20:15.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.066+0000 7fdb42a00640 1 Processor -- start 2024-04-13T05:20:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.066+0000 7fdb42a00640 1 -- start start 2024-04-13T05:20:15.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb42a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c1006d0 0x7fdb3c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb42a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c107c60 0x7fdb3c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb42a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c114180 con 0x7fdb3c1006d0 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb42a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c114000 con 0x7fdb3c0ffd70 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb42a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c114300 con 0x7fdb3c107c60 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.067+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:58612/0 (socket says 172.21.15.57:58612) 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3be00640 1 -- 172.21.15.57:0/3667685943 learned_addr learned my addr 172.21.15.57:0/3667685943 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:15.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3b400640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c1006d0 0x7fdb3c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3b400640 1 -- 172.21.15.57:0/3667685943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c107c60 msgr2=0x7fdb3c1a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3b400640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c107c60 0x7fdb3c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3b400640 1 -- 172.21.15.57:0/3667685943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 msgr2=0x7fdb3c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3b400640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3b400640 1 -- 172.21.15.57:0/3667685943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c1a6470 con 0x7fdb3c1006d0 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.068+0000 7fdb3be00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:15.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.069+0000 7fdb3b400640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c1006d0 0x7fdb3c19ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800d6a0 tx=0x7fdb2800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:15.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.069+0000 7fdb38c00640 1 -- 172.21.15.57:0/3667685943 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28017070 con 0x7fdb3c1006d0 2024-04-13T05:20:15.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.070+0000 7fdb38c00640 1 -- 172.21.15.57:0/3667685943 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb28009cd0 con 0x7fdb3c1006d0 2024-04-13T05:20:15.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.070+0000 7fdb38c00640 1 -- 172.21.15.57:0/3667685943 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28003bf0 con 0x7fdb3c1006d0 2024-04-13T05:20:15.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.070+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c1a6760 con 0x7fdb3c1006d0 2024-04-13T05:20:15.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.070+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c1a6c60 con 0x7fdb3c1006d0 2024-04-13T05:20:15.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.074+0000 7fdb38c00640 1 -- 172.21.15.57:0/3667685943 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdb28009820 con 0x7fdb3c1006d0 2024-04-13T05:20:15.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 Processor -- start 2024-04-13T05:20:15.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 -- start start 2024-04-13T05:20:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x55584198e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x5558419939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x5558419964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5558418f5020 con 0x55584198eec0 2024-04-13T05:20:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55584189b6d0 con 0x55584198e4d0 2024-04-13T05:20:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.075+0000 7fdb45994740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558418f0fe0 con 0x555841994030 2024-04-13T05:20:15.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x5558419939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x5558419939c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44182/0 (socket says 172.21.15.57:44182) 2024-04-13T05:20:15.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb3be00640 1 -- 172.21.15.57:0/630432490 learned_addr learned my addr 172.21.15.57:0/630432490 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:15.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb40a00640 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x55584198e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb3b400640 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x5558419964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb40a00640 1 -- 172.21.15.57:0/630432490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 msgr2=0x5558419964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb40a00640 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x5558419964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb40a00640 1 -- 172.21.15.57:0/630432490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 msgr2=0x5558419939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb40a00640 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x5558419939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.076+0000 7fdb40a00640 1 -- 172.21.15.57:0/630432490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558416d17d0 con 0x55584198e4d0 2024-04-13T05:20:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.077+0000 7fdb40a00640 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x55584198e8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb34007c20 tx=0x7fdb3400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f52f1775f8c519a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.077+0000 7fdb31200640 1 -- 172.21.15.57:0/630432490 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34003b70 con 0x55584198e4d0 2024-04-13T05:20:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.077+0000 7fdb31200640 1 -- 172.21.15.57:0/630432490 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb34003d10 con 0x55584198e4d0 2024-04-13T05:20:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.077+0000 7fdb31200640 1 -- 172.21.15.57:0/630432490 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400f850 con 0x55584198e4d0 2024-04-13T05:20:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.077+0000 7fdb38c00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb200a3ce0 0x7fdb200a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.078+0000 7fdb38c00640 1 -- 172.21.15.57:0/3667685943 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdb280d03b0 con 0x7fdb3c1006d0 2024-04-13T05:20:15.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.078+0000 7fdb3be00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb200a3ce0 0x7fdb200a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.078+0000 7fdb45994740 1 -- 172.21.15.57:0/630432490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 msgr2=0x55584198e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.078+0000 7fdb45994740 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x55584198e8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb34007c20 tx=0x7fdb3400c6a0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.078+0000 7fdb45994740 1 -- 172.21.15.57:0/630432490 shutdown_connections 2024-04-13T05:20:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x5558419964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x5558419939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 --2- 172.21.15.57:0/630432490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x55584198e8f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 -- 172.21.15.57:0/630432490 >> 172.21.15.57:0/630432490 conn(0x555841989a40 msgr2=0x555841989e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:15.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 -- 172.21.15.57:0/630432490 shutdown_connections 2024-04-13T05:20:15.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 -- 172.21.15.57:0/630432490 wait complete. 2024-04-13T05:20:15.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb3be00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb200a3ce0 0x7fdb200a61a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55584198a9b0 tx=0x7fdb2c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:15.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 -- start start 2024-04-13T05:20:15.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.079+0000 7fdb45994740 1 Processor -- start 2024-04-13T05:20:15.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x555841998800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x555841998d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x55584199fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555841996fe0 con 0x55584198eec0 2024-04-13T05:20:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5558418aea90 con 0x55584198e4d0 2024-04-13T05:20:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555841997160 con 0x555841994030 2024-04-13T05:20:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.080+0000 7fdb45994740 10 client.0 _ll_register_callbacks cb 0x5558418ae480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x555841998800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x555841998800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:58640/0 (socket says 172.21.15.57:58640) 2024-04-13T05:20:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 -- 172.21.15.57:0/1945186138 learned_addr learned my addr 172.21.15.57:0/1945186138 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb40a00640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x55584199fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3b400640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x555841998d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 -- 172.21.15.57:0/1945186138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 msgr2=0x55584199fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x55584199fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 -- 172.21.15.57:0/1945186138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 msgr2=0x555841998d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x555841998d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.081+0000 7fdb3be00640 1 -- 172.21.15.57:0/1945186138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558416073f0 con 0x55584198e4d0 2024-04-13T05:20:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.082+0000 7fdb3be00640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x555841998800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c004650 tx=0x7fdb2c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f788cc4c8e2e879 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.082+0000 7fdb25600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.082+0000 7fdb25600640 1 -- 172.21.15.57:0/1945186138 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c04f030 con 0x55584198e4d0 2024-04-13T05:20:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.082+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb34003d10 con 0x55584198e4d0 2024-04-13T05:20:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.082+0000 7fdb25600640 1 -- 172.21.15.57:0/1945186138 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdb2c03a040 con 0x55584198e4d0 2024-04-13T05:20:15.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.083+0000 7fdb25600640 1 -- 172.21.15.57:0/1945186138 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb2c05c050 con 0x55584198e4d0 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.083+0000 7fdb45994740 10 client.35516 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.083+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558415c70f0 con 0x55584198e4d0 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.083+0000 7fdb25600640 1 -- 172.21.15.57:0/1945186138 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb34003d10 con 0x55584198e4d0 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb25600640 1 -- 172.21.15.57:0/1945186138 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdb2c003ab0 con 0x55584198e4d0 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb45994740 10 client.35516 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb45994740 10 client.35516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb45994740 20 client.35516 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb45994740 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5558419a4660 0x5558419a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb45994740 4 client.35516 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.084+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555841894600 con 0x5558419a4660 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.085+0000 7fdb3b400640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5558419a4660 0x5558419a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.086+0000 7fdb3b400640 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5558419a4660 0x5558419a6b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec7d8d709baf38bc server_cookie=2772bb749d1778f0 in_seq=0 out_seq=0 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.086+0000 7fdb25600640 10 client.35516 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.087+0000 7fdb25600640 1 -- 172.21.15.57:0/1945186138 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555841894600 con 0x5558419a4660 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.087+0000 7fdb25600640 10 client.35516 handle_command_reply: tid=0 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.087+0000 7fdb45994740 1 client.35516 shutdown 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 msgr2=0x555841998800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x555841998800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c004650 tx=0x7fdb2c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 shutdown_connections 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5558419a4660 0x5558419a6b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555841994030 0x55584199fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198eec0 0x555841998d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 --2- 172.21.15.57:0/1945186138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198e4d0 0x555841998800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb25600640 0 client.35516 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:15.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.088+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 >> 172.21.15.57:0/1945186138 conn(0x555841989a40 msgr2=0x555841988b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.089+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 shutdown_connections 2024-04-13T05:20:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.089+0000 7fdb45994740 1 -- 172.21.15.57:0/1945186138 wait complete. 2024-04-13T05:20:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.089+0000 7fdb45994740 20 client.35516 trim_cache size 0 max 16384 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.123+0000 7fdb45994740 1 Processor -- start 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 -- start start 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 0x555841998ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198eec0 0x55584199fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555841994030 0x555841989e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b3ab0 con 0x55584198e4d0 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b1350 con 0x555841994030 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb45994740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1af1a0 con 0x55584198eec0 2024-04-13T05:20:15.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.124+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 0x555841998ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 0x555841998ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44206/0 (socket says 172.21.15.57:44206) 2024-04-13T05:20:15.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 -- 172.21.15.57:0/2574237047 learned_addr learned my addr 172.21.15.57:0/2574237047 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:15.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb40a00640 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555841994030 0x555841989e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3b400640 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198eec0 0x55584199fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 -- 172.21.15.57:0/2574237047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198eec0 msgr2=0x55584199fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198eec0 0x55584199fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 -- 172.21.15.57:0/2574237047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555841994030 msgr2=0x555841989e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555841994030 0x555841989e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3be00640 1 -- 172.21.15.57:0/2574237047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555841984030 con 0x55584198e4d0 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.125+0000 7fdb3b400640 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198eec0 0x55584199fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.126+0000 7fdb3be00640 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 0x555841998ca0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c006010 tx=0x7fdb2c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a0ed11a5d8372f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.126+0000 7fdb27e00640 1 -- 172.21.15.57:0/2574237047 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c059070 con 0x55584198e4d0 2024-04-13T05:20:15.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.126+0000 7fdb27e00640 1 -- 172.21.15.57:0/2574237047 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb2c0035f0 con 0x55584198e4d0 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb27e00640 1 -- 172.21.15.57:0/2574237047 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c04f070 con 0x55584198e4d0 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 -- 172.21.15.57:0/2574237047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 msgr2=0x555841998ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 0x555841998ca0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c006010 tx=0x7fdb2c0040f0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 -- 172.21.15.57:0/2574237047 shutdown_connections 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555841994030 0x555841989e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198eec0 0x55584199fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 --2- 172.21.15.57:0/2574237047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55584198e4d0 0x555841998ca0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 -- 172.21.15.57:0/2574237047 >> 172.21.15.57:0/2574237047 conn(0x555841989a40 msgr2=0x55584198cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 -- 172.21.15.57:0/2574237047 shutdown_connections 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.127+0000 7fdb45994740 1 -- 172.21.15.57:0/2574237047 wait complete. 2024-04-13T05:20:15.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 -- start start 2024-04-13T05:20:15.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 Processor -- start 2024-04-13T05:20:15.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198e4d0 0x55584199d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 0x55584199d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555841994030 0x55584199e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1af810 con 0x555841994030 2024-04-13T05:20:15.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b0c80 con 0x55584198eec0 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.128+0000 7fdb45994740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1afee0 con 0x55584198e4d0 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.129+0000 7fdb45994740 10 client.0 _ll_register_callbacks cb 0x7fdb2c005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.129+0000 7fdb3b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 0x55584199d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.129+0000 7fdb40a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555841994030 0x55584199e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.129+0000 7fdb3b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 0x55584199d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:58662/0 (socket says 172.21.15.57:58662) 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.129+0000 7fdb3b400640 1 -- 172.21.15.57:0/1388515787 learned_addr learned my addr 172.21.15.57:0/1388515787 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.129+0000 7fdb3be00640 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198e4d0 0x55584199d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb3b400640 1 -- 172.21.15.57:0/1388515787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198e4d0 msgr2=0x55584199d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb3b400640 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198e4d0 0x55584199d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb3b400640 1 -- 172.21.15.57:0/1388515787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555841994030 msgr2=0x55584199e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb3b400640 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555841994030 0x55584199e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb3b400640 1 -- 172.21.15.57:0/1388515787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584199ece0 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb3b400640 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 0x55584199d6e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009610 tx=0x7fdb2809bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1919213b1d456919 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.130+0000 7fdb25600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.131+0000 7fdb25600640 1 -- 172.21.15.57:0/1388515787 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28020040 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.131+0000 7fdb25600640 1 -- 172.21.15.57:0/1388515787 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdb28003d90 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.131+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2c0035f0 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.131+0000 7fdb25600640 1 -- 172.21.15.57:0/1388515787 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb2809c9d0 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.131+0000 7fdb45994740 10 client.35522 fetch_fsmap learned FSMap version 27 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.131+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558419a0060 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb25600640 1 -- 172.21.15.57:0/1388515787 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb2c0035f0 con 0x55584198eec0 2024-04-13T05:20:15.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb25600640 1 -- 172.21.15.57:0/1388515787 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdb28098940 con 0x55584198eec0 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb45994740 10 client.35522 fetch_fsmap finished waiting for FSMap version 27 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb45994740 10 client.35522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb45994740 20 client.35522 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb45994740 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555841a0b440 0x555841a0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb45994740 4 client.35522 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.132+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55584198fcf0 con 0x555841a0b440 2024-04-13T05:20:15.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.133+0000 7fdb3be00640 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555841a0b440 0x555841a0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:15.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.133+0000 7fdb3be00640 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555841a0b440 0x555841a0d860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1b7aca4fc7ce94 server_cookie=f5010077558b167e in_seq=0 out_seq=0 2024-04-13T05:20:15.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.133+0000 7fdb25600640 10 client.35522 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.134+0000 7fdb25600640 1 -- 172.21.15.57:0/1388515787 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55584198fcf0 con 0x555841a0b440 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.134+0000 7fdb25600640 10 client.35522 handle_command_reply: tid=0 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.134+0000 7fdb45994740 1 client.35522 shutdown 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.134+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 msgr2=0x55584199d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.134+0000 7fdb45994740 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 0x55584199d6e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009610 tx=0x7fdb2809bed0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 shutdown_connections 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x555841a0b440 0x555841a0d860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55584198eec0 0x55584199d6e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555841994030 0x55584199e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 --2- 172.21.15.57:0/1388515787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55584198e4d0 0x55584199d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb25600640 0 client.35522 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 >> 172.21.15.57:0/1388515787 conn(0x555841989a40 msgr2=0x55584198d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 shutdown_connections 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 1 -- 172.21.15.57:0/1388515787 wait complete. 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.135+0000 7fdb45994740 20 client.35522 trim_cache size 0 max 16384 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:20:15.139 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout: "9e8ccdc7-14ac-4a70-bc9a-34a383cee567": { 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "9e8ccdc7-14ac-4a70-bc9a-34a383cee567", 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:20:15.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb200a3ce0 msgr2=0x7fdb200a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb200a3ce0 0x7fdb200a61a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55584198a9b0 tx=0x7fdb2c02c7e0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c1006d0 msgr2=0x7fdb3c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c1006d0 0x7fdb3c19ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800d6a0 tx=0x7fdb2800db60 comp rx=0 tx=0).stop 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 shutdown_connections 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb200a3ce0 0x7fdb200a61a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c107c60 0x7fdb3c1a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb3c1006d0 0x7fdb3c19ede0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 --2- 172.21.15.57:0/3667685943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb3c0ffd70 0x7fdb3c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.136+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 >> 172.21.15.57:0/3667685943 conn(0x7fdb3c0fb7c0 msgr2=0x7fdb3c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.137+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 shutdown_connections 2024-04-13T05:20:15.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:15.137+0000 7fdb42a00640 1 -- 172.21.15.57:0/3667685943 wait complete. 2024-04-13T05:20:15.161 INFO:tasks.cephfs.filesystem:scrub status for tag:9e8ccdc7-14ac-4a70-bc9a-34a383cee567 - {'path': '/', 'tag': '9e8ccdc7-14ac-4a70-bc9a-34a383cee567', 'options': 'recursive,force'} 2024-04-13T05:20:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:15 smithi057 ceph-mon[32331]: pgmap v535: 129 pgs: 129 active+clean; 11 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 488 KiB/s wr, 43 op/s 2024-04-13T05:20:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:15 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:20:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:15 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:15 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:20:15.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:15 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:15 smithi189 ceph-mon[35531]: pgmap v535: 129 pgs: 129 active+clean; 11 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 488 KiB/s wr, 43 op/s 2024-04-13T05:20:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:15 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:20:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:15 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:15 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:20:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:15 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:15 smithi083 ceph-mon[35133]: pgmap v535: 129 pgs: 129 active+clean; 11 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 488 KiB/s wr, 43 op/s 2024-04-13T05:20:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:15 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:20:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:15 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:15 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:20:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:15 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:16 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:20:16.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:16 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:16.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:16 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:20:16.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:16 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:16.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:16 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:20:16.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:16 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:17.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:17 smithi189 ceph-mon[35531]: pgmap v536: 129 pgs: 129 active+clean; 14 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 763 KiB/s wr, 61 op/s 2024-04-13T05:20:17.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:17 smithi083 ceph-mon[35133]: pgmap v536: 129 pgs: 129 active+clean; 14 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 763 KiB/s wr, 61 op/s 2024-04-13T05:20:17.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:17 smithi057 ceph-mon[32331]: pgmap v536: 129 pgs: 129 active+clean; 14 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 763 KiB/s wr, 61 op/s 2024-04-13T05:20:19.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:19 smithi189 ceph-mon[35531]: pgmap v537: 129 pgs: 129 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 789 KiB/s wr, 55 op/s 2024-04-13T05:20:19.635 INFO:tasks.workunit.client.0.smithi057.stdout: 5 7386 392 3600 195 1764 410 2024-04-13T05:20:19.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:19 smithi083 ceph-mon[35133]: pgmap v537: 129 pgs: 129 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 789 KiB/s wr, 55 op/s 2024-04-13T05:20:19.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:19 smithi057 ceph-mon[32331]: pgmap v537: 129 pgs: 129 active+clean; 14 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 789 KiB/s wr, 55 op/s 2024-04-13T05:20:21.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:21 smithi189 ceph-mon[35531]: pgmap v538: 129 pgs: 129 active+clean; 23 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-04-13T05:20:21.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:21 smithi083 ceph-mon[35133]: pgmap v538: 129 pgs: 129 active+clean; 23 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-04-13T05:20:21.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:21 smithi057 ceph-mon[32331]: pgmap v538: 129 pgs: 129 active+clean; 23 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-04-13T05:20:22.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:22 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:20:22.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:22 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:20:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:22 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:20:23.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:23 smithi189 ceph-mon[35531]: pgmap v539: 129 pgs: 129 active+clean; 24 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-04-13T05:20:23.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:23 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:20:23.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:23 smithi057 ceph-mon[32331]: pgmap v539: 129 pgs: 129 active+clean; 24 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-04-13T05:20:23.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:23 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:20:23.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:23 smithi083 ceph-mon[35133]: pgmap v539: 129 pgs: 129 active+clean; 24 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-04-13T05:20:23.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:23 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:20:25.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:25 smithi189 ceph-mon[35531]: pgmap v540: 129 pgs: 129 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-04-13T05:20:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:25 smithi083 ceph-mon[35133]: pgmap v540: 129 pgs: 129 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-04-13T05:20:25.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:25 smithi057 ceph-mon[32331]: pgmap v540: 129 pgs: 129 active+clean; 30 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-04-13T05:20:27.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:27 smithi083 ceph-mon[35133]: pgmap v541: 129 pgs: 129 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-04-13T05:20:27.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:27 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:20:27.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:27 smithi057 ceph-mon[32331]: pgmap v541: 129 pgs: 129 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-04-13T05:20:27.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:27 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:20:27.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:27 smithi189 ceph-mon[35531]: pgmap v541: 129 pgs: 129 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-04-13T05:20:27.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:27 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:20:29.635 INFO:tasks.workunit.client.0.smithi057.stdout: 7 11656 104 5447 146 1250 300 2024-04-13T05:20:29.636 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:29 smithi189 ceph-mon[35531]: pgmap v542: 129 pgs: 129 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-04-13T05:20:29.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:29 smithi083 ceph-mon[35133]: pgmap v542: 129 pgs: 129 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-04-13T05:20:29.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:29 smithi057 ceph-mon[32331]: pgmap v542: 129 pgs: 129 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-04-13T05:20:31.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:31 smithi083 ceph-mon[35133]: pgmap v543: 129 pgs: 129 active+clean; 37 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-13T05:20:31.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:31 smithi057 ceph-mon[32331]: pgmap v543: 129 pgs: 129 active+clean; 37 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-13T05:20:31.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:31 smithi189 ceph-mon[35531]: pgmap v543: 129 pgs: 129 active+clean; 37 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-13T05:20:33.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:33 smithi083 ceph-mon[35133]: pgmap v544: 129 pgs: 129 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-13T05:20:33.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:33 smithi057 ceph-mon[32331]: pgmap v544: 129 pgs: 129 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-13T05:20:33.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:33 smithi189 ceph-mon[35531]: pgmap v544: 129 pgs: 129 active+clean; 38 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-13T05:20:35.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:35 smithi083 ceph-mon[35133]: pgmap v545: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-13T05:20:35.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:35 smithi057 ceph-mon[32331]: pgmap v545: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-13T05:20:35.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:35 smithi189 ceph-mon[35531]: pgmap v545: 129 pgs: 129 active+clean; 41 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-13T05:20:37.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:37 smithi083 ceph-mon[35133]: pgmap v546: 129 pgs: 129 active+clean; 43 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-04-13T05:20:37.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:37 smithi057 ceph-mon[32331]: pgmap v546: 129 pgs: 129 active+clean; 43 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-04-13T05:20:37.835 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:37 smithi189 ceph-mon[35531]: pgmap v546: 129 pgs: 129 active+clean; 43 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-04-13T05:20:38.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:38 smithi083 ceph-mon[35133]: pgmap v547: 129 pgs: 129 active+clean; 43 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-04-13T05:20:38.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:38 smithi057 ceph-mon[32331]: pgmap v547: 129 pgs: 129 active+clean; 43 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-04-13T05:20:38.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:38 smithi189 ceph-mon[35531]: pgmap v547: 129 pgs: 129 active+clean; 43 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-04-13T05:20:39.635 INFO:tasks.workunit.client.0.smithi057.stdout: 9 8305 144 4701 101 4880 289 2024-04-13T05:20:41.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:40 smithi189 ceph-mon[35531]: pgmap v548: 129 pgs: 129 active+clean; 47 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-04-13T05:20:41.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:40 smithi083 ceph-mon[35133]: pgmap v548: 129 pgs: 129 active+clean; 47 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-04-13T05:20:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:40 smithi057 ceph-mon[32331]: pgmap v548: 129 pgs: 129 active+clean; 47 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-04-13T05:20:43.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:42 smithi189 ceph-mon[35531]: pgmap v549: 129 pgs: 129 active+clean; 48 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-04-13T05:20:43.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:42 smithi083 ceph-mon[35133]: pgmap v549: 129 pgs: 129 active+clean; 48 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-04-13T05:20:43.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:42 smithi057 ceph-mon[32331]: pgmap v549: 129 pgs: 129 active+clean; 48 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-04-13T05:20:45.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:44 smithi189 ceph-mon[35531]: pgmap v550: 129 pgs: 129 active+clean; 54 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-04-13T05:20:45.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:20:45.162 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:20:45.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:44 smithi057 ceph-mon[32331]: pgmap v550: 129 pgs: 129 active+clean; 54 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-04-13T05:20:45.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:44 smithi083 ceph-mon[35133]: pgmap v550: 129 pgs: 129 active+clean; 54 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-04-13T05:20:45.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 -- 172.21.15.57:0/2873093263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 msgr2=0x7ff2a4107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 --2- 172.21.15.57:0/2873093263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4107810 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff294009860 tx=0x7ff29402bb70 comp rx=0 tx=0).stop 2024-04-13T05:20:45.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 -- 172.21.15.57:0/2873093263 shutdown_connections 2024-04-13T05:20:45.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 --2- 172.21.15.57:0/2873093263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a4107d50 0x7ff2a410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 --2- 172.21.15.57:0/2873093263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4107810 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 --2- 172.21.15.57:0/2873093263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a4102af0 0x7ff2a4104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 -- 172.21.15.57:0/2873093263 >> 172.21.15.57:0/2873093263 conn(0x7ff2a40fc480 msgr2=0x7ff2a40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 -- 172.21.15.57:0/2873093263 shutdown_connections 2024-04-13T05:20:45.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.422+0000 7ff2ab400640 1 -- 172.21.15.57:0/2873093263 wait complete. 2024-04-13T05:20:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.423+0000 7ff2ab400640 1 Processor -- start 2024-04-13T05:20:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 -- start start 2024-04-13T05:20:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a4102af0 0x7ff2a4196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a4107d50 0x7ff2a41a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410dba0 con 0x7ff2a4105420 2024-04-13T05:20:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410da20 con 0x7ff2a4107d50 2024-04-13T05:20:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.424+0000 7ff2ab400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410dd20 con 0x7ff2a4102af0 2024-04-13T05:20:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45762/0 (socket says 172.21.15.57:45762) 2024-04-13T05:20:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 -- 172.21.15.57:0/3641241772 learned_addr learned my addr 172.21.15.57:0/3641241772 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:45.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a4107d50 0x7ff2a41a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a4102af0 0x7ff2a4196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 -- 172.21.15.57:0/3641241772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a4102af0 msgr2=0x7ff2a4196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a4102af0 0x7ff2a4196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 -- 172.21.15.57:0/3641241772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a4107d50 msgr2=0x7ff2a41a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a4107d50 0x7ff2a41a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.425+0000 7ff2a3e00640 1 -- 172.21.15.57:0/3641241772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a41a6590 con 0x7ff2a4105420 2024-04-13T05:20:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.426+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4196680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff294000c00 tx=0x7ff294004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.426+0000 7ff2a1600640 1 -- 172.21.15.57:0/3641241772 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294039040 con 0x7ff2a4105420 2024-04-13T05:20:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.426+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a41a6820 con 0x7ff2a4105420 2024-04-13T05:20:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.426+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41a6d60 con 0x7ff2a4105420 2024-04-13T05:20:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.426+0000 7ff2a1600640 1 -- 172.21.15.57:0/3641241772 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff294042070 con 0x7ff2a4105420 2024-04-13T05:20:45.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.426+0000 7ff2a1600640 1 -- 172.21.15.57:0/3641241772 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294004920 con 0x7ff2a4105420 2024-04-13T05:20:45.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.431+0000 7ff2a1600640 1 -- 172.21.15.57:0/3641241772 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff294005050 con 0x7ff2a4105420 2024-04-13T05:20:45.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.433+0000 7ff2ae3b7740 1 Processor -- start 2024-04-13T05:20:45.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.433+0000 7ff2ae3b7740 1 -- start start 2024-04-13T05:20:45.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.434+0000 7ff2a1600640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2880a3c80 0x7ff2880a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.434+0000 7ff2a1600640 1 -- 172.21.15.57:0/3641241772 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff2940bc2c0 con 0x7ff2a4105420 2024-04-13T05:20:45.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.434+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2880a3c80 0x7ff2880a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c1238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c1289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c129030 0x55d59c12b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d59c08a020 con 0x55d59c123ec0 2024-04-13T05:20:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d59c0306d0 con 0x55d59c1234d0 2024-04-13T05:20:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d59c085fe0 con 0x55d59c129030 2024-04-13T05:20:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c1289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c1289c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45764/0 (socket says 172.21.15.57:45764) 2024-04-13T05:20:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 -- 172.21.15.57:0/778047374 learned_addr learned my addr 172.21.15.57:0/778047374 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 -- 172.21.15.57:0/778047374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c129030 msgr2=0x55d59c12b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c129030 0x55d59c12b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 -- 172.21.15.57:0/778047374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 msgr2=0x55d59c1238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c1238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.435+0000 7ff2a3e00640 1 -- 172.21.15.57:0/778047374 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59be667d0 con 0x55d59c123ec0 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.436+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c1289c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b230 tx=0x7ff2940c5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2d06e0439b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.436+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2880a3c80 0x7ff2880a6140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff29c005fd0 tx=0x7ff29c00f7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.436+0000 7ff299c00640 1 -- 172.21.15.57:0/778047374 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940ba960 con 0x55d59c123ec0 2024-04-13T05:20:45.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.436+0000 7ff299c00640 1 -- 172.21.15.57:0/778047374 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2940de050 con 0x55d59c123ec0 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.437+0000 7ff299c00640 1 -- 172.21.15.57:0/778047374 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940bab00 con 0x55d59c123ec0 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.437+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/778047374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 msgr2=0x55d59c1289c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.437+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c1289c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b230 tx=0x7ff2940c5680 comp rx=0 tx=0).stop 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/778047374 shutdown_connections 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c129030 0x55d59c12b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c1289c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/778047374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c1238f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/778047374 >> 172.21.15.57:0/778047374 conn(0x55d59c11ea40 msgr2=0x55d59c11ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/778047374 shutdown_connections 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/778047374 wait complete. 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.438+0000 7ff2ae3b7740 1 -- start start 2024-04-13T05:20:45.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 Processor -- start 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c12d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c12d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c129030 0x55d59c1349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d59c08a020 con 0x55d59c129030 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d59c043a90 con 0x55d59c1234d0 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.439+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d59c12bda0 con 0x55d59c123ec0 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2ae3b7740 10 client.0 _ll_register_callbacks cb 0x55d59c043480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c12d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c12d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59482/0 (socket says 172.21.15.57:59482) 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a8a00640 1 -- 172.21.15.57:0/1962416993 learned_addr learned my addr 172.21.15.57:0/1962416993 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a9400640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c129030 0x55d59c1349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c12d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a8a00640 1 -- 172.21.15.57:0/1962416993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 msgr2=0x55d59c12d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c12d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.440+0000 7ff2a8a00640 1 -- 172.21.15.57:0/1962416993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c129030 msgr2=0x55d59c1349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c129030 0x55d59c1349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff2a8a00640 1 -- 172.21.15.57:0/1962416993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59bd9c3f0 con 0x55d59c1234d0 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c12d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff2a9400640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c129030 0x55d59c1349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c12d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff28c011410 tx=0x7ff28c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbf2392b477a154 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff292000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:45.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.441+0000 7ff292000640 1 -- 172.21.15.57:0/1962416993 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004460 con 0x55d59c1234d0 2024-04-13T05:20:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.442+0000 7ff292000640 1 -- 172.21.15.57:0/1962416993 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff28c0035f0 con 0x55d59c1234d0 2024-04-13T05:20:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.442+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2940de050 con 0x55d59c1234d0 2024-04-13T05:20:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.443+0000 7ff292000640 1 -- 172.21.15.57:0/1962416993 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff28c024050 con 0x55d59c1234d0 2024-04-13T05:20:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.443+0000 7ff2ae3b7740 10 client.35531 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.443+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d59bd5c0f0 con 0x55d59c1234d0 2024-04-13T05:20:45.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.443+0000 7ff292000640 1 -- 172.21.15.57:0/1962416993 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2940de050 con 0x55d59c1234d0 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff292000640 1 -- 172.21.15.57:0/1962416993 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff28c012ab0 con 0x55d59c1234d0 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2ae3b7740 10 client.35531 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2ae3b7740 10 client.35531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2ae3b7740 20 client.35531 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c139350 0x55d59c13b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2ae3b7740 4 client.35531 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d59c029600 con 0x55d59c139350 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.445+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c139350 0x55d59c13b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.446+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c139350 0x55d59c13b7b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99dfe6868db1dfc2 server_cookie=70504c5c8e4b23b0 in_seq=0 out_seq=0 2024-04-13T05:20:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.446+0000 7ff292000640 10 client.35531 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.447+0000 7ff292000640 1 -- 172.21.15.57:0/1962416993 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d59c029600 con 0x55d59c139350 2024-04-13T05:20:45.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.447+0000 7ff292000640 10 client.35531 handle_command_reply: tid=0 2024-04-13T05:20:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.447+0000 7ff2ae3b7740 1 client.35531 shutdown 2024-04-13T05:20:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 msgr2=0x55d59c12d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c12d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff28c011410 tx=0x7ff28c0118d0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 shutdown_connections 2024-04-13T05:20:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c139350 0x55d59c13b7b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c129030 0x55d59c1349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c12d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/1962416993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c12d440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 >> 172.21.15.57:0/1962416993 conn(0x55d59c11ea40 msgr2=0x55d59c11db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.448+0000 7ff292000640 0 client.35531 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:45.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.449+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 shutdown_connections 2024-04-13T05:20:45.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.449+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/1962416993 wait complete. 2024-04-13T05:20:45.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.449+0000 7ff2ae3b7740 20 client.35531 trim_cache size 0 max 16384 2024-04-13T05:20:45.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.480+0000 7ff2ae3b7740 1 Processor -- start 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 -- start start 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 0x55d59c119a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c134980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c140070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2800124c0 con 0x55d59c11ea40 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff28000fd30 con 0x55d59c1234d0 2024-04-13T05:20:45.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.481+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff28000db80 con 0x55d59c123ec0 2024-04-13T05:20:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 0x55d59c119a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 0x55d59c119a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45786/0 (socket says 172.21.15.57:45786) 2024-04-13T05:20:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 -- 172.21.15.57:0/4220051744 learned_addr learned my addr 172.21.15.57:0/4220051744 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a3e00640 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a9400640 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 -- 172.21.15.57:0/4220051744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 msgr2=0x55d59c140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 -- 172.21.15.57:0/4220051744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 msgr2=0x55d59c134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c134980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.482+0000 7ff2a8a00640 1 -- 172.21.15.57:0/4220051744 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59c119030 con 0x55d59c11ea40 2024-04-13T05:20:45.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.483+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 0x55d59c119a10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009b40 tx=0x7ff28c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cc8a58d7ec9546 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.483+0000 7ff299200640 1 -- 172.21.15.57:0/4220051744 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c02c070 con 0x55d59c11ea40 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.483+0000 7ff299200640 1 -- 172.21.15.57:0/4220051744 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff28c016070 con 0x55d59c11ea40 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.483+0000 7ff299200640 1 -- 172.21.15.57:0/4220051744 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c003850 con 0x55d59c11ea40 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.484+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/4220051744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 msgr2=0x55d59c119a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.484+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 0x55d59c119a10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009b40 tx=0x7ff28c005c50 comp rx=0 tx=0).stop 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/4220051744 shutdown_connections 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c123ec0 0x55d59c140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/4220051744 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c11ea40 0x55d59c119a10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/4220051744 >> 172.21.15.57:0/4220051744 conn(0x55d59c128b70 msgr2=0x55d59c122530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/4220051744 shutdown_connections 2024-04-13T05:20:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.485+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/4220051744 wait complete. 2024-04-13T05:20:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.486+0000 7ff2ae3b7740 1 -- start start 2024-04-13T05:20:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.486+0000 7ff2ae3b7740 1 Processor -- start 2024-04-13T05:20:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.487+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c11ea40 0x55d59c131740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.487+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c131c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.487+0000 7ff2ae3b7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff28000e1f0 con 0x55d59c123ec0 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff28000f660 con 0x55d59c1234d0 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c11ea40 0x55d59c131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2ae3b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff28000e8c0 con 0x55d59c11ea40 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2ae3b7740 10 client.0 _ll_register_callbacks cb 0x7ff28c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c11ea40 0x55d59c131740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56004/0 (socket says 172.21.15.57:56004) 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2a8a00640 1 -- 172.21.15.57:0/3468501191 learned_addr learned my addr 172.21.15.57:0/3468501191 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:45.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.488+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff2a9400640 1 -- 172.21.15.57:0/3468501191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c11ea40 msgr2=0x55d59c131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c11ea40 0x55d59c131740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff2a9400640 1 -- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 msgr2=0x55d59c131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:45.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c131c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff2a9400640 1 -- 172.21.15.57:0/3468501191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59c19de20 con 0x55d59c123ec0 2024-04-13T05:20:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff2a9400640 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c19d8e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009600 tx=0x7ff29c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9ffbdcf821a86b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.489+0000 7ff292000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.490+0000 7ff292000640 1 -- 172.21.15.57:0/3468501191 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0273f0 con 0x55d59c123ec0 2024-04-13T05:20:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.490+0000 7ff292000640 1 -- 172.21.15.57:0/3468501191 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff29c027590 con 0x55d59c123ec0 2024-04-13T05:20:45.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.490+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c016070 con 0x55d59c123ec0 2024-04-13T05:20:45.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.490+0000 7ff292000640 1 -- 172.21.15.57:0/3468501191 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff29c027790 con 0x55d59c123ec0 2024-04-13T05:20:45.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.490+0000 7ff2ae3b7740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.490+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d59c19f080 con 0x55d59c123ec0 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff292000640 1 -- 172.21.15.57:0/3468501191 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff28c016070 con 0x55d59c123ec0 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff292000640 1 -- 172.21.15.57:0/3468501191 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff29c026070 con 0x55d59c123ec0 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff2ae3b7740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff2ae3b7740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff2ae3b7740 20 client.16077 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c129030 0x55d59c1a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff2ae3b7740 4 client.16077 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.491+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d59c097a40 con 0x55d59c129030 2024-04-13T05:20:45.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.492+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c129030 0x55d59c1a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.494+0000 7ff2a8a00640 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c129030 0x55d59c1a1b20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6904a2d8baad900c server_cookie=3426f4450242a3bd in_seq=0 out_seq=0 2024-04-13T05:20:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.494+0000 7ff292000640 10 client.16077 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:45.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff292000640 1 -- 172.21.15.57:0/3468501191 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d59c097a40 con 0x55d59c129030 2024-04-13T05:20:45.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff292000640 10 client.16077 handle_command_reply: tid=0 2024-04-13T05:20:45.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 client.16077 shutdown 2024-04-13T05:20:45.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 msgr2=0x55d59c19d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c19d8e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009600 tx=0x7ff29c004a40 comp rx=0 tx=0).stop 2024-04-13T05:20:45.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 shutdown_connections 2024-04-13T05:20:45.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d59c129030 0x55d59c1a1b20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59c1234d0 0x55d59c131c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d59c123ec0 0x55d59c19d8e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 --2- 172.21.15.57:0/3468501191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d59c11ea40 0x55d59c131740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.495+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 >> 172.21.15.57:0/3468501191 conn(0x55d59c128b70 msgr2=0x55d59c11d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.496+0000 7ff292000640 0 client.16077 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.496+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 shutdown_connections 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.496+0000 7ff2ae3b7740 1 -- 172.21.15.57:0/3468501191 wait complete. 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.496+0000 7ff2ae3b7740 20 client.16077 trim_cache size 0 max 16384 2024-04-13T05:20:45.499 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:20:45.500 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:20:45.500 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:20:45.500 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2880a3c80 msgr2=0x7ff2880a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2880a3c80 0x7ff2880a6140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff29c005fd0 tx=0x7ff29c00f7d0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 msgr2=0x7ff2a4196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4196680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff294000c00 tx=0x7ff294004270 comp rx=0 tx=0).stop 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 shutdown_connections 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2880a3c80 0x7ff2880a6140 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a4107d50 0x7ff2a41a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a4105420 0x7ff2a4196680 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 --2- 172.21.15.57:0/3641241772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a4102af0 0x7ff2a4196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.497+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 >> 172.21.15.57:0/3641241772 conn(0x7ff2a40fc480 msgr2=0x7ff2a41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.498+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 shutdown_connections 2024-04-13T05:20:45.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.498+0000 7ff2ab400640 1 -- 172.21.15.57:0/3641241772 wait complete. 2024-04-13T05:20:45.521 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:20:45.521 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 -- 172.21.15.57:0/320612108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c075fa0 msgr2=0x7f961c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 --2- 172.21.15.57:0/320612108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c075fa0 0x7f961c0743f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f960c009900 tx=0x7f960c02bcc0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 -- 172.21.15.57:0/320612108 shutdown_connections 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 --2- 172.21.15.57:0/320612108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109cf0 0x7f961c10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 --2- 172.21.15.57:0/320612108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c0749c0 0x7f961c074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 --2- 172.21.15.57:0/320612108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c075fa0 0x7f961c0743f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.785+0000 7f961b200640 1 -- 172.21.15.57:0/320612108 >> 172.21.15.57:0/320612108 conn(0x7f961c0fd3d0 msgr2=0x7f961c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:45.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.786+0000 7f961b200640 1 -- 172.21.15.57:0/320612108 shutdown_connections 2024-04-13T05:20:45.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.786+0000 7f961b200640 1 -- 172.21.15.57:0/320612108 wait complete. 2024-04-13T05:20:45.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.787+0000 7f961b200640 1 Processor -- start 2024-04-13T05:20:45.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.787+0000 7f961b200640 1 -- start start 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c0749c0 0x7f961c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c075fa0 0x7f961c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 0x7f961c1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f961c10fbf0 con 0x7f961c109cf0 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f961c10fa70 con 0x7f961c075fa0 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f961c10fd70 con 0x7f961c0749c0 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 0x7f961c1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 0x7f961c1a5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45822/0 (socket says 172.21.15.57:45822) 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f961a800640 1 -- 172.21.15.57:0/1307636240 learned_addr learned my addr 172.21.15.57:0/1307636240 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f9619400640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c075fa0 0x7f961c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.788+0000 7f9619e00640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c0749c0 0x7f961c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.789+0000 7f961a800640 1 -- 172.21.15.57:0/1307636240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c0749c0 msgr2=0x7f961c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.789+0000 7f961a800640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c0749c0 0x7f961c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.789+0000 7f961a800640 1 -- 172.21.15.57:0/1307636240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c075fa0 msgr2=0x7f961c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:45.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.789+0000 7f961a800640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c075fa0 0x7f961c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:45.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.789+0000 7f961a800640 1 -- 172.21.15.57:0/1307636240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1a6290 con 0x7f961c109cf0 2024-04-13T05:20:45.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.790+0000 7f961a800640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 0x7f961c1a5c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f961400eaa0 tx=0x7f961400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.790+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400cdb0 con 0x7f961c109cf0 2024-04-13T05:20:45.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.790+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9614005020 con 0x7f961c109cf0 2024-04-13T05:20:45.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.790+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c1a6580 con 0x7f961c109cf0 2024-04-13T05:20:45.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.791+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1a6a20 con 0x7f961c109cf0 2024-04-13T05:20:45.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.791+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140116d0 con 0x7f961c109cf0 2024-04-13T05:20:45.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.792+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c1046a0 con 0x7f961c109cf0 2024-04-13T05:20:45.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.796+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f96140051d0 con 0x7f961c109cf0 2024-04-13T05:20:45.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.798+0000 7f9612a00640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:45.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.798+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f96140d0670 con 0x7f961c109cf0 2024-04-13T05:20:45.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.800+0000 7f9619e00640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:45.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.801+0000 7f9619e00640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f960c000c00 tx=0x7f960c02c870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:45.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:45.806+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9614016030 con 0x7f961c109cf0 2024-04-13T05:20:46.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.250+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f961c062c60 con 0x7f961c109cf0 2024-04-13T05:20:46.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.252+0000 7f9612a00640 1 -- 172.21.15.57:0/1307636240 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11859 (secure 0 0 0) 0x7f961409ccb0 con 0x7f961c109cf0 2024-04-13T05:20:46.254 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:20:46.254 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:20:25.642360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:20:46.255 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 28 2024-04-13T05:20:46.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.255+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 msgr2=0x7f96000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.255+0000 7f961b200640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f960c000c00 tx=0x7f960c02c870 comp rx=0 tx=0).stop 2024-04-13T05:20:46.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.255+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 msgr2=0x7f961c1a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.255+0000 7f961b200640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 0x7f961c1a5c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f961400eaa0 tx=0x7f961400ef60 comp rx=0 tx=0).stop 2024-04-13T05:20:46.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 shutdown_connections 2024-04-13T05:20:46.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c075fa0 0x7f961c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109cf0 0x7f961c1a5c20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 --2- 172.21.15.57:0/1307636240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c0749c0 0x7f961c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 >> 172.21.15.57:0/1307636240 conn(0x7f961c0fd3d0 msgr2=0x7f961c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 shutdown_connections 2024-04-13T05:20:46.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.256+0000 7f961b200640 1 -- 172.21.15.57:0/1307636240 wait complete. 2024-04-13T05:20:46.281 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 -- 172.21.15.57:0/2854791183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d00075fa0 msgr2=0x7f1d000743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 --2- 172.21.15.57:0/2854791183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d00075fa0 0x7f1d000743f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009900 tx=0x7f1cf002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 -- 172.21.15.57:0/2854791183 shutdown_connections 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 --2- 172.21.15.57:0/2854791183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d000749c0 0x7f1d00074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 --2- 172.21.15.57:0/2854791183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d00109ea0 0x7f1d0010c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 --2- 172.21.15.57:0/2854791183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d00075fa0 0x7f1d000743f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.548+0000 7f1d07600640 1 -- 172.21.15.57:0/2854791183 >> 172.21.15.57:0/2854791183 conn(0x7f1d000fd5a0 msgr2=0x7f1d000ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.549+0000 7f1d07600640 1 -- 172.21.15.57:0/2854791183 shutdown_connections 2024-04-13T05:20:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.549+0000 7f1d07600640 1 -- 172.21.15.57:0/2854791183 wait complete. 2024-04-13T05:20:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.550+0000 7f1d07600640 1 Processor -- start 2024-04-13T05:20:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.550+0000 7f1d07600640 1 -- start start 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d07600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 0x7f1d00196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d07600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00075fa0 0x7f1d001965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d07600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d00109ea0 0x7f1d0019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d07600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0010fda0 con 0x7f1d000749c0 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d07600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0010fc20 con 0x7f1d00075fa0 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d07600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0010ff20 con 0x7f1d00109ea0 2024-04-13T05:20:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 0x7f1d00196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 0x7f1d00196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45846/0 (socket says 172.21.15.57:45846) 2024-04-13T05:20:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d04c00640 1 -- 172.21.15.57:0/2306830891 learned_addr learned my addr 172.21.15.57:0/2306830891 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00075fa0 0x7f1d001965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.551+0000 7f1d05600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d00109ea0 0x7f1d0019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1d04c00640 1 -- 172.21.15.57:0/2306830891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d00109ea0 msgr2=0x7f1d0019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1d04c00640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d00109ea0 0x7f1d0019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1d04c00640 1 -- 172.21.15.57:0/2306830891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00075fa0 msgr2=0x7f1d001965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1d04c00640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00075fa0 0x7f1d001965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1d04c00640 1 -- 172.21.15.57:0/2306830891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d001aece0 con 0x7f1d000749c0 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00075fa0 0x7f1d001965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.552+0000 7f1d04c00640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 0x7f1d00196080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1cf002c2e0 tx=0x7f1cf002dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.553+0000 7f1cfe000640 1 -- 172.21.15.57:0/2306830891 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0042020 con 0x7f1d000749c0 2024-04-13T05:20:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.553+0000 7f1cfe000640 1 -- 172.21.15.57:0/2306830891 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf00054c0 con 0x7f1d000749c0 2024-04-13T05:20:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.553+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d001aef70 con 0x7f1d000749c0 2024-04-13T05:20:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.553+0000 7f1cfe000640 1 -- 172.21.15.57:0/2306830891 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0003e50 con 0x7f1d000749c0 2024-04-13T05:20:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.553+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d001af430 con 0x7f1d000749c0 2024-04-13T05:20:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.558+0000 7f1d0a5db740 1 Processor -- start 2024-04-13T05:20:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.558+0000 7f1cfe000640 1 -- 172.21.15.57:0/2306830891 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1cf0045020 con 0x7f1d000749c0 2024-04-13T05:20:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 -- start start 2024-04-13T05:20:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 0x55b6fa5ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa50a020 con 0x55b6fa5a9db0 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa4b0950 con 0x55b6fa5a4250 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa505fe0 con 0x55b6fa5a4c40 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.559+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 0x55b6fa5ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 0x55b6fa5ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45850/0 (socket says 172.21.15.57:45850) 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 -- 172.21.15.57:0/4180467470 learned_addr learned my addr 172.21.15.57:0/4180467470 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1d05600640 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cfe000640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1ce40a3e50 0x7f1ce40a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cfe000640 1 -- 172.21.15.57:0/2306830891 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1cf00f2610 con 0x7f1d000749c0 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 -- 172.21.15.57:0/4180467470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 msgr2=0x55b6fa5a9740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:20:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5a9740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 -- 172.21.15.57:0/4180467470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 msgr2=0x55b6fa5a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.560+0000 7f1cf7e00640 1 -- 172.21.15.57:0/4180467470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6fa21c3f0 con 0x55b6fa5a9db0 2024-04-13T05:20:46.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.561+0000 7f1d05600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1ce40a3e50 0x7f1ce40a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.561+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 0x55b6fa5ac240 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009c20 tx=0x7f1cec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aef28f54b18ebc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.561+0000 7f1cf5c00640 1 -- 172.21.15.57:0/4180467470 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec016070 con 0x55b6fa5a9db0 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.561+0000 7f1cf5c00640 1 -- 172.21.15.57:0/4180467470 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cec011920 con 0x55b6fa5a9db0 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.561+0000 7f1cf5c00640 1 -- 172.21.15.57:0/4180467470 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0107a0 con 0x55b6fa5a9db0 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.562+0000 7f1d0a5db740 1 -- 172.21.15.57:0/4180467470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 msgr2=0x55b6fa5ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.562+0000 7f1d05600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1ce40a3e50 0x7f1ce40a6310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55b6fa5a54a0 tx=0x7f1cf8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.562+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 0x55b6fa5ac240 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009c20 tx=0x7f1cec00e6c0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.563+0000 7f1d0a5db740 1 -- 172.21.15.57:0/4180467470 shutdown_connections 2024-04-13T05:20:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.563+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a9db0 0x55b6fa5ac240 secure :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009c20 tx=0x7f1cec00e6c0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.563+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.563+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/4180467470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.563+0000 7f1d0a5db740 1 -- 172.21.15.57:0/4180467470 >> 172.21.15.57:0/4180467470 conn(0x55b6fa59f7c0 msgr2=0x55b6fa59fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.563+0000 7f1d0a5db740 1 -- 172.21.15.57:0/4180467470 shutdown_connections 2024-04-13T05:20:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.564+0000 7f1d0a5db740 1 -- 172.21.15.57:0/4180467470 wait complete. 2024-04-13T05:20:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.564+0000 7f1d0a5db740 1 -- start start 2024-04-13T05:20:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.564+0000 7f1d0a5db740 1 Processor -- start 2024-04-13T05:20:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa5acd60 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa4c3c80 con 0x55b6fa5a4250 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa5acee0 con 0x55b6fa5a4c40 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d0a5db740 10 client.0 _ll_register_callbacks cb 0x55b6fa4c3670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.565+0000 7f1d05600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59540/0 (socket says 172.21.15.57:59540) 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d04c00640 1 -- 172.21.15.57:0/2852694599 learned_addr learned my addr 172.21.15.57:0/2852694599 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d05600640 1 -- 172.21.15.57:0/2852694599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 msgr2=0x55b6fa5aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d05600640 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d05600640 1 -- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 msgr2=0x55b6fa5ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d05600640 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.566+0000 7f1d05600640 1 -- 172.21.15.57:0/2852694599 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6fa2e67d0 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.567+0000 7f1d05600640 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5b62d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8005fd0 tx=0x7f1cf8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f18c005c76c4df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.567+0000 7f1cea000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.567+0000 7f1cea000640 1 -- 172.21.15.57:0/2852694599 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf801ea30 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.567+0000 7f1cea000640 1 -- 172.21.15.57:0/2852694599 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1cf801ebd0 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.567+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cec011920 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.568+0000 7f1cea000640 1 -- 172.21.15.57:0/2852694599 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1cf801edd0 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.568+0000 7f1d0a5db740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.568+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6fa5b9710 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.568+0000 7f1cea000640 1 -- 172.21.15.57:0/2852694599 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cec011920 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.568+0000 7f1cea000640 1 -- 172.21.15.57:0/2852694599 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1cf800f040 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.569+0000 7f1d0a5db740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.569+0000 7f1d0a5db740 10 client.16095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.569+0000 7f1d0a5db740 20 client.16095 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.569+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa5badc0 0x55b6fa5bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.569+0000 7f1d0a5db740 4 client.16095 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.569+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6fa4a9750 con 0x55b6fa5badc0 2024-04-13T05:20:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.570+0000 7f1d04c00640 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa5badc0 0x55b6fa5bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.572+0000 7f1d04c00640 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa5badc0 0x55b6fa5bd1e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac7bdcb03dbe98fd server_cookie=3b3c74910f23b8d in_seq=0 out_seq=0 2024-04-13T05:20:46.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.572+0000 7f1cea000640 10 client.16095 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.573+0000 7f1cea000640 1 -- 172.21.15.57:0/2852694599 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6fa4a9750 con 0x55b6fa5badc0 2024-04-13T05:20:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.573+0000 7f1cea000640 10 client.16095 handle_command_reply: tid=0 2024-04-13T05:20:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 client.16095 shutdown 2024-04-13T05:20:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5b3ee0 msgr2=0x55b6fa5b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5b62d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8005fd0 tx=0x7f1cf8004ab0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 shutdown_connections 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa5badc0 0x55b6fa5bd1e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1cea000640 0 client.16095 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5b62d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2852694599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.574+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 >> 172.21.15.57:0/2852694599 conn(0x55b6fa59f7c0 msgr2=0x55b6fa59ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.575+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 shutdown_connections 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.575+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2852694599 wait complete. 2024-04-13T05:20:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.575+0000 7f1d0a5db740 20 client.16095 trim_cache size 0 max 16384 2024-04-13T05:20:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 Processor -- start 2024-04-13T05:20:46.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 -- start start 2024-04-13T05:20:46.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa59e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001bc2d0 con 0x55b6fa5a4c40 2024-04-13T05:20:46.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001b9b40 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.602+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001b7990 con 0x55b6fa5a4250 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45864/0 (socket says 172.21.15.57:45864) 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 -- 172.21.15.57:0/2244921189 learned_addr learned my addr 172.21.15.57:0/2244921189 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1d05600640 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa59e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1d04c00640 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 -- 172.21.15.57:0/2244921189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4250 msgr2=0x55b6fa5ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 -- 172.21.15.57:0/2244921189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5b3ee0 msgr2=0x55b6fa59e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa59e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.603+0000 7f1cf7e00640 1 -- 172.21.15.57:0/2244921189 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6fa5bd550 con 0x55b6fa5a4c40 2024-04-13T05:20:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.604+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5ad990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1cec018000 tx=0x7f1cec00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75af824d546b13dd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.604+0000 7f1cf5200640 1 -- 172.21.15.57:0/2244921189 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec028070 con 0x55b6fa5a4c40 2024-04-13T05:20:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.604+0000 7f1cf5200640 1 -- 172.21.15.57:0/2244921189 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cec009d90 con 0x55b6fa5a4c40 2024-04-13T05:20:46.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.604+0000 7f1cf5200640 1 -- 172.21.15.57:0/2244921189 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec016040 con 0x55b6fa5a4c40 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.605+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2244921189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 msgr2=0x55b6fa5ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.605+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5ad990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1cec018000 tx=0x7f1cec00eef0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.605+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2244921189 shutdown_connections 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.605+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa59e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.606+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4c40 0x55b6fa5ad990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.606+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/2244921189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5a4250 0x55b6fa5ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.606+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2244921189 >> 172.21.15.57:0/2244921189 conn(0x55b6fa59f7c0 msgr2=0x55b6fa5ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.606+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2244921189 shutdown_connections 2024-04-13T05:20:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.606+0000 7f1d0a5db740 1 -- 172.21.15.57:0/2244921189 wait complete. 2024-04-13T05:20:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.606+0000 7f1d0a5db740 1 -- start start 2024-04-13T05:20:46.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 Processor -- start 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 0x55b6fa5cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5badc0 0x55b6fa5cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001b8000 con 0x55b6fa5a4250 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001b9470 con 0x55b6fa5badc0 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001b86d0 con 0x55b6fa5b3ee0 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d0a5db740 10 client.0 _ll_register_callbacks cb 0x7f1cf8013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.607+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 0x55b6fa5cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 0x55b6fa5cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45872/0 (socket says 172.21.15.57:45872) 2024-04-13T05:20:46.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 -- 172.21.15.57:0/107344380 learned_addr learned my addr 172.21.15.57:0/107344380 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d05600640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5badc0 0x55b6fa5cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 -- 172.21.15.57:0/107344380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5b3ee0 msgr2=0x55b6fa5cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 -- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5badc0 msgr2=0x55b6fa5cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5badc0 0x55b6fa5cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d04c00640 1 -- 172.21.15.57:0/107344380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6fa5cd9f0 con 0x55b6fa5a4250 2024-04-13T05:20:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.608+0000 7f1d05600640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5badc0 0x55b6fa5cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.609+0000 7f1d04c00640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 0x55b6fa5cbf50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00a90b0 tx=0x7f1cf0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb20984a6569101 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.609+0000 7f1cea000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.609+0000 7f1cea000640 1 -- 172.21.15.57:0/107344380 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00be4f0 con 0x55b6fa5a4250 2024-04-13T05:20:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.609+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cec009d90 con 0x55b6fa5a4250 2024-04-13T05:20:46.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.609+0000 7f1cea000640 1 -- 172.21.15.57:0/107344380 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1cf00efe20 con 0x55b6fa5a4250 2024-04-13T05:20:46.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.610+0000 7f1cea000640 1 -- 172.21.15.57:0/107344380 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1cf00c28a0 con 0x55b6fa5a4250 2024-04-13T05:20:46.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.610+0000 7f1d0a5db740 10 client.16101 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.610+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6fa621050 con 0x55b6fa5a4250 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1cea000640 1 -- 172.21.15.57:0/107344380 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cec009d90 con 0x55b6fa5a4250 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1cea000640 1 -- 172.21.15.57:0/107344380 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1cf00035f0 con 0x55b6fa5a4250 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1d0a5db740 10 client.16101 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1d0a5db740 10 client.16101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1d0a5db740 20 client.16101 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa621a00 0x55b6fa623e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1d0a5db740 4 client.16101 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:20:46.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.611+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6fa517a40 con 0x55b6fa621a00 2024-04-13T05:20:46.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.612+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa621a00 0x55b6fa623e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.613+0000 7f1cea000640 10 client.16101 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:46.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.613+0000 7f1cf7e00640 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa621a00 0x55b6fa623e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d6fb762cd235d86 server_cookie=f59ff4c3da80dabe in_seq=0 out_seq=0 2024-04-13T05:20:46.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.614+0000 7f1cea000640 1 -- 172.21.15.57:0/107344380 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6fa517a40 con 0x55b6fa621a00 2024-04-13T05:20:46.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.614+0000 7f1cea000640 10 client.16101 handle_command_reply: tid=0 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.614+0000 7f1d0a5db740 1 client.16101 shutdown 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.614+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 msgr2=0x55b6fa5cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.614+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 0x55b6fa5cbf50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00a90b0 tx=0x7f1cf0005e50 comp rx=0 tx=0).stop 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 shutdown_connections 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55b6fa621a00 0x55b6fa623e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6fa5badc0 0x55b6fa5cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6fa5b3ee0 0x55b6fa5cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 --2- 172.21.15.57:0/107344380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b6fa5a4250 0x55b6fa5cbf50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 >> 172.21.15.57:0/107344380 conn(0x55b6fa59f7c0 msgr2=0x55b6fa59e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1cea000640 0 client.16101 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:20:46.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 shutdown_connections 2024-04-13T05:20:46.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.615+0000 7f1d0a5db740 1 -- 172.21.15.57:0/107344380 wait complete. 2024-04-13T05:20:46.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.616+0000 7f1d0a5db740 20 client.16101 trim_cache size 0 max 16384 2024-04-13T05:20:46.620 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:20:46.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1ce40a3e50 msgr2=0x7f1ce40a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1ce40a3e50 0x7f1ce40a6310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55b6fa5a54a0 tx=0x7f1cf8004a40 comp rx=0 tx=0).stop 2024-04-13T05:20:46.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 msgr2=0x7f1d00196080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 0x7f1d00196080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1cf002c2e0 tx=0x7f1cf002dc40 comp rx=0 tx=0).stop 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 shutdown_connections 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1ce40a3e50 0x7f1ce40a6310 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00075fa0 0x7f1d001965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d00109ea0 0x7f1d0019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 --2- 172.21.15.57:0/2306830891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d000749c0 0x7f1d00196080 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.617+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 >> 172.21.15.57:0/2306830891 conn(0x7f1d000fd5a0 msgr2=0x7f1d00101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.618+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 shutdown_connections 2024-04-13T05:20:46.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.618+0000 7f1d07600640 1 -- 172.21.15.57:0/2306830891 wait complete. 2024-04-13T05:20:46.642 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:20:46.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.938+0000 7f7a62c00640 1 -- 172.21.15.57:0/3706184942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c109810 msgr2=0x7f7a5c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.938+0000 7f7a62c00640 1 --2- 172.21.15.57:0/3706184942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c109810 0x7f7a5c109c90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009950 tx=0x7f7a4802bc80 comp rx=0 tx=0).stop 2024-04-13T05:20:46.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 -- 172.21.15.57:0/3706184942 shutdown_connections 2024-04-13T05:20:46.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 --2- 172.21.15.57:0/3706184942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c10a1d0 0x7f7a5c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 --2- 172.21.15.57:0/3706184942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c109810 0x7f7a5c109c90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 --2- 172.21.15.57:0/3706184942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a5c108620 0x7f7a5c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 -- 172.21.15.57:0/3706184942 >> 172.21.15.57:0/3706184942 conn(0x7f7a5c0775c0 msgr2=0x7f7a5c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 -- 172.21.15.57:0/3706184942 shutdown_connections 2024-04-13T05:20:46.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.939+0000 7f7a62c00640 1 -- 172.21.15.57:0/3706184942 wait complete. 2024-04-13T05:20:46.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.940+0000 7f7a62c00640 1 Processor -- start 2024-04-13T05:20:46.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.940+0000 7f7a62c00640 1 -- start start 2024-04-13T05:20:46.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a62c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 0x7f7a5c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a62c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c109810 0x7f7a5c070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a62c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a5c10a1d0 0x7f7a5c1ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a62c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c114220 con 0x7f7a5c108620 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a62c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1140a0 con 0x7f7a5c10a1d0 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a62c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1143a0 con 0x7f7a5c109810 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.941+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 0x7f7a5c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 0x7f7a5c0709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45896/0 (socket says 172.21.15.57:45896) 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 -- 172.21.15.57:0/1461703225 learned_addr learned my addr 172.21.15.57:0/1461703225 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5b400640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c109810 0x7f7a5c070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 -- 172.21.15.57:0/1461703225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c109810 msgr2=0x7f7a5c070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c109810 0x7f7a5c070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 -- 172.21.15.57:0/1461703225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a5c10a1d0 msgr2=0x7f7a5c1ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a5c10a1d0 0x7f7a5c1ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5be00640 1 -- 172.21.15.57:0/1461703225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c1aee70 con 0x7f7a5c108620 2024-04-13T05:20:46.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.942+0000 7f7a5b400640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c109810 0x7f7a5c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:46.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.943+0000 7f7a5be00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 0x7f7a5c0709d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b450 tx=0x7f7a4c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.943+0000 7f7a58c00640 1 -- 172.21.15.57:0/1461703225 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c015020 con 0x7f7a5c108620 2024-04-13T05:20:46.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.943+0000 7f7a58c00640 1 -- 172.21.15.57:0/1461703225 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a4c011d40 con 0x7f7a5c108620 2024-04-13T05:20:46.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.943+0000 7f7a58c00640 1 -- 172.21.15.57:0/1461703225 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00f340 con 0x7f7a5c108620 2024-04-13T05:20:46.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.943+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c1af040 con 0x7f7a5c108620 2024-04-13T05:20:46.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.944+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1af5e0 con 0x7f7a5c108620 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.949+0000 7f7a58c00640 1 -- 172.21.15.57:0/1461703225 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a4c018020 con 0x7f7a5c108620 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.949+0000 7f7a65bce740 1 Processor -- start 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.949+0000 7f7a65bce740 1 -- start start 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.950+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dc2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.950+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dc7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.950+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.950+0000 7f7a65bce740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555e25d17020 con 0x555e25dc2260 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.950+0000 7f7a65bce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e25cbd950 con 0x555e25dc7dc0 2024-04-13T05:20:46.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.950+0000 7f7a65bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e25d12fe0 con 0x555e25dc2c50 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.951+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.951+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59568/0 (socket says 172.21.15.57:59568) 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.951+0000 7f7a5b400640 1 -- 172.21.15.57:0/3185362536 learned_addr learned my addr 172.21.15.57:0/3185362536 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.951+0000 7f7a5be00640 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dc7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.951+0000 7f7a60c00640 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dc2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a58c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a400a3c30 0x7f7a400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a5b400640 1 -- 172.21.15.57:0/3185362536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 msgr2=0x555e25dc7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a5b400640 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dc7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a5b400640 1 -- 172.21.15.57:0/3185362536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 msgr2=0x555e25dc2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a5b400640 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dc2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a5b400640 1 -- 172.21.15.57:0/3185362536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e25a293f0 con 0x555e25dc7dc0 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a58c00640 1 -- 172.21.15.57:0/1461703225 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7a4c0d02c0 con 0x7f7a5c108620 2024-04-13T05:20:46.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a60c00640 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dc2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:46.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.952+0000 7f7a5b400640 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dca250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a480058c0 tx=0x7f7a48005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1850fff4fa64b66a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.953+0000 7f7a51200640 1 -- 172.21.15.57:0/3185362536 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48042070 con 0x555e25dc7dc0 2024-04-13T05:20:46.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.953+0000 7f7a51200640 1 -- 172.21.15.57:0/3185362536 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a4802cbb0 con 0x555e25dc7dc0 2024-04-13T05:20:46.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.953+0000 7f7a51200640 1 -- 172.21.15.57:0/3185362536 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48004920 con 0x555e25dc7dc0 2024-04-13T05:20:46.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.953+0000 7f7a60c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a400a3c30 0x7f7a400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.954+0000 7f7a65bce740 1 -- 172.21.15.57:0/3185362536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 msgr2=0x555e25dca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.954+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dca250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a480058c0 tx=0x7f7a48005d50 comp rx=0 tx=0).stop 2024-04-13T05:20:46.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.955+0000 7f7a60c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a400a3c30 0x7f7a400a60f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7a540049d0 tx=0x7f7a540096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.955+0000 7f7a65bce740 1 -- 172.21.15.57:0/3185362536 shutdown_connections 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.955+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dca250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.955+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dc7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.955+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3185362536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dc2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.955+0000 7f7a65bce740 1 -- 172.21.15.57:0/3185362536 >> 172.21.15.57:0/3185362536 conn(0x555e25dac7c0 msgr2=0x555e25dacbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.956+0000 7f7a65bce740 1 -- 172.21.15.57:0/3185362536 shutdown_connections 2024-04-13T05:20:46.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.956+0000 7f7a65bce740 1 -- 172.21.15.57:0/3185362536 wait complete. 2024-04-13T05:20:46.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.956+0000 7f7a65bce740 1 -- start start 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.956+0000 7f7a65bce740 1 Processor -- start 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dd3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555e25dcad10 con 0x555e25dc2260 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e25cd0c80 con 0x555e25dc7dc0 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e25dcae90 con 0x555e25dc2c50 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a65bce740 10 client.0 _ll_register_callbacks cb 0x555e25cd0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45926/0 (socket says 172.21.15.57:45926) 2024-04-13T05:20:46.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.957+0000 7f7a5be00640 1 -- 172.21.15.57:0/3792917642 learned_addr learned my addr 172.21.15.57:0/3792917642 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a60c00640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dd3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a5b400640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a5be00640 1 -- 172.21.15.57:0/3792917642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 msgr2=0x555e25dccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a5be00640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a5be00640 1 -- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 msgr2=0x555e25dd3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a5be00640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dd3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.958+0000 7f7a5be00640 1 -- 172.21.15.57:0/3792917642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e25af37d0 con 0x555e25dc2260 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a5be00640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00be10 tx=0x7f7a4c0104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e6249d85448aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a45600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a45600640 1 -- 172.21.15.57:0/3792917642 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c09bb00 con 0x555e25dc2260 2024-04-13T05:20:46.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a45600640 1 -- 172.21.15.57:0/3792917642 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7a4c098810 con 0x555e25dc2260 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a4802cbb0 con 0x555e25dc2260 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a45600640 1 -- 172.21.15.57:0/3792917642 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a4c009280 con 0x555e25dc2260 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a65bce740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.959+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e25dd6fe0 con 0x555e25dc2260 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a45600640 1 -- 172.21.15.57:0/3792917642 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a4802cbb0 con 0x555e25dc2260 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a45600640 1 -- 172.21.15.57:0/3792917642 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f7a4c0a0100 con 0x555e25dc2260 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a65bce740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a65bce740 10 client.16110 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a65bce740 20 client.16110 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25dd8730 0x555e25ddab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a65bce740 4 client.16110 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.960+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e25cb6750 con 0x555e25dd8730 2024-04-13T05:20:46.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.961+0000 7f7a5b400640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25dd8730 0x555e25ddab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.961+0000 7f7a5b400640 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25dd8730 0x555e25ddab80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18530c250738cc4c server_cookie=d2b15c92a9a72274 in_seq=0 out_seq=0 2024-04-13T05:20:46.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.961+0000 7f7a45600640 10 client.16110 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:46.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.964+0000 7f7a45600640 1 -- 172.21.15.57:0/3792917642 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e25cb6750 con 0x555e25dd8730 2024-04-13T05:20:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.964+0000 7f7a45600640 10 client.16110 handle_command_reply: tid=0 2024-04-13T05:20:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.964+0000 7f7a65bce740 1 client.16110 shutdown 2024-04-13T05:20:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 msgr2=0x555e25dcc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:46.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00be10 tx=0x7f7a4c0104b0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 shutdown_connections 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25dd8730 0x555e25ddab80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dd3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 --2- 172.21.15.57:0/3792917642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc710 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 >> 172.21.15.57:0/3792917642 conn(0x555e25dac7c0 msgr2=0x555e25daba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:46.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a45600640 0 client.16110 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.965+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 shutdown_connections 2024-04-13T05:20:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.966+0000 7f7a65bce740 1 -- 172.21.15.57:0/3792917642 wait complete. 2024-04-13T05:20:46.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.966+0000 7f7a65bce740 20 client.16110 trim_cache size 0 max 16384 2024-04-13T05:20:46.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.989+0000 7f7a65bce740 1 Processor -- start 2024-04-13T05:20:46.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 -- start start 2024-04-13T05:20:46.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dcc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:46.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1b3bd0 con 0x555e25dc2260 2024-04-13T05:20:46.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1b1440 con 0x555e25dc7dc0 2024-04-13T05:20:46.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a65bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c0829a0 con 0x555e25dc2c50 2024-04-13T05:20:46.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.990+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dcc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:46.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dcc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59576/0 (socket says 172.21.15.57:59576) 2024-04-13T05:20:46.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 -- 172.21.15.57:0/721432675 learned_addr learned my addr 172.21.15.57:0/721432675 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:46.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a5b400640 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 -- 172.21.15.57:0/721432675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 msgr2=0x555e25dd3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dd3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 -- 172.21.15.57:0/721432675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 msgr2=0x555e25dcc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.991+0000 7f7a60c00640 1 -- 172.21.15.57:0/721432675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e25dcb7d0 con 0x555e25dc7dc0 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.992+0000 7f7a60c00640 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dcc860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x555e25dc34b0 tx=0x7f7a54015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e919bfd78ea07c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.992+0000 7f7a47e00640 1 -- 172.21.15.57:0/721432675 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54004d20 con 0x555e25dc7dc0 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.992+0000 7f7a47e00640 1 -- 172.21.15.57:0/721432675 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a54017070 con 0x555e25dc7dc0 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.992+0000 7f7a47e00640 1 -- 172.21.15.57:0/721432675 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540277e0 con 0x555e25dc7dc0 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 -- 172.21.15.57:0/721432675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 msgr2=0x555e25dcc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dcc860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x555e25dc34b0 tx=0x7f7a54015000 comp rx=0 tx=0).stop 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 -- 172.21.15.57:0/721432675 shutdown_connections 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc7dc0 0x555e25dcc860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2c50 0x555e25dd3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 --2- 172.21.15.57:0/721432675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc2260 0x555e25dcc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.993+0000 7f7a65bce740 1 -- 172.21.15.57:0/721432675 >> 172.21.15.57:0/721432675 conn(0x555e25dac7c0 msgr2=0x555e25dacba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.994+0000 7f7a65bce740 1 -- 172.21.15.57:0/721432675 shutdown_connections 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.994+0000 7f7a65bce740 1 -- 172.21.15.57:0/721432675 wait complete. 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.994+0000 7f7a65bce740 1 -- start start 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.994+0000 7f7a65bce740 1 Processor -- start 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 0x555e25dd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc2c50 0x555e25dd1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc7dc0 0x555e25dd2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1af900 con 0x555e25dc7dc0 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1b0d70 con 0x555e25dc2c50 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1affd0 con 0x555e25dc2260 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a65bce740 10 client.0 _ll_register_callbacks cb 0x7f7a4c098440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 0x555e25dd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 0x555e25dd0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56144/0 (socket says 172.21.15.57:56144) 2024-04-13T05:20:47.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.995+0000 7f7a5be00640 1 -- 172.21.15.57:0/599891984 learned_addr learned my addr 172.21.15.57:0/599891984 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.996+0000 7f7a60c00640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc7dc0 0x555e25dd2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.996+0000 7f7a5b400640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc2c50 0x555e25dd1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a5be00640 1 -- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc2c50 msgr2=0x555e25dd1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a5be00640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc2c50 0x555e25dd1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a5be00640 1 -- 172.21.15.57:0/599891984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc7dc0 msgr2=0x555e25dd2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a5be00640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc7dc0 0x555e25dd2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a5be00640 1 -- 172.21.15.57:0/599891984 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e25dd2830 con 0x555e25dc2260 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a5be00640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 0x555e25dd0d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009fd0 tx=0x7f7a4c09c0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e71845c8b8bd84a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.997+0000 7f7a45600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.998+0000 7f7a45600640 1 -- 172.21.15.57:0/599891984 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b4050 con 0x555e25dc2260 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.998+0000 7f7a45600640 1 -- 172.21.15.57:0/599891984 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7a4c017b50 con 0x555e25dc2260 2024-04-13T05:20:47.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.998+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a54017070 con 0x555e25dc2260 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.999+0000 7f7a45600640 1 -- 172.21.15.57:0/599891984 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a4c09bb00 con 0x555e25dc2260 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.999+0000 7f7a65bce740 10 client.25513 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.999+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e25dd3ba0 con 0x555e25dc2260 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.999+0000 7f7a45600640 1 -- 172.21.15.57:0/599891984 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a54017070 con 0x555e25dc2260 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:46.999+0000 7f7a45600640 1 -- 172.21.15.57:0/599891984 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f7a4c09b060 con 0x555e25dc2260 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a65bce740 10 client.25513 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a65bce740 10 client.25513 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:20:47.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a65bce740 20 client.25513 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a65bce740 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25e3ef10 0x555e25e41300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a65bce740 4 client.25513 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e25d24a40 con 0x555e25e3ef10 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.000+0000 7f7a5b400640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25e3ef10 0x555e25e41300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.001+0000 7f7a45600640 10 client.25513 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.001+0000 7f7a5b400640 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25e3ef10 0x555e25e41300 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfe0aa88ae39e36 server_cookie=220d21f3102bcf78 in_seq=0 out_seq=0 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.002+0000 7f7a45600640 1 -- 172.21.15.57:0/599891984 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e25d24a40 con 0x555e25e3ef10 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.002+0000 7f7a45600640 10 client.25513 handle_command_reply: tid=0 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.002+0000 7f7a65bce740 1 client.25513 shutdown 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.002+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 msgr2=0x555e25dd0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.002+0000 7f7a65bce740 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 0x555e25dd0d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009fd0 tx=0x7f7a4c09c0a0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 shutdown_connections 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x555e25e3ef10 0x555e25e41300 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e25dc2c50 0x555e25dd1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a45600640 0 client.25513 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e25dc7dc0 0x555e25dd2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 --2- 172.21.15.57:0/599891984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e25dc2260 0x555e25dd0d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 >> 172.21.15.57:0/599891984 conn(0x555e25dac7c0 msgr2=0x555e25dcc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.003+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 shutdown_connections 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.004+0000 7f7a65bce740 1 -- 172.21.15.57:0/599891984 wait complete. 2024-04-13T05:20:47.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.004+0000 7f7a65bce740 20 client.25513 trim_cache size 0 max 16384 2024-04-13T05:20:47.009 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:20:47.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.004+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a400a3c30 msgr2=0x7f7a400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a400a3c30 0x7f7a400a60f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7a540049d0 tx=0x7f7a540096b0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 msgr2=0x7f7a5c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 0x7f7a5c0709d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b450 tx=0x7f7a4c00b910 comp rx=0 tx=0).stop 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 shutdown_connections 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7a400a3c30 0x7f7a400a60f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a5c10a1d0 0x7f7a5c1ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a5c109810 0x7f7a5c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 --2- 172.21.15.57:0/1461703225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c108620 0x7f7a5c0709d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.005+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 >> 172.21.15.57:0/1461703225 conn(0x7f7a5c0775c0 msgr2=0x7f7a5c10cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.006+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 shutdown_connections 2024-04-13T05:20:47.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.006+0000 7f7a62c00640 1 -- 172.21.15.57:0/1461703225 wait complete. 2024-04-13T05:20:47.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:46 smithi057 ceph-mon[32331]: pgmap v551: 129 pgs: 129 active+clean; 58 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-04-13T05:20:47.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:46 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1307636240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:20:47.028 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:20:47.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:46 smithi189 ceph-mon[35531]: pgmap v551: 129 pgs: 129 active+clean; 58 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-04-13T05:20:47.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:46 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1307636240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:20:47.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:46 smithi083 ceph-mon[35133]: pgmap v551: 129 pgs: 129 active+clean; 58 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-04-13T05:20:47.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:46 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1307636240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:20:47.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 -- 172.21.15.57:0/4217582373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10881052f0 msgr2=0x7f1088105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 --2- 172.21.15.57:0/4217582373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10881052f0 0x7f1088105770 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1070009900 tx=0x7f107002bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:47.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 -- 172.21.15.57:0/4217582373 shutdown_connections 2024-04-13T05:20:47.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 --2- 172.21.15.57:0/4217582373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 0x7f108810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 --2- 172.21.15.57:0/4217582373 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10881052f0 0x7f1088105770 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 --2- 172.21.15.57:0/4217582373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1088104100 0x7f1088104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 -- 172.21.15.57:0/4217582373 >> 172.21.15.57:0/4217582373 conn(0x7f10880ff670 msgr2=0x7f1088101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.290+0000 7f1087e00640 1 -- 172.21.15.57:0/4217582373 shutdown_connections 2024-04-13T05:20:47.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.291+0000 7f1087e00640 1 -- 172.21.15.57:0/4217582373 wait complete. 2024-04-13T05:20:47.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.292+0000 7f1087e00640 1 Processor -- start 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.292+0000 7f1087e00640 1 -- start start 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.293+0000 7f1087e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 0x7f108806e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.293+0000 7f1087e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10881052f0 0x7f108806e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1087e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 0x7f10880735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1087e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1088077450 con 0x7f1088104100 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1087e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10880772d0 con 0x7f1088105cb0 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1087e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f10880775d0 con 0x7f10881052f0 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1086a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 0x7f108806e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1086a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 0x7f108806e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45966/0 (socket says 172.21.15.57:45966) 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1086a00640 1 -- 172.21.15.57:0/3795664346 learned_addr learned my addr 172.21.15.57:0/3795664346 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.294+0000 7f1087400640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 0x7f10880735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1086000640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10881052f0 0x7f108806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1087400640 1 -- 172.21.15.57:0/3795664346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10881052f0 msgr2=0x7f108806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1087400640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10881052f0 0x7f108806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1087400640 1 -- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 msgr2=0x7f108806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1087400640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 0x7f108806e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1087400640 1 -- 172.21.15.57:0/3795664346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1088073c20 con 0x7f1088105cb0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1086000640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10881052f0 0x7f108806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.295+0000 7f1086a00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 0x7f108806e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.296+0000 7f1087400640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 0x7f10880735b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f107800da10 tx=0x7f107800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.296+0000 7f108ca00640 1 -- 172.21.15.57:0/3795664346 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107800bd70 con 0x7f1088105cb0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.296+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1088073f10 con 0x7f1088105cb0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.296+0000 7f108ca00640 1 -- 172.21.15.57:0/3795664346 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1078003e20 con 0x7f1088105cb0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.296+0000 7f108ca00640 1 -- 172.21.15.57:0/3795664346 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10780106c0 con 0x7f1088105cb0 2024-04-13T05:20:47.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.296+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10881af3c0 con 0x7f1088105cb0 2024-04-13T05:20:47.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f108ca00640 1 -- 172.21.15.57:0/3795664346 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1078003970 con 0x7f1088105cb0 2024-04-13T05:20:47.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 Processor -- start 2024-04-13T05:20:47.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 -- start start 2024-04-13T05:20:47.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fb440 0x556c3a7fb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 0x556c3a800930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a800fa0 0x556c3a803430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556c3a761020 con 0x556c3a800fa0 2024-04-13T05:20:47.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556c3a707950 con 0x556c3a7fb440 2024-04-13T05:20:47.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.301+0000 7f1090d7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556c3a75cfe0 con 0x556c3a7fbe30 2024-04-13T05:20:47.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.302+0000 7f1086a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 0x556c3a800930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.302+0000 7f1086a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 0x556c3a800930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56174/0 (socket says 172.21.15.57:56174) 2024-04-13T05:20:47.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.302+0000 7f1086a00640 1 -- 172.21.15.57:0/92289841 learned_addr learned my addr 172.21.15.57:0/92289841 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:47.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086000640 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fb440 0x556c3a7fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086000640 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a800fa0 0x556c3a803430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086a00640 1 -- 172.21.15.57:0/92289841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fb440 msgr2=0x556c3a7fb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086a00640 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fb440 0x556c3a7fb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086a00640 1 -- 172.21.15.57:0/92289841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a800fa0 msgr2=0x556c3a803430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086a00640 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a800fa0 0x556c3a803430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086a00640 1 -- 172.21.15.57:0/92289841 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3a4733f0 con 0x556c3a7fbe30 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.303+0000 7f1086000640 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a800fa0 0x556c3a803430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.304+0000 7f108ca00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f106c0a3c30 0x7f106c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.304+0000 7f108ca00640 1 -- 172.21.15.57:0/3795664346 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f10780d0a70 con 0x7f1088105cb0 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.304+0000 7f1086000640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f106c0a3c30 0x7f106c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.304+0000 7f1086a00640 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 0x556c3a800930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1080010500 tx=0x7f10800109c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe86f15420aeaa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.305+0000 7f1086000640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f106c0a3c30 0x7f106c0a60f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1070006010 tx=0x7f1070000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.305+0000 7f1077e00640 1 -- 172.21.15.57:0/92289841 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080011760 con 0x556c3a7fbe30 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.305+0000 7f1077e00640 1 -- 172.21.15.57:0/92289841 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1080011db0 con 0x556c3a7fbe30 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.305+0000 7f1077e00640 1 -- 172.21.15.57:0/92289841 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108000e430 con 0x556c3a7fbe30 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 -- 172.21.15.57:0/92289841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 msgr2=0x556c3a800930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 0x556c3a800930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1080010500 tx=0x7f10800109c0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 -- 172.21.15.57:0/92289841 shutdown_connections 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a800fa0 0x556c3a803430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fbe30 0x556c3a800930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 --2- 172.21.15.57:0/92289841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fb440 0x556c3a7fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.306+0000 7f1090d7e740 1 -- 172.21.15.57:0/92289841 >> 172.21.15.57:0/92289841 conn(0x556c3a7f69b0 msgr2=0x556c3a7f6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.307+0000 7f1090d7e740 1 -- 172.21.15.57:0/92289841 shutdown_connections 2024-04-13T05:20:47.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.307+0000 7f1090d7e740 1 -- 172.21.15.57:0/92289841 wait complete. 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.307+0000 7f1090d7e740 1 -- start start 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.307+0000 7f1090d7e740 1 Processor -- start 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a805980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a805ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a800fa0 0x556c3a80cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556c3a803f50 con 0x556c3a7fb440 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556c3a71ac80 con 0x556c3a7fbe30 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556c3a8040d0 con 0x556c3a800fa0 2024-04-13T05:20:47.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.308+0000 7f1090d7e740 10 client.0 _ll_register_callbacks cb 0x556c3a71a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:47.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1087400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a800fa0 0x556c3a80cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a805980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1087400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a800fa0 0x556c3a80cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56178/0 (socket says 172.21.15.57:56178) 2024-04-13T05:20:47.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1087400640 1 -- 172.21.15.57:0/523310606 learned_addr learned my addr 172.21.15.57:0/523310606 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086000640 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a805ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086a00640 1 -- 172.21.15.57:0/523310606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a800fa0 msgr2=0x556c3a80cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086a00640 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a800fa0 0x556c3a80cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086a00640 1 -- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 msgr2=0x556c3a805ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086a00640 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a805ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.309+0000 7f1086a00640 1 -- 172.21.15.57:0/523310606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3a53d7d0 con 0x556c3a7fb440 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.310+0000 7f1086a00640 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a805980 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x556c3a7fc230 tx=0x7f108001dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f06b4be66733d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.310+0000 7f1074c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.310+0000 7f1074c00640 1 -- 172.21.15.57:0/523310606 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080005250 con 0x556c3a7fb440 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.310+0000 7f1074c00640 1 -- 172.21.15.57:0/523310606 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f10800053f0 con 0x556c3a7fb440 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.310+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1080011db0 con 0x556c3a7fb440 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.311+0000 7f1074c00640 1 -- 172.21.15.57:0/523310606 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1080033050 con 0x556c3a7fb440 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.311+0000 7f1090d7e740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:47.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.311+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c3a8102e0 con 0x556c3a7fb440 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.311+0000 7f1074c00640 1 -- 172.21.15.57:0/523310606 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1080011db0 con 0x556c3a7fb440 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1074c00640 1 -- 172.21.15.57:0/523310606 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1080004750 con 0x556c3a7fb440 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1090d7e740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1090d7e740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1090d7e740 20 client.16128 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1090d7e740 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a8116e0 0x556c3a813af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1090d7e740 4 client.16128 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c3a700750 con 0x556c3a8116e0 2024-04-13T05:20:47.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.312+0000 7f1086000640 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a8116e0 0x556c3a813af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.314+0000 7f1086000640 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a8116e0 0x556c3a813af0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c43c5bf99c92a9e server_cookie=269624a77c28f9cc in_seq=0 out_seq=0 2024-04-13T05:20:47.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.314+0000 7f1074c00640 10 client.16128 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:47.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.315+0000 7f1074c00640 1 -- 172.21.15.57:0/523310606 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c3a700750 con 0x556c3a8116e0 2024-04-13T05:20:47.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.315+0000 7f1074c00640 10 client.16128 handle_command_reply: tid=0 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.315+0000 7f1090d7e740 1 client.16128 shutdown 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 msgr2=0x556c3a805980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a805980 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x556c3a7fc230 tx=0x7f108001dcf0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 shutdown_connections 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a8116e0 0x556c3a813af0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a805ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a800fa0 0x556c3a80cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 --2- 172.21.15.57:0/523310606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a805980 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1074c00640 0 client.16128 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:47.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.316+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 >> 172.21.15.57:0/523310606 conn(0x556c3a7f69b0 msgr2=0x556c3a7f5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.317+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 shutdown_connections 2024-04-13T05:20:47.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.317+0000 7f1090d7e740 1 -- 172.21.15.57:0/523310606 wait complete. 2024-04-13T05:20:47.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.317+0000 7f1090d7e740 20 client.16128 trim_cache size 0 max 16384 2024-04-13T05:20:47.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 Processor -- start 2024-04-13T05:20:47.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 -- start start 2024-04-13T05:20:47.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7f69b0 0x556c3a813ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fb440 0x556c3a80cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 0x556c3a804fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f10640124f0 con 0x556c3a7fbe30 2024-04-13T05:20:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f106400fd60 con 0x556c3a7f69b0 2024-04-13T05:20:47.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1090d7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f106400dbb0 con 0x556c3a7fb440 2024-04-13T05:20:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1087400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 0x556c3a804fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1086a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7f69b0 0x556c3a813ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.344+0000 7f1087400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 0x556c3a804fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45998/0 (socket says 172.21.15.57:45998) 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 -- 172.21.15.57:0/3266486176 learned_addr learned my addr 172.21.15.57:0/3266486176 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1086000640 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fb440 0x556c3a80cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 -- 172.21.15.57:0/3266486176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fb440 msgr2=0x556c3a80cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fb440 0x556c3a80cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 -- 172.21.15.57:0/3266486176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7f69b0 msgr2=0x556c3a813ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7f69b0 0x556c3a813ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 -- 172.21.15.57:0/3266486176 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3a8005d0 con 0x556c3a7fbe30 2024-04-13T05:20:47.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.345+0000 7f1087400640 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 0x556c3a804fa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1078005fd0 tx=0x7f10780bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcb617383a94c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.346+0000 7f1077400640 1 -- 172.21.15.57:0/3266486176 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078003c60 con 0x556c3a7fbe30 2024-04-13T05:20:47.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.346+0000 7f1077400640 1 -- 172.21.15.57:0/3266486176 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10780b8070 con 0x556c3a7fbe30 2024-04-13T05:20:47.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.346+0000 7f1077400640 1 -- 172.21.15.57:0/3266486176 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078099020 con 0x556c3a7fbe30 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 -- 172.21.15.57:0/3266486176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 msgr2=0x556c3a804fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 0x556c3a804fa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1078005fd0 tx=0x7f10780bd0d0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 -- 172.21.15.57:0/3266486176 shutdown_connections 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fbe30 0x556c3a804fa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7fb440 0x556c3a80cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 --2- 172.21.15.57:0/3266486176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7f69b0 0x556c3a813ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 -- 172.21.15.57:0/3266486176 >> 172.21.15.57:0/3266486176 conn(0x556c3a7fcc10 msgr2=0x556c3a7f5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 -- 172.21.15.57:0/3266486176 shutdown_connections 2024-04-13T05:20:47.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.347+0000 7f1090d7e740 1 -- 172.21.15.57:0/3266486176 wait complete. 2024-04-13T05:20:47.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 -- start start 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 Processor -- start 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7f69b0 0x556c3a80a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a80ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a80bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f106400e220 con 0x556c3a7fb440 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f106400f690 con 0x556c3a7fbe30 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f106400e8f0 con 0x556c3a7f69b0 2024-04-13T05:20:47.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.348+0000 7f1090d7e740 10 client.0 _ll_register_callbacks cb 0x7f1080005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:47.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1086000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a80ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a80bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a80bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59642/0 (socket says 172.21.15.57:59642) 2024-04-13T05:20:47.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 -- 172.21.15.57:0/2518707897 learned_addr learned my addr 172.21.15.57:0/2518707897 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1086a00640 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7f69b0 0x556c3a80a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 -- 172.21.15.57:0/2518707897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7f69b0 msgr2=0x556c3a80a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7f69b0 0x556c3a80a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 -- 172.21.15.57:0/2518707897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 msgr2=0x556c3a80ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a80ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.349+0000 7f1087400640 1 -- 172.21.15.57:0/2518707897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3a80c300 con 0x556c3a7fbe30 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1087400640 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a80bd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f107801c000 tx=0x7f1078003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa97ad46ed8627b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1074c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1074c00640 1 -- 172.21.15.57:0/2518707897 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078080060 con 0x556c3a7fbe30 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1074c00640 1 -- 172.21.15.57:0/2518707897 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f10780a5190 con 0x556c3a7fbe30 2024-04-13T05:20:47.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10780b8070 con 0x556c3a7fbe30 2024-04-13T05:20:47.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1074c00640 1 -- 172.21.15.57:0/2518707897 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1078005e30 con 0x556c3a7fbe30 2024-04-13T05:20:47.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1090d7e740 10 client.35585 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.350+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c3a877e30 con 0x556c3a7fbe30 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1074c00640 1 -- 172.21.15.57:0/2518707897 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10780b8070 con 0x556c3a7fbe30 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1074c00640 1 -- 172.21.15.57:0/2518707897 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f107809dce0 con 0x556c3a7fbe30 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1090d7e740 10 client.35585 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1090d7e740 10 client.35585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1090d7e740 20 client.35585 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1090d7e740 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a800fa0 0x556c3a87a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1090d7e740 4 client.35585 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.351+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c3a76ea40 con 0x556c3a800fa0 2024-04-13T05:20:47.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.352+0000 7f1086a00640 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a800fa0 0x556c3a87a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.353+0000 7f1074c00640 10 client.35585 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.353+0000 7f1086a00640 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a800fa0 0x556c3a87a7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232b187db7fdf426 server_cookie=1781c315b2570020 in_seq=0 out_seq=0 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.353+0000 7f1074c00640 1 -- 172.21.15.57:0/2518707897 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c3a76ea40 con 0x556c3a800fa0 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.353+0000 7f1074c00640 10 client.35585 handle_command_reply: tid=0 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.353+0000 7f1090d7e740 1 client.35585 shutdown 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 msgr2=0x556c3a80bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a80bd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f107801c000 tx=0x7f1078003d50 comp rx=0 tx=0).stop 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 shutdown_connections 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556c3a800fa0 0x556c3a87a7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3a7fbe30 0x556c3a80bd10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c3a7fb440 0x556c3a80ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 --2- 172.21.15.57:0/2518707897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c3a7f69b0 0x556c3a80a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1074c00640 0 client.35585 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 >> 172.21.15.57:0/2518707897 conn(0x556c3a7fcc10 msgr2=0x556c3a7f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 shutdown_connections 2024-04-13T05:20:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 1 -- 172.21.15.57:0/2518707897 wait complete. 2024-04-13T05:20:47.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.354+0000 7f1090d7e740 20 client.35585 trim_cache size 0 max 16384 2024-04-13T05:20:47.360 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.355+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f106c0a3c30 msgr2=0x7f106c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.355+0000 7f1087e00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f106c0a3c30 0x7f106c0a60f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1070006010 tx=0x7f1070000bc0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.355+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 msgr2=0x7f10880735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.355+0000 7f1087e00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 0x7f10880735b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f107800da10 tx=0x7f107800ded0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.355+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 shutdown_connections 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.355+0000 7f1087e00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f106c0a3c30 0x7f106c0a60f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.356+0000 7f1087e00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1088105cb0 0x7f10880735b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.356+0000 7f1087e00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10881052f0 0x7f108806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.356+0000 7f1087e00640 1 --2- 172.21.15.57:0/3795664346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1088104100 0x7f108806e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.356+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 >> 172.21.15.57:0/3795664346 conn(0x7f10880ff670 msgr2=0x7f1088079f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.356+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 shutdown_connections 2024-04-13T05:20:47.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:47.356+0000 7f1087e00640 1 -- 172.21.15.57:0/3795664346 wait complete. 2024-04-13T05:20:48.384 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:20:48.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:20:48.384 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:20:48.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 -- 172.21.15.57:0/3340441425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 msgr2=0x7f5eb410a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/3340441425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb410a1a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009ca0 tx=0x7f5eac02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 -- 172.21.15.57:0/3340441425 shutdown_connections 2024-04-13T05:20:48.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/3340441425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb410a1a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/3340441425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eb4105480 0x7f5eb4107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/3340441425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eb4102b50 0x7f5eb4104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 -- 172.21.15.57:0/3340441425 >> 172.21.15.57:0/3340441425 conn(0x7f5eb40fc480 msgr2=0x7f5eb40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:48.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 -- 172.21.15.57:0/3340441425 shutdown_connections 2024-04-13T05:20:48.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.597+0000 7f5eb9e00640 1 -- 172.21.15.57:0/3340441425 wait complete. 2024-04-13T05:20:48.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 Processor -- start 2024-04-13T05:20:48.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 -- start start 2024-04-13T05:20:48.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eb4102b50 0x7f5eb419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eb4105480 0x7f5eb419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410db60 con 0x7f5eb4107db0 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410d9e0 con 0x7f5eb4102b50 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.598+0000 7f5eb9e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410dce0 con 0x7f5eb4105480 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb41a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46012/0 (socket says 172.21.15.57:46012) 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 -- 172.21.15.57:0/1978957006 learned_addr learned my addr 172.21.15.57:0/1978957006 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eb4102b50 0x7f5eb419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eb4105480 0x7f5eb419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 -- 172.21.15.57:0/1978957006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eb4105480 msgr2=0x7f5eb419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eb4105480 0x7f5eb419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 -- 172.21.15.57:0/1978957006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eb4102b50 msgr2=0x7f5eb419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eb4102b50 0x7f5eb419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.599+0000 7f5eb3400640 1 -- 172.21.15.57:0/1978957006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb41a20b0 con 0x7f5eb4107db0 2024-04-13T05:20:48.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.600+0000 7f5eb3400640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb41a1a40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5eac037040 tx=0x7f5eac032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:48.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.600+0000 7f5eb8a00640 1 -- 172.21.15.57:0/1978957006 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac041020 con 0x7f5eb4107db0 2024-04-13T05:20:48.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.600+0000 7f5eb8a00640 1 -- 172.21.15.57:0/1978957006 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eac003e20 con 0x7f5eb4107db0 2024-04-13T05:20:48.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.600+0000 7f5eb8a00640 1 -- 172.21.15.57:0/1978957006 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac039040 con 0x7f5eb4107db0 2024-04-13T05:20:48.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.600+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb41a2340 con 0x7f5eb4107db0 2024-04-13T05:20:48.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.600+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb41a2880 con 0x7f5eb4107db0 2024-04-13T05:20:48.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.603+0000 7f5ebcde4740 1 Processor -- start 2024-04-13T05:20:48.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.604+0000 7f5eb8a00640 1 -- 172.21.15.57:0/1978957006 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5eac044020 con 0x7f5eb4107db0 2024-04-13T05:20:48.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.604+0000 7f5ebcde4740 1 -- start start 2024-04-13T05:20:48.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 0x55bc55ef26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef2c70 0x55bc55ef7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55efa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bc55e58bc0 con 0x55bc55ef2280 2024-04-13T05:20:48.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc55dff660 con 0x55bc55ef7de0 2024-04-13T05:20:48.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc55e54b80 con 0x55bc55ef2c70 2024-04-13T05:20:48.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 0x55bc55ef26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 0x55bc55ef26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46020/0 (socket says 172.21.15.57:46020) 2024-04-13T05:20:48.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.605+0000 7f5eb2000640 1 -- 172.21.15.57:0/1479170709 learned_addr learned my addr 172.21.15.57:0/1479170709 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef2c70 0x55bc55ef7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55efa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 -- 172.21.15.57:0/1479170709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef2c70 msgr2=0x55bc55ef7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef2c70 0x55bc55ef7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 -- 172.21.15.57:0/1479170709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 msgr2=0x55bc55efa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55efa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 -- 172.21.15.57:0/1479170709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc55e22220 con 0x55bc55ef2280 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb8a00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e980a3c30 0x7f5e980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 0x55bc55ef26a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00134f0 tx=0x7f5ea000b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b8daad275771b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb8a00640 1 -- 172.21.15.57:0/1978957006 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5eac0f2a90 con 0x7f5eb4107db0 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.606+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e980a3c30 0x7f5e980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.607+0000 7f5e9fe00640 1 -- 172.21.15.57:0/1479170709 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0019030 con 0x55bc55ef2280 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.607+0000 7f5e9fe00640 1 -- 172.21.15.57:0/1479170709 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ea0005bf0 con 0x55bc55ef2280 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.607+0000 7f5e9fe00640 1 -- 172.21.15.57:0/1479170709 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea001d690 con 0x55bc55ef2280 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e980a3c30 0x7f5e980a60f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55bc55ef3070 tx=0x7f5ea400d300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1479170709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 msgr2=0x55bc55ef26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 0x55bc55ef26a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00134f0 tx=0x7f5ea000b600 comp rx=0 tx=0).stop 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1479170709 shutdown_connections 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55efa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef2c70 0x55bc55ef7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1479170709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2280 0x55bc55ef26a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1479170709 >> 172.21.15.57:0/1479170709 conn(0x55bc55eed7f0 msgr2=0x55bc55eedbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:48.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1479170709 shutdown_connections 2024-04-13T05:20:48.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1479170709 wait complete. 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.608+0000 7f5ebcde4740 1 -- start start 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 Processor -- start 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 0x55bc55efcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bc55efad90 con 0x55bc55ef7de0 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc55e12a30 con 0x55bc55ef2c70 2024-04-13T05:20:48.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc55efaf10 con 0x55bc55f01f00 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5ebcde4740 10 client.0 _ll_register_callbacks cb 0x55bc55b6ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 0x55bc55efcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 0x55bc55efcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46032/0 (socket says 172.21.15.57:46032) 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.609+0000 7f5eb2000640 1 -- 172.21.15.57:0/1860071919 learned_addr learned my addr 172.21.15.57:0/1860071919 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2000640 1 -- 172.21.15.57:0/1860071919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 msgr2=0x55bc55f042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2000640 1 -- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 msgr2=0x55bc55efca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2000640 1 -- 172.21.15.57:0/1860071919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc55c337e0 con 0x55bc55ef7de0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 0x55bc55efcf40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005f70 tx=0x7f5ea00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dc0583566df096 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.610+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1860071919 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0019040 con 0x55bc55ef7de0 2024-04-13T05:20:48.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.611+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1860071919 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5ea0004030 con 0x55bc55ef7de0 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.611+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc55e665e0 con 0x55bc55ef7de0 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.611+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1860071919 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ea002c050 con 0x55bc55ef7de0 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.611+0000 7f5ebcde4740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.611+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc55c93f70 con 0x55bc55ef7de0 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.611+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1860071919 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bc55e665e0 con 0x55bc55ef7de0 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1860071919 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f5ea00046c0 con 0x55bc55ef7de0 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5ebcde4740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5ebcde4740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5ebcde4740 20 client.16146 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:48.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f08ac0 0x55bc55f0af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5ebcde4740 4 client.16146 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:48.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.612+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc55e60450 con 0x55bc55f08ac0 2024-04-13T05:20:48.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.679+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f08ac0 0x55bc55f0af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.681+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f08ac0 0x55bc55f0af00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb942027654945b3 server_cookie=12ea09a5ba97110a in_seq=0 out_seq=0 2024-04-13T05:20:48.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.681+0000 7f5e9cc00640 10 client.16146 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:48.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.684+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1860071919 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc55e60450 con 0x55bc55f08ac0 2024-04-13T05:20:48.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.684+0000 7f5e9cc00640 10 client.16146 handle_command_reply: tid=0 2024-04-13T05:20:48.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.684+0000 7f5ebcde4740 1 client.16146 shutdown 2024-04-13T05:20:48.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 msgr2=0x55bc55efcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 0x55bc55efcf40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005f70 tx=0x7f5ea00052f0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 shutdown_connections 2024-04-13T05:20:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f08ac0 0x55bc55f0af00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef7de0 0x55bc55efcf40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1860071919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 >> 172.21.15.57:0/1860071919 conn(0x55bc55eed7f0 msgr2=0x55bc55eec9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:48.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.685+0000 7f5e9cc00640 0 client.16146 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:48.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.686+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 shutdown_connections 2024-04-13T05:20:48.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.686+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1860071919 wait complete. 2024-04-13T05:20:48.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.686+0000 7f5ebcde4740 20 client.16146 trim_cache size 0 max 16384 2024-04-13T05:20:48.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.736+0000 7f5ebcde4740 1 Processor -- start 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.736+0000 7f5ebcde4740 1 -- start start 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.737+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.737+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef7de0 0x55bc55efb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.737+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 0x55bc55eec160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.737+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41af690 con 0x55bc55f01f00 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.737+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41acf00 con 0x55bc55ef2c70 2024-04-13T05:20:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.737+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41aad50 con 0x55bc55ef7de0 2024-04-13T05:20:48.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 0x55bc55eec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 0x55bc55eec160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46044/0 (socket says 172.21.15.57:46044) 2024-04-13T05:20:48.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 -- 172.21.15.57:0/752199548 learned_addr learned my addr 172.21.15.57:0/752199548 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb2000640 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef7de0 0x55bc55efb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 -- 172.21.15.57:0/752199548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef7de0 msgr2=0x55bc55efb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef7de0 0x55bc55efb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 -- 172.21.15.57:0/752199548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 msgr2=0x55bc55efb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.738+0000 7f5eb3400640 1 -- 172.21.15.57:0/752199548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc55b2b170 con 0x55bc55f01f00 2024-04-13T05:20:48.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.739+0000 7f5eb3400640 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 0x55bc55eec160 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0be670 tx=0x7f5eac0ba4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb9d598c4056b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.739+0000 7f5e9f400640 1 -- 172.21.15.57:0/752199548 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0be4b0 con 0x55bc55f01f00 2024-04-13T05:20:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.740+0000 7f5e9f400640 1 -- 172.21.15.57:0/752199548 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eac03b2f0 con 0x55bc55f01f00 2024-04-13T05:20:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.740+0000 7f5e9f400640 1 -- 172.21.15.57:0/752199548 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0be2d0 con 0x55bc55f01f00 2024-04-13T05:20:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 -- 172.21.15.57:0/752199548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 msgr2=0x55bc55eec160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 0x55bc55eec160 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0be670 tx=0x7f5eac0ba4f0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 -- 172.21.15.57:0/752199548 shutdown_connections 2024-04-13T05:20:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55f01f00 0x55bc55eec160 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55ef7de0 0x55bc55efb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/752199548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef2c70 0x55bc55efb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 -- 172.21.15.57:0/752199548 >> 172.21.15.57:0/752199548 conn(0x55bc55eed7f0 msgr2=0x55bc55ef43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 -- 172.21.15.57:0/752199548 shutdown_connections 2024-04-13T05:20:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.741+0000 7f5ebcde4740 1 -- 172.21.15.57:0/752199548 wait complete. 2024-04-13T05:20:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.742+0000 7f5ebcde4740 1 -- start start 2024-04-13T05:20:48.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.742+0000 7f5ebcde4740 1 Processor -- start 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 0x55bc55f6b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55f6bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f1b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41ab3c0 con 0x55bc55ef2c70 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41ac830 con 0x55bc55ef7de0 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41aba90 con 0x55bc55f01f00 2024-04-13T05:20:48.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5ebcde4740 10 client.0 _ll_register_callbacks cb 0x7f5ea000bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 0x55bc55f6b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.746 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 0x55bc55f6b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46052/0 (socket says 172.21.15.57:46052) 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.743+0000 7f5eb2a00640 1 -- 172.21.15.57:0/1046723329 learned_addr learned my addr 172.21.15.57:0/1046723329 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.744+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55f6bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.744+0000 7f5eb3400640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f1b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.744+0000 7f5eb2a00640 1 -- 172.21.15.57:0/1046723329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 msgr2=0x55bc55f1b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.744+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f1b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.744+0000 7f5eb2a00640 1 -- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 msgr2=0x55bc55f6bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.744+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55f6bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.745+0000 7f5eb2a00640 1 -- 172.21.15.57:0/1046723329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc55e13be0 con 0x55bc55ef2c70 2024-04-13T05:20:48.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.745+0000 7f5eb3400640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.745+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55f6bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.745+0000 7f5eb2a00640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 0x55bc55f6b990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55bc55f0b740 tx=0x7f5ea4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc279c7928aa92c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.745+0000 7f5e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.746+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1046723329 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4004230 con 0x55bc55ef2c70 2024-04-13T05:20:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.746+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1046723329 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5ea40043d0 con 0x55bc55ef2c70 2024-04-13T05:20:48.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.746+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eac03b2f0 con 0x55bc55ef2c70 2024-04-13T05:20:48.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.747+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1046723329 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ea4038050 con 0x55bc55ef2c70 2024-04-13T05:20:48.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.747+0000 7f5ebcde4740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:48.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.747+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc55f77460 con 0x55bc55ef2c70 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.747+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1046723329 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eac03b2f0 con 0x55bc55ef2c70 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.747+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1046723329 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f5ea4003ae0 con 0x55bc55ef2c70 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5ebcde4740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5ebcde4740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5ebcde4740 20 client.16152 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f77e70 0x55bc55f7a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5ebcde4740 4 client.16152 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:20:48.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc55efc7f0 con 0x55bc55f77e70 2024-04-13T05:20:48.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.748+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f77e70 0x55bc55f7a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:48.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.749+0000 7f5e9cc00640 10 client.16152 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:48.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.749+0000 7f5eb2000640 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f77e70 0x55bc55f7a290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4a1d588e721bfc server_cookie=6b2e3780a1c482d4 in_seq=0 out_seq=0 2024-04-13T05:20:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.751+0000 7f5e9cc00640 1 -- 172.21.15.57:0/1046723329 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc55efc7f0 con 0x55bc55f77e70 2024-04-13T05:20:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.751+0000 7f5e9cc00640 10 client.16152 handle_command_reply: tid=0 2024-04-13T05:20:48.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.751+0000 7f5ebcde4740 1 client.16152 shutdown 2024-04-13T05:20:48.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.751+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 msgr2=0x55bc55f6b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.751+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 0x55bc55f6b990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55bc55f0b740 tx=0x7f5ea4004730 comp rx=0 tx=0).stop 2024-04-13T05:20:48.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 shutdown_connections 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bc55f77e70 0x55bc55f7a290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc55ef7de0 0x55bc55f6bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc55f01f00 0x55bc55f1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 --2- 172.21.15.57:0/1046723329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bc55ef2c70 0x55bc55f6b990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5e9cc00640 0 client.16152 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 >> 172.21.15.57:0/1046723329 conn(0x55bc55eed7f0 msgr2=0x55bc55efb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 shutdown_connections 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.752+0000 7f5ebcde4740 1 -- 172.21.15.57:0/1046723329 wait complete. 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.753+0000 7f5ebcde4740 20 client.16152 trim_cache size 0 max 16384 2024-04-13T05:20:48.755 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "62669c43-84a4-4aaf-b057-257b92963edf", 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.753+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e980a3c30 msgr2=0x7f5e980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.753+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e980a3c30 0x7f5e980a60f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55bc55ef3070 tx=0x7f5ea400d300 comp rx=0 tx=0).stop 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.753+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 msgr2=0x7f5eb41a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.753+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb41a1a40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5eac037040 tx=0x7f5eac032820 comp rx=0 tx=0).stop 2024-04-13T05:20:48.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 shutdown_connections 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5e980a3c30 0x7f5e980a60f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4107db0 0x7f5eb41a1a40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eb4105480 0x7f5eb419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 --2- 172.21.15.57:0/1978957006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eb4102b50 0x7f5eb419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 >> 172.21.15.57:0/1978957006 conn(0x7f5eb40fc480 msgr2=0x7f5eb4102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.754+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 shutdown_connections 2024-04-13T05:20:48.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:48.755+0000 7f5eb9e00640 1 -- 172.21.15.57:0/1978957006 wait complete. 2024-04-13T05:20:48.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:20:48.781 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:20:49.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:48 smithi057 ceph-mon[32331]: pgmap v552: 129 pgs: 129 active+clean; 58 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:20:49.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 -- 172.21.15.57:0/2565105565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 msgr2=0x7f74e0105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 --2- 172.21.15.57:0/2565105565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 0x7f74e0105870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009900 tx=0x7f74cc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 -- 172.21.15.57:0/2565105565 shutdown_connections 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 --2- 172.21.15.57:0/2565105565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 0x7f74e0105870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 --2- 172.21.15.57:0/2565105565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74e0105db0 0x7f74e010c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 --2- 172.21.15.57:0/2565105565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 -- 172.21.15.57:0/2565105565 >> 172.21.15.57:0/2565105565 conn(0x7f74e00ff770 msgr2=0x7f74e0101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 -- 172.21.15.57:0/2565105565 shutdown_connections 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.058+0000 7f74e6600640 1 -- 172.21.15.57:0/2565105565 wait complete. 2024-04-13T05:20:49.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.059+0000 7f74e6600640 1 Processor -- start 2024-04-13T05:20:49.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 -- start start 2024-04-13T05:20:49.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 0x7f74e019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74e0105db0 0x7f74e01a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f74e010fd90 con 0x7f74e0104200 2024-04-13T05:20:49.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74e010fc10 con 0x7f74e01053f0 2024-04-13T05:20:49.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.060+0000 7f74e6600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74e010ff10 con 0x7f74e0105db0 2024-04-13T05:20:49.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.061+0000 7f74df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.061+0000 7f74df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46064/0 (socket says 172.21.15.57:46064) 2024-04-13T05:20:49.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.061+0000 7f74df400640 1 -- 172.21.15.57:0/1372847177 learned_addr learned my addr 172.21.15.57:0/1372847177 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:49.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.061+0000 7f74dea00640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 0x7f74e019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.061+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74e0105db0 0x7f74e01a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74df400640 1 -- 172.21.15.57:0/1372847177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74e0105db0 msgr2=0x7f74e01a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74df400640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74e0105db0 0x7f74e01a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74df400640 1 -- 172.21.15.57:0/1372847177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 msgr2=0x7f74e019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74df400640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 0x7f74e019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74df400640 1 -- 172.21.15.57:0/1372847177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e01a6470 con 0x7f74e0104200 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74df400640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e019e8a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74d000e9c0 tx=0x7f74d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74d7e00640 1 -- 172.21.15.57:0/1372847177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d000cde0 con 0x7f74e0104200 2024-04-13T05:20:49.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.062+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e01a6760 con 0x7f74e0104200 2024-04-13T05:20:49.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.063+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e01a6cd0 con 0x7f74e0104200 2024-04-13T05:20:49.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.063+0000 7f74d7e00640 1 -- 172.21.15.57:0/1372847177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d00108e0 con 0x7f74e0104200 2024-04-13T05:20:49.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.063+0000 7f74d7e00640 1 -- 172.21.15.57:0/1372847177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00187a0 con 0x7f74e0104200 2024-04-13T05:20:49.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 Processor -- start 2024-04-13T05:20:49.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 -- start start 2024-04-13T05:20:49.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151cd29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cd54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561151c34020 con 0x561151ccdec0 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561151bda6d0 con 0x561151cd3030 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.066+0000 7f74e95df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561151c2ffe0 con 0x561151ccd4d0 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.067+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.067+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56246/0 (socket says 172.21.15.57:56246) 2024-04-13T05:20:49.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.067+0000 7f74dfe00640 1 -- 172.21.15.57:0/3315540612 learned_addr learned my addr 172.21.15.57:0/3315540612 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:49.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.068+0000 7f74d7e00640 1 -- 172.21.15.57:0/1372847177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f74d001f020 con 0x7f74e0104200 2024-04-13T05:20:49.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.068+0000 7f74dea00640 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cd54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.068+0000 7f74df400640 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151cd29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74dfe00640 1 -- 172.21.15.57:0/3315540612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 msgr2=0x561151cd54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cd54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74dfe00640 1 -- 172.21.15.57:0/3315540612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 msgr2=0x561151cd29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151cd29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74dfe00640 1 -- 172.21.15.57:0/3315540612 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561151a107d0 con 0x561151ccd4d0 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74df400640 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151cd29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.069+0000 7f74d7e00640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74c40a3c30 0x7f74c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.070+0000 7f74d7e00640 1 -- 172.21.15.57:0/1372847177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f74d00d0920 con 0x7f74e0104200 2024-04-13T05:20:49.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.070+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccd8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009ee0 tx=0x7f74d8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f2920616db193a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:49.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.070+0000 7f74df400640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74c40a3c30 0x7f74c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.071+0000 7f74d4c00640 1 -- 172.21.15.57:0/3315540612 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8016070 con 0x561151ccd4d0 2024-04-13T05:20:49.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.071+0000 7f74d4c00640 1 -- 172.21.15.57:0/3315540612 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d8004d20 con 0x561151ccd4d0 2024-04-13T05:20:49.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.071+0000 7f74d4c00640 1 -- 172.21.15.57:0/3315540612 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8012740 con 0x561151ccd4d0 2024-04-13T05:20:49.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.072+0000 7f74e95df740 1 -- 172.21.15.57:0/3315540612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 msgr2=0x561151ccd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.072+0000 7f74e95df740 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccd8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009ee0 tx=0x7f74d8010740 comp rx=0 tx=0).stop 2024-04-13T05:20:49.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.072+0000 7f74df400640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74c40a3c30 0x7f74c40a60f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74d00ce3f0 tx=0x7f74d000efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:49.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 -- 172.21.15.57:0/3315540612 shutdown_connections 2024-04-13T05:20:49.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cd54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151cd29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 --2- 172.21.15.57:0/3315540612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccd8f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 -- 172.21.15.57:0/3315540612 >> 172.21.15.57:0/3315540612 conn(0x561151cc8a40 msgr2=0x561151cc8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:49.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 -- 172.21.15.57:0/3315540612 shutdown_connections 2024-04-13T05:20:49.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 -- 172.21.15.57:0/3315540612 wait complete. 2024-04-13T05:20:49.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.073+0000 7f74e95df740 1 -- start start 2024-04-13T05:20:49.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.074+0000 7f74e95df740 1 Processor -- start 2024-04-13T05:20:49.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.074+0000 7f74e95df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151cd7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74e95df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccdec0 0x561151cd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74e95df740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151cdecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74e95df740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561151cd5fe0 con 0x561151cd3030 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74e95df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561151beda90 con 0x561151ccdec0 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74e95df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561151cd6160 con 0x561151ccd4d0 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74e95df740 10 client.0 _ll_register_callbacks cb 0x561151bed480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151cd7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151cd7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56256/0 (socket says 172.21.15.57:56256) 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74dfe00640 1 -- 172.21.15.57:0/3870846018 learned_addr learned my addr 172.21.15.57:0/3870846018 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151cdecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.075+0000 7f74dea00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccdec0 0x561151cd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.076+0000 7f74dfe00640 1 -- 172.21.15.57:0/3870846018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 msgr2=0x561151cd7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.076+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151cd7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.076+0000 7f74dfe00640 1 -- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccdec0 msgr2=0x561151cd7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.076+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccdec0 0x561151cd7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.076+0000 7f74dfe00640 1 -- 172.21.15.57:0/3870846018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611519463f0 con 0x561151cd3030 2024-04-13T05:20:49.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.077+0000 7f74dfe00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151cdecf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005cc0 tx=0x7f74d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8547fd12600449 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.077+0000 7f74c9600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:20:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.077+0000 7f74c9600640 1 -- 172.21.15.57:0/3870846018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8016070 con 0x561151cd3030 2024-04-13T05:20:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.078+0000 7f74c9600640 1 -- 172.21.15.57:0/3870846018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f74d80054e0 con 0x561151cd3030 2024-04-13T05:20:49.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.078+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d8004d20 con 0x561151cd3030 2024-04-13T05:20:49.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.079+0000 7f74c9600640 1 -- 172.21.15.57:0/3870846018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74d80124c0 con 0x561151cd3030 2024-04-13T05:20:49.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.079+0000 7f74e95df740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:49.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.079+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611519060f0 con 0x561151cd3030 2024-04-13T05:20:49.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.080+0000 7f74c9600640 1 -- 172.21.15.57:0/3870846018 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d8004d20 con 0x561151cd3030 2024-04-13T05:20:49.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.080+0000 7f74c9600640 1 -- 172.21.15.57:0/3870846018 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f74d800f040 con 0x561151cd3030 2024-04-13T05:20:49.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.080+0000 7f74e95df740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:49.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.080+0000 7f74e95df740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.080+0000 7f74e95df740 20 client.16164 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.080+0000 7f74e95df740 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151ce32d0 0x561151ce56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.081+0000 7f74e95df740 4 client.16164 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:20:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.081+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561151bd3600 con 0x561151ce32d0 2024-04-13T05:20:49.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.081+0000 7f74dea00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151ce32d0 0x561151ce56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.082+0000 7f74dea00640 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151ce32d0 0x561151ce56c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ee20225d47aa0a server_cookie=71a887ca183726d in_seq=0 out_seq=0 2024-04-13T05:20:49.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.082+0000 7f74c9600640 10 client.16164 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.083+0000 7f74c9600640 1 -- 172.21.15.57:0/3870846018 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561151bd3600 con 0x561151ce32d0 2024-04-13T05:20:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.083+0000 7f74c9600640 10 client.16164 handle_command_reply: tid=0 2024-04-13T05:20:49.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.083+0000 7f74e95df740 1 client.16164 shutdown 2024-04-13T05:20:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 msgr2=0x561151cdecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151cdecf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005cc0 tx=0x7f74d8005fd0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 shutdown_connections 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151ce32d0 0x561151ce56c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccdec0 0x561151cd7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151cdecf0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 --2- 172.21.15.57:0/3870846018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151cd7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 >> 172.21.15.57:0/3870846018 conn(0x561151cc8a40 msgr2=0x561151cc7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74c9600640 0 client.16164 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 shutdown_connections 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.084+0000 7f74e95df740 1 -- 172.21.15.57:0/3870846018 wait complete. 2024-04-13T05:20:49.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.085+0000 7f74e95df740 20 client.16164 trim_cache size 0 max 16384 2024-04-13T05:20:49.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.116+0000 7f74e95df740 1 Processor -- start 2024-04-13T05:20:49.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.116+0000 7f74e95df740 1 -- start start 2024-04-13T05:20:49.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.117+0000 7f74e95df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.117+0000 7f74e95df740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151ccf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.117+0000 7f74e95df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cdec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74e95df740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00124f0 con 0x561151ccdec0 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74e95df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74c000fd60 con 0x561151cd3030 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74e95df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74c000dbb0 con 0x561151ccd4d0 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:56264/0 (socket says 172.21.15.57:56264) 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74dfe00640 1 -- 172.21.15.57:0/1236585730 learned_addr learned my addr 172.21.15.57:0/1236585730 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:49.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74dea00640 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151ccf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.118+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cdec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dfe00640 1 -- 172.21.15.57:0/1236585730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 msgr2=0x561151cdec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cdec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dfe00640 1 -- 172.21.15.57:0/1236585730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 msgr2=0x561151ccf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151ccf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dfe00640 1 -- 172.21.15.57:0/1236585730 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561151ce5a30 con 0x561151ccd4d0 2024-04-13T05:20:49.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dea00640 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151ccf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:20:49.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.119+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccf1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74d8012490 tx=0x7f74d8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca54eed9f2f697e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:49.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.120+0000 7f74cbe00640 1 -- 172.21.15.57:0/1236585730 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8016070 con 0x561151ccd4d0 2024-04-13T05:20:49.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.120+0000 7f74cbe00640 1 -- 172.21.15.57:0/1236585730 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d8005ab0 con 0x561151ccd4d0 2024-04-13T05:20:49.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.120+0000 7f74cbe00640 1 -- 172.21.15.57:0/1236585730 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8012660 con 0x561151ccd4d0 2024-04-13T05:20:49.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 -- 172.21.15.57:0/1236585730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 msgr2=0x561151ccf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccf1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74d8012490 tx=0x7f74d8010740 comp rx=0 tx=0).stop 2024-04-13T05:20:49.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 -- 172.21.15.57:0/1236585730 shutdown_connections 2024-04-13T05:20:49.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151cd3030 0x561151cdec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151ccdec0 0x561151ccf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 --2- 172.21.15.57:0/1236585730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccd4d0 0x561151ccf1e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 -- 172.21.15.57:0/1236585730 >> 172.21.15.57:0/1236585730 conn(0x561151cc8a40 msgr2=0x561151cc3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.121+0000 7f74e95df740 1 -- 172.21.15.57:0/1236585730 shutdown_connections 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.122+0000 7f74e95df740 1 -- 172.21.15.57:0/1236585730 wait complete. 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.122+0000 7f74e95df740 1 -- start start 2024-04-13T05:20:49.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.122+0000 7f74e95df740 1 Processor -- start 2024-04-13T05:20:49.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccd4d0 0x561151cdbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccdec0 0x561151cdc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151d47750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f74c000e220 con 0x561151cd3030 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74c000f690 con 0x561151ccd4d0 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74c000e8f0 con 0x561151ccdec0 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74e95df740 10 client.0 _ll_register_callbacks cb 0x7f74d8004f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccd4d0 0x561151cdbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccd4d0 0x561151cdbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:59742/0 (socket says 172.21.15.57:59742) 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.123+0000 7f74dfe00640 1 -- 172.21.15.57:0/1345982244 learned_addr learned my addr 172.21.15.57:0/1345982244 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:20:49.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.124+0000 7f74dea00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccdec0 0x561151cdc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.124+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151d47750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.125+0000 7f74dea00640 1 -- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccd4d0 msgr2=0x561151cdbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.125+0000 7f74dea00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccd4d0 0x561151cdbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.125+0000 7f74dea00640 1 -- 172.21.15.57:0/1345982244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 msgr2=0x561151d47750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:20:49.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.125+0000 7f74dea00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151d47750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.126+0000 7f74dea00640 1 -- 172.21.15.57:0/1345982244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561151d47c90 con 0x561151ccdec0 2024-04-13T05:20:49.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.126+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151d47750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:20:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.127+0000 7f74dea00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccdec0 0x561151cdc0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02c170 tx=0x7f74cc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aef7c32d5ca6a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:20:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.127+0000 7f74c9600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:20:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.127+0000 7f74c9600640 1 -- 172.21.15.57:0/1345982244 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc04e020 con 0x561151ccdec0 2024-04-13T05:20:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.127+0000 7f74c9600640 1 -- 172.21.15.57:0/1345982244 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f74cc03a040 con 0x561151ccdec0 2024-04-13T05:20:49.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.128+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d8005ab0 con 0x561151ccdec0 2024-04-13T05:20:49.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.128+0000 7f74c9600640 1 -- 172.21.15.57:0/1345982244 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74cc059050 con 0x561151ccdec0 2024-04-13T05:20:49.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.129+0000 7f74e95df740 10 client.25561 fetch_fsmap learned FSMap version 28 2024-04-13T05:20:49.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.129+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561151d48ec0 con 0x561151ccdec0 2024-04-13T05:20:49.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.130+0000 7f74c9600640 1 -- 172.21.15.57:0/1345982244 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d8005ab0 con 0x561151ccdec0 2024-04-13T05:20:49.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.131+0000 7f74c9600640 1 -- 172.21.15.57:0/1345982244 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f74cc04a660 con 0x561151ccdec0 2024-04-13T05:20:49.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.131+0000 7f74e95df740 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:20:49.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.131+0000 7f74e95df740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:20:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.131+0000 7f74e95df740 20 client.25561 populate_metadata read hostname 'smithi057' 2024-04-13T05:20:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.131+0000 7f74e95df740 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151d498a0 0x561151d4bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:20:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.132+0000 7f74e95df740 4 client.25561 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:20:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.132+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561151c41a40 con 0x561151d498a0 2024-04-13T05:20:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.133+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151d498a0 0x561151d4bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:20:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.134+0000 7f74dfe00640 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151d498a0 0x561151d4bcc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215a048434557faa server_cookie=8feac5d53d1b8dbc in_seq=0 out_seq=0 2024-04-13T05:20:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.134+0000 7f74c9600640 10 client.25561 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.135+0000 7f74c9600640 1 -- 172.21.15.57:0/1345982244 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561151c41a40 con 0x561151d498a0 2024-04-13T05:20:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.135+0000 7f74c9600640 10 client.25561 handle_command_reply: tid=0 2024-04-13T05:20:49.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.135+0000 7f74e95df740 1 client.25561 shutdown 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccdec0 msgr2=0x561151cdc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccdec0 0x561151cdc0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74cc02c170 tx=0x7f74cc005850 comp rx=0 tx=0).stop 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 shutdown_connections 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561151d498a0 0x561151d4bcc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74c9600640 0 client.25561 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561151cd3030 0x561151d47750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561151ccdec0 0x561151cdc0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 --2- 172.21.15.57:0/1345982244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561151ccd4d0 0x561151cdbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 >> 172.21.15.57:0/1345982244 conn(0x561151cc8a40 msgr2=0x561151ccf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.136+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 shutdown_connections 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.137+0000 7f74e95df740 1 -- 172.21.15.57:0/1345982244 wait complete. 2024-04-13T05:20:49.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.137+0000 7f74e95df740 20 client.25561 trim_cache size 0 max 16384 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: "62669c43-84a4-4aaf-b057-257b92963edf": { 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "62669c43-84a4-4aaf-b057-257b92963edf", 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:20:49.140 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74c40a3c30 msgr2=0x7f74c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74c40a3c30 0x7f74c40a60f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74d00ce3f0 tx=0x7f74d000efb0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 msgr2=0x7f74e019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e019e8a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74d000e9c0 tx=0x7f74d000ee80 comp rx=0 tx=0).stop 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 shutdown_connections 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74c40a3c30 0x7f74c40a60f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74e01053f0 0x7f74e019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74e0105db0 0x7f74e01a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:20:49.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 --2- 172.21.15.57:0/1372847177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74e0104200 0x7f74e019e8a0 secure :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f74d000e9c0 tx=0x7f74d000ee80 comp rx=0 tx=0).stop 2024-04-13T05:20:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 >> 172.21.15.57:0/1372847177 conn(0x7f74e00ff770 msgr2=0x7f74e0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:20:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.138+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 shutdown_connections 2024-04-13T05:20:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:20:49.139+0000 7f74e6600640 1 -- 172.21.15.57:0/1372847177 wait complete. 2024-04-13T05:20:49.162 INFO:tasks.cephfs.filesystem:scrub status for tag:62669c43-84a4-4aaf-b057-257b92963edf - {'path': '/', 'tag': '62669c43-84a4-4aaf-b057-257b92963edf', 'options': 'recursive,force'} 2024-04-13T05:20:49.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:48 smithi083 ceph-mon[35133]: pgmap v552: 129 pgs: 129 active+clean; 58 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:20:49.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:48 smithi189 ceph-mon[35531]: pgmap v552: 129 pgs: 129 active+clean; 58 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:20:49.635 INFO:tasks.workunit.client.0.smithi057.stdout: 13 8155 293 4514 194 2710 206 2024-04-13T05:20:50.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:49 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:20:50.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:49 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:20:50.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:50.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:49 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:20:51.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:50 smithi083 ceph-mon[35133]: pgmap v553: 129 pgs: 129 active+clean; 64 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-04-13T05:20:51.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:50 smithi057 ceph-mon[32331]: pgmap v553: 129 pgs: 129 active+clean; 64 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-04-13T05:20:51.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:50 smithi189 ceph-mon[35531]: pgmap v553: 129 pgs: 129 active+clean; 64 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-04-13T05:20:53.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:52 smithi083 ceph-mon[35133]: pgmap v554: 129 pgs: 129 active+clean; 66 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-04-13T05:20:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:52 smithi057 ceph-mon[32331]: pgmap v554: 129 pgs: 129 active+clean; 66 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-04-13T05:20:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:52 smithi189 ceph-mon[35531]: pgmap v554: 129 pgs: 129 active+clean; 66 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-04-13T05:20:55.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:54 smithi083 ceph-mon[35133]: pgmap v555: 129 pgs: 129 active+clean; 70 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-13T05:20:55.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:54 smithi057 ceph-mon[32331]: pgmap v555: 129 pgs: 129 active+clean; 70 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-13T05:20:55.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:54 smithi189 ceph-mon[35531]: pgmap v555: 129 pgs: 129 active+clean; 70 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-04-13T05:20:56.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:55 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:20:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:55 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:20:56.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:55 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:20:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:56 smithi083 ceph-mon[35133]: pgmap v556: 129 pgs: 129 active+clean; 72 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.7 MiB/s wr, 149 op/s 2024-04-13T05:20:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:20:57.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:20:57.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:56 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:20:57.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:56 smithi057 ceph-mon[32331]: pgmap v556: 129 pgs: 129 active+clean; 72 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.7 MiB/s wr, 149 op/s 2024-04-13T05:20:57.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:20:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:20:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:56 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:20:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:56 smithi189 ceph-mon[35531]: pgmap v556: 129 pgs: 129 active+clean; 72 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.7 MiB/s wr, 149 op/s 2024-04-13T05:20:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:20:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:20:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:56 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:20:59.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:20:58 smithi083 ceph-mon[35133]: pgmap v557: 129 pgs: 129 active+clean; 72 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:20:59.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:20:58 smithi057 ceph-mon[32331]: pgmap v557: 129 pgs: 129 active+clean; 72 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:20:59.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:20:58 smithi189 ceph-mon[35531]: pgmap v557: 129 pgs: 129 active+clean; 72 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:20:59.635 INFO:tasks.workunit.client.0.smithi057.stdout: 16 9822 119 6625 150 2793 165 2024-04-13T05:21:01.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:00 smithi083 ceph-mon[35133]: pgmap v558: 129 pgs: 129 active+clean; 77 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-04-13T05:21:01.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:00 smithi057 ceph-mon[32331]: pgmap v558: 129 pgs: 129 active+clean; 77 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-04-13T05:21:01.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:00 smithi189 ceph-mon[35531]: pgmap v558: 129 pgs: 129 active+clean; 77 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-04-13T05:21:03.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:02 smithi083 ceph-mon[35133]: pgmap v559: 129 pgs: 129 active+clean; 78 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-04-13T05:21:03.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:02 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:21:03.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:02 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:21:03.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:02 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:21:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:02 smithi057 ceph-mon[32331]: pgmap v559: 129 pgs: 129 active+clean; 78 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-04-13T05:21:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:02 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:21:03.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:02 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:21:03.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:02 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:21:03.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:02 smithi189 ceph-mon[35531]: pgmap v559: 129 pgs: 129 active+clean; 78 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-04-13T05:21:03.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:02 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:21:03.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:02 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:21:03.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:02 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:21:05.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:04 smithi083 ceph-mon[35133]: pgmap v560: 129 pgs: 129 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-04-13T05:21:05.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:04 smithi057 ceph-mon[32331]: pgmap v560: 129 pgs: 129 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-04-13T05:21:05.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:04 smithi189 ceph-mon[35531]: pgmap v560: 129 pgs: 129 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-04-13T05:21:07.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:06 smithi083 ceph-mon[35133]: pgmap v561: 129 pgs: 129 active+clean; 85 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:21:07.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:06 smithi057 ceph-mon[32331]: pgmap v561: 129 pgs: 129 active+clean; 85 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:21:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:06 smithi189 ceph-mon[35531]: pgmap v561: 129 pgs: 129 active+clean; 85 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:21:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:08 smithi083 ceph-mon[35133]: pgmap v562: 129 pgs: 129 active+clean; 85 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-04-13T05:21:09.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:08 smithi057 ceph-mon[32331]: pgmap v562: 129 pgs: 129 active+clean; 85 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-04-13T05:21:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:08 smithi189 ceph-mon[35531]: pgmap v562: 129 pgs: 129 active+clean; 85 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-04-13T05:21:09.635 INFO:tasks.workunit.client.0.smithi057.stdout: 21 8520 244 5113 289 3393 115 2024-04-13T05:21:11.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:10 smithi083 ceph-mon[35133]: pgmap v563: 129 pgs: 129 active+clean; 95 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-04-13T05:21:11.269 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:10 smithi189 ceph-mon[35531]: pgmap v563: 129 pgs: 129 active+clean; 95 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-04-13T05:21:11.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:10 smithi057 ceph-mon[32331]: pgmap v563: 129 pgs: 129 active+clean; 95 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-04-13T05:21:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:12 smithi083 ceph-mon[35133]: pgmap v564: 129 pgs: 129 active+clean; 97 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-04-13T05:21:13.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:12 smithi057 ceph-mon[32331]: pgmap v564: 129 pgs: 129 active+clean; 97 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-04-13T05:21:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:12 smithi189 ceph-mon[35531]: pgmap v564: 129 pgs: 129 active+clean; 97 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-04-13T05:21:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:14 smithi083 ceph-mon[35133]: pgmap v565: 129 pgs: 129 active+clean; 103 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-04-13T05:21:15.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:14 smithi057 ceph-mon[32331]: pgmap v565: 129 pgs: 129 active+clean; 103 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-04-13T05:21:15.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:14 smithi189 ceph-mon[35531]: pgmap v565: 129 pgs: 129 active+clean; 103 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-04-13T05:21:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:16 smithi083 ceph-mon[35133]: pgmap v566: 129 pgs: 129 active+clean; 110 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-04-13T05:21:17.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:16 smithi057 ceph-mon[32331]: pgmap v566: 129 pgs: 129 active+clean; 110 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-04-13T05:21:17.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:16 smithi189 ceph-mon[35531]: pgmap v566: 129 pgs: 129 active+clean; 110 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-04-13T05:21:19.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:21:19.163 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:21:19.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:18 smithi057 ceph-mon[32331]: pgmap v567: 129 pgs: 129 active+clean; 111 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-04-13T05:21:19.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:18 smithi083 ceph-mon[35133]: pgmap v567: 129 pgs: 129 active+clean; 111 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-04-13T05:21:19.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:18 smithi189 ceph-mon[35531]: pgmap v567: 129 pgs: 129 active+clean; 111 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-04-13T05:21:19.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 -- 172.21.15.57:0/3595994894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 msgr2=0x7f06f0078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 --2- 172.21.15.57:0/3595994894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f0078e00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009900 tx=0x7f06e802bc40 comp rx=0 tx=0).stop 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 -- 172.21.15.57:0/3595994894 shutdown_connections 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 --2- 172.21.15.57:0/3595994894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f0078e00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 --2- 172.21.15.57:0/3595994894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0077bc0 0x7f06f0078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 --2- 172.21.15.57:0/3595994894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 0x7f06f0076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.422+0000 7f06f5200640 1 -- 172.21.15.57:0/3595994894 >> 172.21.15.57:0/3595994894 conn(0x7f06f00ff670 msgr2=0x7f06f0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.423+0000 7f06f5200640 1 -- 172.21.15.57:0/3595994894 shutdown_connections 2024-04-13T05:21:19.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.423+0000 7f06f5200640 1 -- 172.21.15.57:0/3595994894 wait complete. 2024-04-13T05:21:19.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 Processor -- start 2024-04-13T05:21:19.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 -- start start 2024-04-13T05:21:19.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 0x7f06f019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0077bc0 0x7f06f019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f01a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0114080 con 0x7f06f0078560 2024-04-13T05:21:19.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0113f00 con 0x7f06f0076980 2024-04-13T05:21:19.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.424+0000 7f06f5200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0114200 con 0x7f06f0077bc0 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.425+0000 7f06efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f01a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.425+0000 7f06efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f01a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40338/0 (socket says 172.21.15.57:40338) 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.425+0000 7f06efe00640 1 -- 172.21.15.57:0/913325863 learned_addr learned my addr 172.21.15.57:0/913325863 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.425+0000 7f06ef400640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 0x7f06f019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.425+0000 7f06eea00640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0077bc0 0x7f06f019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06efe00640 1 -- 172.21.15.57:0/913325863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0077bc0 msgr2=0x7f06f019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06efe00640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0077bc0 0x7f06f019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06efe00640 1 -- 172.21.15.57:0/913325863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 msgr2=0x7f06f019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06efe00640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 0x7f06f019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06efe00640 1 -- 172.21.15.57:0/913325863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f01a62a0 con 0x7f06f0078560 2024-04-13T05:21:19.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06ef400640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 0x7f06f019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:19.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.426+0000 7f06efe00640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f01a5c30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f06e8036040 tx=0x7f06e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.427+0000 7f06e7e00640 1 -- 172.21.15.57:0/913325863 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8043020 con 0x7f06f0078560 2024-04-13T05:21:19.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.427+0000 7f06e7e00640 1 -- 172.21.15.57:0/913325863 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e80052b0 con 0x7f06f0078560 2024-04-13T05:21:19.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.427+0000 7f06e7e00640 1 -- 172.21.15.57:0/913325863 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8003c60 con 0x7f06f0078560 2024-04-13T05:21:19.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.427+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f01a6530 con 0x7f06f0078560 2024-04-13T05:21:19.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.427+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01a6a30 con 0x7f06f0078560 2024-04-13T05:21:19.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.433+0000 7f06e7e00640 1 -- 172.21.15.57:0/913325863 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f06e8046020 con 0x7f06f0078560 2024-04-13T05:21:19.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.435+0000 7f06e7e00640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06d40a3c30 0x7f06d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.435+0000 7f06e7e00640 1 -- 172.21.15.57:0/913325863 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f06e80f2960 con 0x7f06f0078560 2024-04-13T05:21:19.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.435+0000 7f06f9641740 1 Processor -- start 2024-04-13T05:21:19.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.435+0000 7f06ef400640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06d40a3c30 0x7f06d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.435+0000 7f06f9641740 1 -- start start 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06f9641740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d43bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06f9641740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 0x55895d440be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06f9641740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06f9641740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55895d3a2020 con 0x55895d43c0e0 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06f9641740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55895d3486d0 con 0x55895d441250 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06f9641740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55895d39dfe0 con 0x55895d43b6f0 2024-04-13T05:21:19.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.436+0000 7f06ef400640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06d40a3c30 0x7f06d40a60f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009fd0 tx=0x7f06e0009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 0x55895d440be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 0x55895d440be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40350/0 (socket says 172.21.15.57:40350) 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 -- 172.21.15.57:0/2171563759 learned_addr learned my addr 172.21.15.57:0/2171563759 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06efe00640 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06ef400640 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d43bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 -- 172.21.15.57:0/2171563759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 msgr2=0x55895d43bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d43bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 -- 172.21.15.57:0/2171563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 msgr2=0x55895d4436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.437+0000 7f06eea00640 1 -- 172.21.15.57:0/2171563759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895d17e7d0 con 0x55895d43c0e0 2024-04-13T05:21:19.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.438+0000 7f06eea00640 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 0x55895d440be0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009c90 tx=0x7f06d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de25abf2fde48199 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.439+0000 7f06e4c00640 1 -- 172.21.15.57:0/2171563759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d801e070 con 0x55895d43c0e0 2024-04-13T05:21:19.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.439+0000 7f06e4c00640 1 -- 172.21.15.57:0/2171563759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d8019ae0 con 0x55895d43c0e0 2024-04-13T05:21:19.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.439+0000 7f06e4c00640 1 -- 172.21.15.57:0/2171563759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d800ea40 con 0x55895d43c0e0 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.440+0000 7f06f9641740 1 -- 172.21.15.57:0/2171563759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 msgr2=0x55895d440be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.440+0000 7f06f9641740 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 0x55895d440be0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009c90 tx=0x7f06d8011810 comp rx=0 tx=0).stop 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 -- 172.21.15.57:0/2171563759 shutdown_connections 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43c0e0 0x55895d440be0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 --2- 172.21.15.57:0/2171563759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d43bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 -- 172.21.15.57:0/2171563759 >> 172.21.15.57:0/2171563759 conn(0x55895d436c60 msgr2=0x55895d437060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 -- 172.21.15.57:0/2171563759 shutdown_connections 2024-04-13T05:21:19.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.441+0000 7f06f9641740 1 -- 172.21.15.57:0/2171563759 wait complete. 2024-04-13T05:21:19.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.442+0000 7f06f9641740 1 -- start start 2024-04-13T05:21:19.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.442+0000 7f06f9641740 1 Processor -- start 2024-04-13T05:21:19.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43b6f0 0x55895d445a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d445fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40356/0 (socket says 172.21.15.57:40356) 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06efe00640 1 -- 172.21.15.57:0/1019182738 learned_addr learned my addr 172.21.15.57:0/1019182738 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55895d444200 con 0x55895d441250 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55895d35ba90 con 0x55895d43b6f0 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55895d444380 con 0x55895d43c0e0 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06ef400640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43b6f0 0x55895d445a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06f9641740 10 client.0 _ll_register_callbacks cb 0x55895d35b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:19.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.443+0000 7f06eea00640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d445fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.444+0000 7f06efe00640 1 -- 172.21.15.57:0/1019182738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 msgr2=0x55895d445fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.444+0000 7f06efe00640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d445fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.444+0000 7f06efe00640 1 -- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43b6f0 msgr2=0x55895d445a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.444+0000 7f06efe00640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43b6f0 0x55895d445a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.444+0000 7f06efe00640 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895d0b43f0 con 0x55895d441250 2024-04-13T05:21:19.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.444+0000 7f06efe00640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44cfe0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f06e80f0430 tx=0x7f06e80be2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93a3c6eb7c4be37 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.445+0000 7f06dd600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:19.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.445+0000 7f06dd600640 1 -- 172.21.15.57:0/1019182738 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e80c2c50 con 0x55895d441250 2024-04-13T05:21:19.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.445+0000 7f06dd600640 1 -- 172.21.15.57:0/1019182738 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f06e80baf10 con 0x55895d441250 2024-04-13T05:21:19.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.445+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d8019ae0 con 0x55895d441250 2024-04-13T05:21:19.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.446+0000 7f06dd600640 1 -- 172.21.15.57:0/1019182738 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06e80c7510 con 0x55895d441250 2024-04-13T05:21:19.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.446+0000 7f06f9641740 10 client.16179 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:19.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.446+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55895d0740f0 con 0x55895d441250 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.446+0000 7f06dd600640 1 -- 172.21.15.57:0/1019182738 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d8019ae0 con 0x55895d441250 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06dd600640 1 -- 172.21.15.57:0/1019182738 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f06e80bfbd0 con 0x55895d441250 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06f9641740 10 client.16179 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06f9641740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06f9641740 20 client.16179 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06f9641740 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4517a0 0x55895d453be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06f9641740 4 client.16179 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:19.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.447+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895d341600 con 0x55895d4517a0 2024-04-13T05:21:19.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.448+0000 7f06ef400640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4517a0 0x55895d453be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.451+0000 7f06dd600640 10 client.16179 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:19.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.451+0000 7f06ef400640 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4517a0 0x55895d453be0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a761d6b8f32b755c server_cookie=c57142df87f90e1f in_seq=0 out_seq=0 2024-04-13T05:21:19.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.453+0000 7f06dd600640 1 -- 172.21.15.57:0/1019182738 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55895d341600 con 0x55895d4517a0 2024-04-13T05:21:19.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.453+0000 7f06dd600640 10 client.16179 handle_command_reply: tid=0 2024-04-13T05:21:19.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.453+0000 7f06f9641740 1 client.16179 shutdown 2024-04-13T05:21:19.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.453+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 msgr2=0x55895d44cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.453+0000 7f06f9641740 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44cfe0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f06e80f0430 tx=0x7f06e80be2c0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 shutdown_connections 2024-04-13T05:21:19.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4517a0 0x55895d453be0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44cfe0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d445fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 --2- 172.21.15.57:0/1019182738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43b6f0 0x55895d445a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 >> 172.21.15.57:0/1019182738 conn(0x55895d436c60 msgr2=0x55895d435e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06dd600640 0 client.16179 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 shutdown_connections 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 1 -- 172.21.15.57:0/1019182738 wait complete. 2024-04-13T05:21:19.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.454+0000 7f06f9641740 20 client.16179 trim_cache size 0 max 16384 2024-04-13T05:21:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 Processor -- start 2024-04-13T05:21:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 -- start start 2024-04-13T05:21:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 0x55895d445050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d44cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01b3790 con 0x55895d43b6f0 2024-04-13T05:21:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01b1000 con 0x55895d441250 2024-04-13T05:21:19.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.487+0000 7f06f9641740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aee50 con 0x55895d43c0e0 2024-04-13T05:21:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.488+0000 7f06ef400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 0x55895d445050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.488+0000 7f06ef400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 0x55895d445050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40368/0 (socket says 172.21.15.57:40368) 2024-04-13T05:21:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.488+0000 7f06ef400640 1 -- 172.21.15.57:0/690755800 learned_addr learned my addr 172.21.15.57:0/690755800 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.488+0000 7f06efe00640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.488+0000 7f06eea00640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d44cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06efe00640 1 -- 172.21.15.57:0/690755800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 msgr2=0x55895d44cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06efe00640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d44cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06efe00640 1 -- 172.21.15.57:0/690755800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 msgr2=0x55895d445050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06efe00640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 0x55895d445050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06efe00640 1 -- 172.21.15.57:0/690755800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895d43c970 con 0x55895d441250 2024-04-13T05:21:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06eea00640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d44cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.489+0000 7f06ef400640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 0x55895d445050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.490+0000 7f06efe00640 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4355f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f06e80c72c0 tx=0x7f06e80bcdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b587a0ece41093 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.490+0000 7f06dfe00640 1 -- 172.21.15.57:0/690755800 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e80c2c50 con 0x55895d441250 2024-04-13T05:21:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.490+0000 7f06dfe00640 1 -- 172.21.15.57:0/690755800 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e80baf10 con 0x55895d441250 2024-04-13T05:21:19.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.490+0000 7f06dfe00640 1 -- 172.21.15.57:0/690755800 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e80ba5f0 con 0x55895d441250 2024-04-13T05:21:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 -- 172.21.15.57:0/690755800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 msgr2=0x55895d4355f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4355f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f06e80c72c0 tx=0x7f06e80bcdb0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 -- 172.21.15.57:0/690755800 shutdown_connections 2024-04-13T05:21:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d441250 0x55895d4355f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43c0e0 0x55895d44cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 --2- 172.21.15.57:0/690755800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d43b6f0 0x55895d445050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 -- 172.21.15.57:0/690755800 >> 172.21.15.57:0/690755800 conn(0x55895d436c60 msgr2=0x55895d445a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.492+0000 7f06f9641740 1 -- 172.21.15.57:0/690755800 shutdown_connections 2024-04-13T05:21:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.493+0000 7f06f9641740 1 -- 172.21.15.57:0/690755800 wait complete. 2024-04-13T05:21:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.493+0000 7f06f9641740 1 -- start start 2024-04-13T05:21:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 Processor -- start 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d44a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43c0e0 0x55895d44ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01af4c0 con 0x55895d441250 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01b0930 con 0x55895d43c0e0 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01afb90 con 0x55895d43b6f0 2024-04-13T05:21:19.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.494+0000 7f06f9641740 10 client.0 _ll_register_callbacks cb 0x7f06e80be490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.495+0000 7f06ef400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d44a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.495+0000 7f06ef400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d44a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:40854/0 (socket says 172.21.15.57:40854) 2024-04-13T05:21:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06ef400640 1 -- 172.21.15.57:0/2279516558 learned_addr learned my addr 172.21.15.57:0/2279516558 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:19.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06eea00640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43c0e0 0x55895d44ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06efe00640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06ef400640 1 -- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43c0e0 msgr2=0x55895d44ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06ef400640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43c0e0 0x55895d44ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06ef400640 1 -- 172.21.15.57:0/2279516558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 msgr2=0x55895d44bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06ef400640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.496+0000 7f06ef400640 1 -- 172.21.15.57:0/2279516558 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895d44c320 con 0x55895d43b6f0 2024-04-13T05:21:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.497+0000 7f06ef400640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d44a7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55895d43c940 tx=0x7f06e0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22a449a81565cb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.497+0000 7f06dd600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:21:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.498+0000 7f06dd600640 1 -- 172.21.15.57:0/2279516558 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0012aa0 con 0x55895d43b6f0 2024-04-13T05:21:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.498+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e80baf10 con 0x55895d43b6f0 2024-04-13T05:21:19.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.498+0000 7f06dd600640 1 -- 172.21.15.57:0/2279516558 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f06e0012c40 con 0x55895d43b6f0 2024-04-13T05:21:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.499+0000 7f06dd600640 1 -- 172.21.15.57:0/2279516558 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06e0012e40 con 0x55895d43b6f0 2024-04-13T05:21:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.499+0000 7f06f9641740 10 client.25579 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.499+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55895d4c8f20 con 0x55895d43b6f0 2024-04-13T05:21:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06dd600640 1 -- 172.21.15.57:0/2279516558 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e80baf10 con 0x55895d43b6f0 2024-04-13T05:21:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06dd600640 1 -- 172.21.15.57:0/2279516558 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f06e0033070 con 0x55895d43b6f0 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06f9641740 10 client.25579 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06f9641740 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06f9641740 20 client.25579 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06f9641740 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4c9a00 0x55895d4cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06f9641740 4 client.25579 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.500+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55895d445850 con 0x55895d4c9a00 2024-04-13T05:21:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.501+0000 7f06eea00640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4c9a00 0x55895d4cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.502+0000 7f06eea00640 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4c9a00 0x55895d4cbe20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fafe5ca8ba8cf1 server_cookie=4c7afe670bbd93d2 in_seq=0 out_seq=0 2024-04-13T05:21:19.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.502+0000 7f06dd600640 10 client.25579 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:19.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.503+0000 7f06dd600640 1 -- 172.21.15.57:0/2279516558 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55895d445850 con 0x55895d4c9a00 2024-04-13T05:21:19.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.503+0000 7f06dd600640 10 client.25579 handle_command_reply: tid=0 2024-04-13T05:21:19.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.503+0000 7f06f9641740 1 client.25579 shutdown 2024-04-13T05:21:19.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.503+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 msgr2=0x55895d44a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.503+0000 7f06f9641740 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d44a7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55895d43c940 tx=0x7f06e0004a40 comp rx=0 tx=0).stop 2024-04-13T05:21:19.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 shutdown_connections 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55895d4c9a00 0x55895d4cbe20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55895d43c0e0 0x55895d44ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55895d441250 0x55895d44bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 --2- 172.21.15.57:0/2279516558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55895d43b6f0 0x55895d44a7e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 >> 172.21.15.57:0/2279516558 conn(0x55895d436c60 msgr2=0x55895d439ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06dd600640 0 client.25579 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 shutdown_connections 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.504+0000 7f06f9641740 1 -- 172.21.15.57:0/2279516558 wait complete. 2024-04-13T05:21:19.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.505+0000 7f06f9641740 20 client.25579 trim_cache size 0 max 16384 2024-04-13T05:21:19.508 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:21:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:21:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:21:19.508 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06d40a3c30 msgr2=0x7f06d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06d40a3c30 0x7f06d40a60f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009fd0 tx=0x7f06e0009680 comp rx=0 tx=0).stop 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 msgr2=0x7f06f01a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f01a5c30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f06e8036040 tx=0x7f06e8004270 comp rx=0 tx=0).stop 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 shutdown_connections 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f06d40a3c30 0x7f06d40a60f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f06f0078560 0x7f06f01a5c30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0077bc0 0x7f06f019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 --2- 172.21.15.57:0/913325863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0076980 0x7f06f019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.506+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 >> 172.21.15.57:0/913325863 conn(0x7f06f00ff670 msgr2=0x7f06f0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.507+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 shutdown_connections 2024-04-13T05:21:19.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.507+0000 7f06f5200640 1 -- 172.21.15.57:0/913325863 wait complete. 2024-04-13T05:21:19.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:21:19.531 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:21:19.635 INFO:tasks.workunit.client.0.smithi057.stdout: 28 7897 340 4777 302 1351 223 2024-04-13T05:21:19.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.777+0000 7f267be00640 1 -- 172.21.15.57:0/2652019828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 msgr2=0x7f267c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.777+0000 7f267be00640 1 --2- 172.21.15.57:0/2652019828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 0x7f267c101e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ca0 tx=0x7f267402c100 comp rx=0 tx=0).stop 2024-04-13T05:21:19.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.777+0000 7f267be00640 1 -- 172.21.15.57:0/2652019828 shutdown_connections 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.777+0000 7f267be00640 1 --2- 172.21.15.57:0/2652019828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 0x7f267c101e90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.778+0000 7f267be00640 1 --2- 172.21.15.57:0/2652019828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c101070 0x7f267c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.778+0000 7f267be00640 1 --2- 172.21.15.57:0/2652019828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.778+0000 7f267be00640 1 -- 172.21.15.57:0/2652019828 >> 172.21.15.57:0/2652019828 conn(0x7f267c0fc290 msgr2=0x7f267c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.778+0000 7f267be00640 1 -- 172.21.15.57:0/2652019828 shutdown_connections 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.778+0000 7f267be00640 1 -- 172.21.15.57:0/2652019828 wait complete. 2024-04-13T05:21:19.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.779+0000 7f267be00640 1 Processor -- start 2024-04-13T05:21:19.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.779+0000 7f267be00640 1 -- start start 2024-04-13T05:21:19.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.781+0000 7f267be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c105170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.781+0000 7f267be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c101070 0x7f267c1056b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.781+0000 7f267be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 0x7f267c10c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.781+0000 7f267be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f267c10fb90 con 0x7f267c100710 2024-04-13T05:21:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.781+0000 7f267be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f267c10fa10 con 0x7f267c101a10 2024-04-13T05:21:19.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.781+0000 7f267aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c105170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c105170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40398/0 (socket says 172.21.15.57:40398) 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 -- 172.21.15.57:0/626846308 learned_addr learned my addr 172.21.15.57:0/626846308 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f267c10fd10 con 0x7f267c101070 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267b400640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 0x7f267c10c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267a000640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c101070 0x7f267c1056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 -- 172.21.15.57:0/626846308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c101070 msgr2=0x7f267c1056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c101070 0x7f267c1056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 -- 172.21.15.57:0/626846308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 msgr2=0x7f267c10c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 0x7f267c10c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.782+0000 7f267aa00640 1 -- 172.21.15.57:0/626846308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1aeaa0 con 0x7f267c100710 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.783+0000 7f267aa00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c105170 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f266c00ea90 tx=0x7f266c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.783+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c00ce40 con 0x7f267c100710 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.783+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1aed90 con 0x7f267c100710 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.784+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1af220 con 0x7f267c100710 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.784+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f266c0035f0 con 0x7f267c100710 2024-04-13T05:21:19.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.784+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c010670 con 0x7f267c100710 2024-04-13T05:21:19.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.785+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c0052c0 con 0x7f267c100710 2024-04-13T05:21:19.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.788+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f266c003b10 con 0x7f267c100710 2024-04-13T05:21:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.791+0000 7f2680a00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26600a3d80 0x7f26600a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:19.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.791+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f266c0d0b40 con 0x7f267c100710 2024-04-13T05:21:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.797+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f266c09d230 con 0x7f267c100710 2024-04-13T05:21:19.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.797+0000 7f267a000640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26600a3d80 0x7f26600a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:19.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:19.798+0000 7f267a000640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26600a3d80 0x7f26600a6240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f267c106680 tx=0x7f2668005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.208+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f265c0055b0 con 0x7f267c100710 2024-04-13T05:21:20.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.209+0000 7f2680a00640 1 -- 172.21.15.57:0/626846308 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11859 (secure 0 0 0) 0x7f266c09cb30 con 0x7f267c100710 2024-04-13T05:21:20.212 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:21:20.212 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:20:25.642360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:21:20.212 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 28 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.212+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26600a3d80 msgr2=0x7f26600a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.212+0000 7f267be00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26600a3d80 0x7f26600a6240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f267c106680 tx=0x7f2668005f90 comp rx=0 tx=0).stop 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.212+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 msgr2=0x7f267c105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.212+0000 7f267be00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c105170 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f266c00ea90 tx=0x7f266c00ef50 comp rx=0 tx=0).stop 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 shutdown_connections 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f26600a3d80 0x7f26600a6240 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f267c101a10 0x7f267c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c101070 0x7f267c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 --2- 172.21.15.57:0/626846308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c100710 0x7f267c105170 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 >> 172.21.15.57:0/626846308 conn(0x7f267c0fc290 msgr2=0x7f267c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 shutdown_connections 2024-04-13T05:21:20.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.213+0000 7f267be00640 1 -- 172.21.15.57:0/626846308 wait complete. 2024-04-13T05:21:20.239 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:21:20.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 -- 172.21.15.57:0/1117645107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f80749c0 msgr2=0x7f34f8074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 --2- 172.21.15.57:0/1117645107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f80749c0 0x7f34f8074e20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009900 tx=0x7f34e402bcd0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 -- 172.21.15.57:0/1117645107 shutdown_connections 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 --2- 172.21.15.57:0/1117645107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34f8109ea0 0x7f34f810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 --2- 172.21.15.57:0/1117645107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f80749c0 0x7f34f8074e20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 --2- 172.21.15.57:0/1117645107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f8075fa0 0x7f34f80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 -- 172.21.15.57:0/1117645107 >> 172.21.15.57:0/1117645107 conn(0x7f34f80fd5a0 msgr2=0x7f34f80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 -- 172.21.15.57:0/1117645107 shutdown_connections 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.488+0000 7f3500600640 1 -- 172.21.15.57:0/1117645107 wait complete. 2024-04-13T05:21:20.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.489+0000 7f3500600640 1 Processor -- start 2024-04-13T05:21:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.489+0000 7f3500600640 1 -- start start 2024-04-13T05:21:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f3500600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f80749c0 0x7f34f8105390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f3500600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34f8075fa0 0x7f34f81058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f3500600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 0x7f34f8101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f3500600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f34f810fda0 con 0x7f34f8109ea0 2024-04-13T05:21:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f3500600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34f810fc20 con 0x7f34f8075fa0 2024-04-13T05:21:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f3500600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34f810ff20 con 0x7f34f80749c0 2024-04-13T05:21:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f34fe600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 0x7f34f8101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f34fe600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 0x7f34f8101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40428/0 (socket says 172.21.15.57:40428) 2024-04-13T05:21:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f34fe600640 1 -- 172.21.15.57:0/1279820544 learned_addr learned my addr 172.21.15.57:0/1279820544 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f34fd200640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34f8075fa0 0x7f34f81058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.490+0000 7f34fdc00640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f80749c0 0x7f34f8105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34fe600640 1 -- 172.21.15.57:0/1279820544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f80749c0 msgr2=0x7f34f8105390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34fe600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f80749c0 0x7f34f8105390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34fe600640 1 -- 172.21.15.57:0/1279820544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34f8075fa0 msgr2=0x7f34f81058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34fe600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34f8075fa0 0x7f34f81058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34fe600640 1 -- 172.21.15.57:0/1279820544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f81027c0 con 0x7f34f8109ea0 2024-04-13T05:21:20.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34fe600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 0x7f34f8101f00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f34f000eaa0 tx=0x7f34f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34f6000640 1 -- 172.21.15.57:0/1279820544 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f000cdb0 con 0x7f34f8109ea0 2024-04-13T05:21:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.491+0000 7f34f6000640 1 -- 172.21.15.57:0/1279820544 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34f0005020 con 0x7f34f8109ea0 2024-04-13T05:21:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.492+0000 7f34f6000640 1 -- 172.21.15.57:0/1279820544 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00116d0 con 0x7f34f8109ea0 2024-04-13T05:21:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.492+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81a68d0 con 0x7f34f8109ea0 2024-04-13T05:21:20.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.492+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81a6ce0 con 0x7f34f8109ea0 2024-04-13T05:21:20.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.495+0000 7f3503619740 1 Processor -- start 2024-04-13T05:21:20.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.496+0000 7f34f6000640 1 -- 172.21.15.57:0/1279820544 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f34f00051d0 con 0x7f34f8109ea0 2024-04-13T05:21:20.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.496+0000 7f3503619740 1 -- start start 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f3503619740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 0x55e31c0d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f3503619740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0d9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f3503619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f3503619740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e31c03a020 con 0x55e31c0d4250 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f3503619740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e31bfe0950 con 0x55e31c0d4c40 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f3503619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e31c035fe0 con 0x55e31c0d9db0 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f34fd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 0x55e31c0d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f34fd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 0x55e31c0d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40438/0 (socket says 172.21.15.57:40438) 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.497+0000 7f34fd200640 1 -- 172.21.15.57:0/1272643455 learned_addr learned my addr 172.21.15.57:0/1272643455 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34fd200640 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34fd200640 1 -- 172.21.15.57:0/1272643455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 msgr2=0x55e31c0dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34fd200640 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0dc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34fd200640 1 -- 172.21.15.57:0/1272643455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 msgr2=0x55e31c0d9740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34fd200640 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0d9740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34fd200640 1 -- 172.21.15.57:0/1272643455 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31bd4c3f0 con 0x55e31c0d4250 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.498+0000 7f34f6000640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f34e00a3c30 0x7f34e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.499+0000 7f34f6000640 1 -- 172.21.15.57:0/1279820544 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f34f00d0610 con 0x7f34f8109ea0 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.499+0000 7f34fdc00640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f34e00a3c30 0x7f34e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.500+0000 7f34fd200640 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 0x55e31c0d4670 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34e4036040 tx=0x7f34e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da5fe82033a84d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.500+0000 7f34fdc00640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f34e00a3c30 0x7f34e00a60f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f34ec005d90 tx=0x7f34ec005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.501+0000 7f34eaa00640 1 -- 172.21.15.57:0/1272643455 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4047070 con 0x55e31c0d4250 2024-04-13T05:21:20.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.502+0000 7f34eaa00640 1 -- 172.21.15.57:0/1272643455 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34e402ce30 con 0x55e31c0d4250 2024-04-13T05:21:20.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.502+0000 7f34eaa00640 1 -- 172.21.15.57:0/1272643455 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4047070 con 0x55e31c0d4250 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.502+0000 7f3503619740 1 -- 172.21.15.57:0/1272643455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 msgr2=0x55e31c0d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.502+0000 7f3503619740 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 0x55e31c0d4670 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34e4036040 tx=0x7f34e4000bc0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 -- 172.21.15.57:0/1272643455 shutdown_connections 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0d9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 --2- 172.21.15.57:0/1272643455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d4250 0x55e31c0d4670 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 -- 172.21.15.57:0/1272643455 >> 172.21.15.57:0/1272643455 conn(0x55e31c0cf7c0 msgr2=0x55e31c0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 -- 172.21.15.57:0/1272643455 shutdown_connections 2024-04-13T05:21:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 -- 172.21.15.57:0/1272643455 wait complete. 2024-04-13T05:21:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.503+0000 7f3503619740 1 -- start start 2024-04-13T05:21:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.504+0000 7f3503619740 1 Processor -- start 2024-04-13T05:21:20.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.504+0000 7f3503619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d4c40 0x55e31c0de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.504+0000 7f3503619740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d9db0 0x55e31c0dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.504+0000 7f3503619740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0e6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f34fd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d9db0 0x55e31c0dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f3503619740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e31c0dcd00 con 0x55e31c0e3e30 2024-04-13T05:21:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f34fd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d9db0 0x55e31c0dee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55706/0 (socket says 172.21.15.57:55706) 2024-04-13T05:21:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f34fd200640 1 -- 172.21.15.57:0/2360234751 learned_addr learned my addr 172.21.15.57:0/2360234751 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e31bff3c80 con 0x55e31c0d9db0 2024-04-13T05:21:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e31c0dce80 con 0x55e31c0d4c40 2024-04-13T05:21:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f3503619740 10 client.0 _ll_register_callbacks cb 0x55e31bff3670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:20.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f34fe600640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.505+0000 7f34fdc00640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d4c40 0x55e31c0de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fe600640 1 -- 172.21.15.57:0/2360234751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d4c40 msgr2=0x55e31c0de8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fe600640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d4c40 0x55e31c0de8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fe600640 1 -- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d9db0 msgr2=0x55e31c0dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fe600640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d9db0 0x55e31c0dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fe600640 1 -- 172.21.15.57:0/2360234751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31be167d0 con 0x55e31c0e3e30 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fdc00640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d4c40 0x55e31c0de8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34fe600640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0e6220 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34f00cddc0 tx=0x7f34f009cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f67e8bfa02e5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34df400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34df400640 1 -- 172.21.15.57:0/2360234751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f001f040 con 0x55e31c0e3e30 2024-04-13T05:21:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f34df400640 1 -- 172.21.15.57:0/2360234751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f34f00d0340 con 0x55e31c0e3e30 2024-04-13T05:21:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.506+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e402ce30 con 0x55e31c0e3e30 2024-04-13T05:21:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f34df400640 1 -- 172.21.15.57:0/2360234751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34f00a51c0 con 0x55e31c0e3e30 2024-04-13T05:21:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f3503619740 10 client.16203 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:20.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e31c0e9660 con 0x55e31c0e3e30 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f34df400640 1 -- 172.21.15.57:0/2360234751 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34e402ce30 con 0x55e31c0e3e30 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f34df400640 1 -- 172.21.15.57:0/2360234751 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f34f000f040 con 0x55e31c0e3e30 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f3503619740 10 client.16203 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f3503619740 10 client.16203 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f3503619740 20 client.16203 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.507+0000 7f3503619740 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c0ead40 0x55e31c0ed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.508+0000 7f3503619740 4 client.16203 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.508+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e31bfd9750 con 0x55e31c0ead40 2024-04-13T05:21:20.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.508+0000 7f34fdc00640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c0ead40 0x55e31c0ed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.509+0000 7f34fdc00640 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c0ead40 0x55e31c0ed1e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dade38ce7b51882f server_cookie=a4eacfc7d435dc85 in_seq=0 out_seq=0 2024-04-13T05:21:20.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.509+0000 7f34df400640 10 client.16203 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:20.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.511+0000 7f34df400640 1 -- 172.21.15.57:0/2360234751 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e31bfd9750 con 0x55e31c0ead40 2024-04-13T05:21:20.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.511+0000 7f34df400640 10 client.16203 handle_command_reply: tid=0 2024-04-13T05:21:20.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.511+0000 7f3503619740 1 client.16203 shutdown 2024-04-13T05:21:20.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.511+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 msgr2=0x55e31c0e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.511+0000 7f3503619740 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0e6220 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34f00cddc0 tx=0x7f34f009cd10 comp rx=0 tx=0).stop 2024-04-13T05:21:20.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 shutdown_connections 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f3503619740 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c0ead40 0x55e31c0ed1e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f3503619740 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d9db0 0x55e31c0dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f3503619740 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0e6220 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f3503619740 1 --2- 172.21.15.57:0/2360234751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d4c40 0x55e31c0de8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 >> 172.21.15.57:0/2360234751 conn(0x55e31c0cf7c0 msgr2=0x55e31c0cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.512+0000 7f34df400640 0 client.16203 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:20.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.513+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 shutdown_connections 2024-04-13T05:21:20.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.513+0000 7f3503619740 1 -- 172.21.15.57:0/2360234751 wait complete. 2024-04-13T05:21:20.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.513+0000 7f3503619740 20 client.16203 trim_cache size 0 max 16384 2024-04-13T05:21:20.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 Processor -- start 2024-04-13T05:21:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 -- start start 2024-04-13T05:21:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b3b10 con 0x55e31c0e3e30 2024-04-13T05:21:20.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.537+0000 7f3503619740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b1380 con 0x55e31c0d4c40 2024-04-13T05:21:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f3503619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81af1d0 con 0x55e31c0d9db0 2024-04-13T05:21:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f34fdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f34fd200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f34fdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55718/0 (socket says 172.21.15.57:55718) 2024-04-13T05:21:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f34fe600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f34fd200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:40918/0 (socket says 172.21.15.57:40918) 2024-04-13T05:21:20.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.538+0000 7f34fd200640 1 -- 172.21.15.57:0/372825677 learned_addr learned my addr 172.21.15.57:0/372825677 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.539+0000 7f34fd200640 1 -- 172.21.15.57:0/372825677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 msgr2=0x55e31c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.539+0000 7f34fd200640 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.539+0000 7f34fd200640 1 -- 172.21.15.57:0/372825677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 msgr2=0x55e31c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.539+0000 7f34fd200640 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.539+0000 7f34fd200640 1 -- 172.21.15.57:0/372825677 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31c0d3e90 con 0x55e31c0d9db0 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.539+0000 7f34fe600640 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.540+0000 7f34fd200640 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0ce730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009900 tx=0x7f34e4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232c2c3b4567a16e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.541+0000 7f34ea000640 1 -- 172.21.15.57:0/372825677 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4050070 con 0x55e31c0d9db0 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.541+0000 7f34ea000640 1 -- 172.21.15.57:0/372825677 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34e402ce30 con 0x55e31c0d9db0 2024-04-13T05:21:20.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.541+0000 7f34ea000640 1 -- 172.21.15.57:0/372825677 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4047040 con 0x55e31c0d9db0 2024-04-13T05:21:20.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 -- 172.21.15.57:0/372825677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 msgr2=0x55e31c0ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0ce730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009900 tx=0x7f34e4005fd0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 -- 172.21.15.57:0/372825677 shutdown_connections 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0e3e30 0x55e31c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0d9db0 0x55e31c0ce730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 --2- 172.21.15.57:0/372825677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.542+0000 7f3503619740 1 -- 172.21.15.57:0/372825677 >> 172.21.15.57:0/372825677 conn(0x55e31c0cf7c0 msgr2=0x55e31c0d6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.543+0000 7f3503619740 1 -- 172.21.15.57:0/372825677 shutdown_connections 2024-04-13T05:21:20.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.543+0000 7f3503619740 1 -- 172.21.15.57:0/372825677 wait complete. 2024-04-13T05:21:20.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.543+0000 7f3503619740 1 -- start start 2024-04-13T05:21:20.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 Processor -- start 2024-04-13T05:21:20.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 0x55e31c0fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0e3e30 0x55e31c0fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81af840 con 0x55e31c0d9db0 2024-04-13T05:21:20.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b0cb0 con 0x55e31c0d4c40 2024-04-13T05:21:20.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81aff10 con 0x55e31c0e3e30 2024-04-13T05:21:20.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.544+0000 7f3503619740 10 client.0 _ll_register_callbacks cb 0x7f34f009c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:20.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 0x55e31c0fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 0x55e31c0fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40470/0 (socket says 172.21.15.57:40470) 2024-04-13T05:21:20.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fd200640 1 -- 172.21.15.57:0/458899820 learned_addr learned my addr 172.21.15.57:0/458899820 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fe600640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0e3e30 0x55e31c0fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fd200640 1 -- 172.21.15.57:0/458899820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0e3e30 msgr2=0x55e31c0fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fd200640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0e3e30 0x55e31c0fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fd200640 1 -- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 msgr2=0x55e31c0fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.545+0000 7f34fdc00640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34fd200640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0fb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34fd200640 1 -- 172.21.15.57:0/458899820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31c0eab20 con 0x55e31c0d9db0 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e402ce30 con 0x55e31c0d9db0 2024-04-13T05:21:20.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34fdc00640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34fd200640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 0x55e31c0fd5c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f34e4001eb0 tx=0x7f34e40092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94932b6041f9c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34df400640 10 client.16209 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34df400640 1 -- 172.21.15.57:0/458899820 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4050070 con 0x55e31c0d9db0 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.546+0000 7f34df400640 1 -- 172.21.15.57:0/458899820 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f34e40035f0 con 0x55e31c0d9db0 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.547+0000 7f34df400640 1 -- 172.21.15.57:0/458899820 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34e4003830 con 0x55e31c0d9db0 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.547+0000 7f3503619740 10 client.16209 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:20.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.547+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55e31c1735e0 con 0x55e31c0d9db0 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.547+0000 7f34df400640 1 -- 172.21.15.57:0/458899820 <== mon.0 v2:172.21.15.57:3300/0 4 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f34e40039d0 con 0x55e31c0d9db0 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f3503619740 10 client.16209 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f3503619740 10 client.16209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f3503619740 20 client.16209 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f3503619740 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c173950 0x55e31c175d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f3503619740 4 client.16209 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e31c047a40 con 0x55e31c173950 2024-04-13T05:21:20.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.548+0000 7f34fdc00640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c173950 0x55e31c175d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.549+0000 7f34fdc00640 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c173950 0x55e31c175d40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=428e9cefb7893699 server_cookie=85984a8a0c2c7e78 in_seq=0 out_seq=0 2024-04-13T05:21:20.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.549+0000 7f34df400640 10 client.16209 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:20.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.549+0000 7f34df400640 1 -- 172.21.15.57:0/458899820 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e31c047a40 con 0x55e31c173950 2024-04-13T05:21:20.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.549+0000 7f34df400640 10 client.16209 handle_command_reply: tid=0 2024-04-13T05:21:20.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.549+0000 7f3503619740 1 client.16209 shutdown 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 msgr2=0x55e31c0fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 0x55e31c0fd5c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f34e4001eb0 tx=0x7f34e40092a0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 shutdown_connections 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e31c173950 0x55e31c175d40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c0e3e30 0x55e31c0fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e31c0d9db0 0x55e31c0fd5c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 --2- 172.21.15.57:0/458899820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e31c0d4c40 0x55e31c0fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 >> 172.21.15.57:0/458899820 conn(0x55e31c0cf7c0 msgr2=0x55e31c0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f34df400640 0 client.16209 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:20.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.550+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 shutdown_connections 2024-04-13T05:21:20.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.551+0000 7f3503619740 1 -- 172.21.15.57:0/458899820 wait complete. 2024-04-13T05:21:20.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.551+0000 7f3503619740 20 client.16209 trim_cache size 0 max 16384 2024-04-13T05:21:20.554 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f34e00a3c30 msgr2=0x7f34e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f34e00a3c30 0x7f34e00a60f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f34ec005d90 tx=0x7f34ec005d20 comp rx=0 tx=0).stop 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 msgr2=0x7f34f8101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 0x7f34f8101f00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f34f000eaa0 tx=0x7f34f000ef60 comp rx=0 tx=0).stop 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 shutdown_connections 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f34e00a3c30 0x7f34e00a60f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34f8075fa0 0x7f34f81058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34f8109ea0 0x7f34f8101f00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 --2- 172.21.15.57:0/1279820544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f80749c0 0x7f34f8105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.552+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 >> 172.21.15.57:0/1279820544 conn(0x7f34f80fd5a0 msgr2=0x7f34f8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.553+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 shutdown_connections 2024-04-13T05:21:20.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.553+0000 7f3500600640 1 -- 172.21.15.57:0/1279820544 wait complete. 2024-04-13T05:21:20.575 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:21:20.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 -- 172.21.15.57:0/105600086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8078560 msgr2=0x7f42c8078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 --2- 172.21.15.57:0/105600086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8078560 0x7f42c8078e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009ca0 tx=0x7f42c002c100 comp rx=0 tx=0).stop 2024-04-13T05:21:20.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 -- 172.21.15.57:0/105600086 shutdown_connections 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 --2- 172.21.15.57:0/105600086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8078560 0x7f42c8078e00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 --2- 172.21.15.57:0/105600086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42c8077bc0 0x7f42c8078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 --2- 172.21.15.57:0/105600086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8076980 0x7f42c8076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.834+0000 7f42cca00640 1 -- 172.21.15.57:0/105600086 >> 172.21.15.57:0/105600086 conn(0x7f42c80ff7b0 msgr2=0x7f42c8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.835+0000 7f42cca00640 1 -- 172.21.15.57:0/105600086 shutdown_connections 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.835+0000 7f42cca00640 1 -- 172.21.15.57:0/105600086 wait complete. 2024-04-13T05:21:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 Processor -- start 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 -- start start 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 0x7f42c819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8077bc0 0x7f42c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42c8078560 0x7f42c81a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f42c8114180 con 0x7f42c8077bc0 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42c8114000 con 0x7f42c8076980 2024-04-13T05:21:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.836+0000 7f42cca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42c8114300 con 0x7f42c8078560 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.837+0000 7f42c5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8077bc0 0x7f42c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.837+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 0x7f42c819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.837+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 0x7f42c819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55740/0 (socket says 172.21.15.57:55740) 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.837+0000 7f42c5e00640 1 -- 172.21.15.57:0/917788790 learned_addr learned my addr 172.21.15.57:0/917788790 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.837+0000 7f42c6800640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42c8078560 0x7f42c81a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5400640 1 -- 172.21.15.57:0/917788790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42c8078560 msgr2=0x7f42c81a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5400640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42c8078560 0x7f42c81a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5400640 1 -- 172.21.15.57:0/917788790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 msgr2=0x7f42c819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5400640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 0x7f42c819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5400640 1 -- 172.21.15.57:0/917788790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c8103da0 con 0x7f42c8077bc0 2024-04-13T05:21:20.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5e00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 0x7f42c819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:20.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.838+0000 7f42c5400640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8077bc0 0x7f42c819ed60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9f0 tx=0x7f42b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.839+0000 7f42bea00640 1 -- 172.21.15.57:0/917788790 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b400ccc0 con 0x7f42c8077bc0 2024-04-13T05:21:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.839+0000 7f42bea00640 1 -- 172.21.15.57:0/917788790 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42b40108e0 con 0x7f42c8077bc0 2024-04-13T05:21:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.839+0000 7f42bea00640 1 -- 172.21.15.57:0/917788790 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4018880 con 0x7f42c8077bc0 2024-04-13T05:21:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.839+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c8104090 con 0x7f42c8077bc0 2024-04-13T05:21:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.839+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c8104590 con 0x7f42c8077bc0 2024-04-13T05:21:20.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.843+0000 7f42bea00640 1 -- 172.21.15.57:0/917788790 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f42b4010a90 con 0x7f42c8077bc0 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 Processor -- start 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 -- start start 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaaba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabac40 0x562dbaabf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabfdb0 0x562dbaac2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562dbaa20020 con 0x562dbaabfdb0 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562dba9c6950 con 0x562dbaaba250 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42cf984740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562dbaa1bfe0 con 0x562dbaabac40 2024-04-13T05:21:20.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.844+0000 7f42bea00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f42ac0a3d80 0x7f42ac0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaaba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabfdb0 0x562dbaac2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaaba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55744/0 (socket says 172.21.15.57:55744) 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 -- 172.21.15.57:0/3249850960 learned_addr learned my addr 172.21.15.57:0/3249850960 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c5e00640 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabac40 0x562dbaabf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 -- 172.21.15.57:0/3249850960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabac40 msgr2=0x562dbaabf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabac40 0x562dbaabf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 -- 172.21.15.57:0/3249850960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabfdb0 msgr2=0x562dbaac2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabfdb0 0x562dbaac2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c6800640 1 -- 172.21.15.57:0/3249850960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dba7323f0 con 0x562dbaaba250 2024-04-13T05:21:20.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.845+0000 7f42c5400640 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabfdb0 0x562dbaac2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:20.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.846+0000 7f42c6800640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f42ac0a3d80 0x7f42ac0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.846+0000 7f42bea00640 1 -- 172.21.15.57:0/917788790 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f42b40d0b00 con 0x7f42c8077bc0 2024-04-13T05:21:20.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.846+0000 7f42c6800640 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaaba670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42c0037040 tx=0x7f42c0009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585f133932535c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.846+0000 7f42b3000640 1 -- 172.21.15.57:0/3249850960 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0051070 con 0x562dbaaba250 2024-04-13T05:21:20.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.847+0000 7f42b3000640 1 -- 172.21.15.57:0/3249850960 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c0043050 con 0x562dbaaba250 2024-04-13T05:21:20.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.847+0000 7f42b3000640 1 -- 172.21.15.57:0/3249850960 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0039040 con 0x562dbaaba250 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.847+0000 7f42cf984740 1 -- 172.21.15.57:0/3249850960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 msgr2=0x562dbaaba670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.847+0000 7f42cf984740 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaaba670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42c0037040 tx=0x7f42c0009800 comp rx=0 tx=0).stop 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42c6800640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f42ac0a3d80 0x7f42ac0a6240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f42c004a000 tx=0x7f42c004b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 -- 172.21.15.57:0/3249850960 shutdown_connections 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabfdb0 0x562dbaac2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabac40 0x562dbaabf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 --2- 172.21.15.57:0/3249850960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaaba670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 -- 172.21.15.57:0/3249850960 >> 172.21.15.57:0/3249850960 conn(0x562dbaab57c0 msgr2=0x562dbaab5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 -- 172.21.15.57:0/3249850960 shutdown_connections 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 -- 172.21.15.57:0/3249850960 wait complete. 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.848+0000 7f42cf984740 1 -- start start 2024-04-13T05:21:20.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 Processor -- start 2024-04-13T05:21:20.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaaba250 0x562dbaad5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 0x562dbaad5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaadcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562dbaad3d70 con 0x562dbaaba250 2024-04-13T05:21:20.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562dba9d9c80 con 0x562dbaabac40 2024-04-13T05:21:20.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562dbaad3ef0 con 0x562dbaabfdb0 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42cf984740 10 client.0 _ll_register_callbacks cb 0x562dba9d9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.849+0000 7f42c5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 0x562dbaad5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaaba250 0x562dbaad5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 0x562dbaad5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55748/0 (socket says 172.21.15.57:55748) 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 -- 172.21.15.57:0/296703928 learned_addr learned my addr 172.21.15.57:0/296703928 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 -- 172.21.15.57:0/296703928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 msgr2=0x562dbaadcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaadcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 -- 172.21.15.57:0/296703928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaaba250 msgr2=0x562dbaad5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaaba250 0x562dbaad5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 -- 172.21.15.57:0/296703928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dba7fc7d0 con 0x562dbaabac40 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5e00640 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaaba250 0x562dbaad5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.850+0000 7f42c5400640 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 0x562dbaad5cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42b40a5120 tx=0x7f42b409c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4583c1ea84525b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42a7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:21:20.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42a7400640 1 -- 172.21.15.57:0/296703928 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40bbda0 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42a7400640 1 -- 172.21.15.57:0/296703928 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f42b4010430 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c0043050 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42a7400640 1 -- 172.21.15.57:0/296703928 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42b409d0e0 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42cf984740 10 client.35663 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.851+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562dbaae00e0 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42a7400640 1 -- 172.21.15.57:0/296703928 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c0043050 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42a7400640 1 -- 172.21.15.57:0/296703928 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f42b4098730 con 0x562dbaabac40 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42cf984740 10 client.35663 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42cf984740 10 client.35663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42cf984740 20 client.35663 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42cf984740 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbaae1740 0x562dbaae3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42cf984740 4 client.35663 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dba9bf750 con 0x562dbaae1740 2024-04-13T05:21:20.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.852+0000 7f42c5e00640 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbaae1740 0x562dbaae3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.853+0000 7f42c5e00640 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbaae1740 0x562dbaae3be0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f72fe69d76df11b7 server_cookie=cf186e065d60864c in_seq=0 out_seq=0 2024-04-13T05:21:20.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.853+0000 7f42a7400640 10 client.35663 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42a7400640 1 -- 172.21.15.57:0/296703928 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dba9bf750 con 0x562dbaae1740 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42a7400640 10 client.35663 handle_command_reply: tid=0 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 client.35663 shutdown 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 msgr2=0x562dbaad5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 0x562dbaad5cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f42b40a5120 tx=0x7f42b409c610 comp rx=0 tx=0).stop 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 shutdown_connections 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbaae1740 0x562dbaae3be0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaabac40 0x562dbaad5cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42a7400640 0 client.35663 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaadcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 --2- 172.21.15.57:0/296703928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaaba250 0x562dbaad5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.854+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 >> 172.21.15.57:0/296703928 conn(0x562dbaab57c0 msgr2=0x562dbaab4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.855+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 shutdown_connections 2024-04-13T05:21:20.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.855+0000 7f42cf984740 1 -- 172.21.15.57:0/296703928 wait complete. 2024-04-13T05:21:20.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.855+0000 7f42cf984740 20 client.35663 trim_cache size 0 max 16384 2024-04-13T05:21:20.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 Processor -- start 2024-04-13T05:21:20.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 -- start start 2024-04-13T05:21:20.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaabb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaab41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f42c80709b0 con 0x562dbaabac40 2024-04-13T05:21:20.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42c806e220 con 0x562dbaaba250 2024-04-13T05:21:20.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.878+0000 7f42cf984740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42c810c200 con 0x562dbaabfdb0 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaabb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaab41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaabb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55764/0 (socket says 172.21.15.57:55764) 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 -- 172.21.15.57:0/3916440420 learned_addr learned my addr 172.21.15.57:0/3916440420 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5400640 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 -- 172.21.15.57:0/3916440420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 msgr2=0x562dbaab41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaab41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 -- 172.21.15.57:0/3916440420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 msgr2=0x562dbaadccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.879+0000 7f42c5e00640 1 -- 172.21.15.57:0/3916440420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbaae4260 con 0x562dbaaba250 2024-04-13T05:21:20.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.880+0000 7f42c5e00640 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaabb910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005fd0 tx=0x7f42b8014cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a400eadfe07f6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.880+0000 7f42b2600640 1 -- 172.21.15.57:0/3916440420 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80142d0 con 0x562dbaaba250 2024-04-13T05:21:20.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.880+0000 7f42b2600640 1 -- 172.21.15.57:0/3916440420 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42b8011d50 con 0x562dbaaba250 2024-04-13T05:21:20.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.880+0000 7f42b2600640 1 -- 172.21.15.57:0/3916440420 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80047a0 con 0x562dbaaba250 2024-04-13T05:21:20.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 -- 172.21.15.57:0/3916440420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 msgr2=0x562dbaabb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaabb910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005fd0 tx=0x7f42b8014cd0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 -- 172.21.15.57:0/3916440420 shutdown_connections 2024-04-13T05:21:20.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaab41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 --2- 172.21.15.57:0/3916440420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaabb910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 -- 172.21.15.57:0/3916440420 >> 172.21.15.57:0/3916440420 conn(0x562dbaab57c0 msgr2=0x562dbaad4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 -- 172.21.15.57:0/3916440420 shutdown_connections 2024-04-13T05:21:20.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.881+0000 7f42cf984740 1 -- 172.21.15.57:0/3916440420 wait complete. 2024-04-13T05:21:20.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 -- start start 2024-04-13T05:21:20.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 Processor -- start 2024-04-13T05:21:20.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaadab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaadc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f42c806c6e0 con 0x562dbaabac40 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42c806db50 con 0x562dbaaba250 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42c806cdb0 con 0x562dbaabfdb0 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.882+0000 7f42cf984740 10 client.0 _ll_register_callbacks cb 0x7f42b409c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.883+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaadab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.883+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaadab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55772/0 (socket says 172.21.15.57:55772) 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.883+0000 7f42c5e00640 1 -- 172.21.15.57:0/694464163 learned_addr learned my addr 172.21.15.57:0/694464163 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.883+0000 7f42c5400640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5e00640 1 -- 172.21.15.57:0/694464163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 msgr2=0x562dbaadc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5e00640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaadc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5e00640 1 -- 172.21.15.57:0/694464163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 msgr2=0x562dbaadb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5e00640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5e00640 1 -- 172.21.15.57:0/694464163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbaadc670 con 0x562dbaaba250 2024-04-13T05:21:20.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5400640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:20.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42c5e00640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaadab30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b800b770 tx=0x7f42b800bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e6c52157978370 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:20.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42a7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:21:20.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.884+0000 7f42a7400640 1 -- 172.21.15.57:0/694464163 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80118a0 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.885+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b8011d50 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.885+0000 7f42a7400640 1 -- 172.21.15.57:0/694464163 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f42b8011a40 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.885+0000 7f42a7400640 1 -- 172.21.15.57:0/694464163 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42b8038050 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.885+0000 7f42cf984740 10 client.35669 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.885+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562dbab484e0 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42a7400640 1 -- 172.21.15.57:0/694464163 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42b8011d50 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42a7400640 1 -- 172.21.15.57:0/694464163 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f42b8031060 con 0x562dbaaba250 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42cf984740 10 client.35669 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42cf984740 10 client.35669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42cf984740 20 client.35669 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42cf984740 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbab48eb0 0x562dbab4b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42cf984740 4 client.35669 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562dbaa2da40 con 0x562dbab48eb0 2024-04-13T05:21:20.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.886+0000 7f42c5400640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbab48eb0 0x562dbab4b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:20.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.887+0000 7f42c5400640 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbab48eb0 0x562dbab4b2d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8adb4628584b5b2f server_cookie=4c5900dc1d2c2881 in_seq=0 out_seq=0 2024-04-13T05:21:20.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.887+0000 7f42a7400640 10 client.35669 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42a7400640 1 -- 172.21.15.57:0/694464163 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562dbaa2da40 con 0x562dbab48eb0 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42a7400640 10 client.35669 handle_command_reply: tid=0 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 client.35669 shutdown 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 msgr2=0x562dbaadab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaadab30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42b800b770 tx=0x7f42b800bcd0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 shutdown_connections 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x562dbab48eb0 0x562dbab4b2d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562dbaabfdb0 0x562dbaadc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562dbaabac40 0x562dbaadb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 --2- 172.21.15.57:0/694464163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562dbaaba250 0x562dbaadab30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 >> 172.21.15.57:0/694464163 conn(0x562dbaab57c0 msgr2=0x562dbaab8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42a7400640 0 client.35669 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 shutdown_connections 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.890+0000 7f42cf984740 1 -- 172.21.15.57:0/694464163 wait complete. 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.891+0000 7f42cf984740 20 client.35669 trim_cache size 0 max 16384 2024-04-13T05:21:20.894 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:21:20.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.891+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f42ac0a3d80 msgr2=0x7f42ac0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.891+0000 7f42cca00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f42ac0a3d80 0x7f42ac0a6240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f42c004a000 tx=0x7f42c004b000 comp rx=0 tx=0).stop 2024-04-13T05:21:20.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.891+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8077bc0 msgr2=0x7f42c819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:20.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.891+0000 7f42cca00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8077bc0 0x7f42c819ed60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9f0 tx=0x7f42b400eeb0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 shutdown_connections 2024-04-13T05:21:20.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f42ac0a3d80 0x7f42ac0a6240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42c8078560 0x7f42c81a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f42c8077bc0 0x7f42c819ed60 secure :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9f0 tx=0x7f42b400eeb0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 --2- 172.21.15.57:0/917788790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42c8076980 0x7f42c819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:20.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 >> 172.21.15.57:0/917788790 conn(0x7f42c80ff7b0 msgr2=0x7f42c81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:20.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 shutdown_connections 2024-04-13T05:21:20.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:20.892+0000 7f42cca00640 1 -- 172.21.15.57:0/917788790 wait complete. 2024-04-13T05:21:20.917 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 -- 172.21.15.57:0/257316952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c410b930 msgr2=0x7f35c4112700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 --2- 172.21.15.57:0/257316952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c410b930 0x7f35c4112700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009950 tx=0x7f35b002bc80 comp rx=0 tx=0).stop 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 -- 172.21.15.57:0/257316952 shutdown_connections 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 --2- 172.21.15.57:0/257316952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35c410bdb0 0x7f35c4114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 --2- 172.21.15.57:0/257316952 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c410b930 0x7f35c4112700 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 --2- 172.21.15.57:0/257316952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4108fc0 0x7f35c410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 -- 172.21.15.57:0/257316952 >> 172.21.15.57:0/257316952 conn(0x7f35c406f610 msgr2=0x7f35c4071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 -- 172.21.15.57:0/257316952 shutdown_connections 2024-04-13T05:21:21.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.184+0000 7f35caa00640 1 -- 172.21.15.57:0/257316952 wait complete. 2024-04-13T05:21:21.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.185+0000 7f35caa00640 1 Processor -- start 2024-04-13T05:21:21.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.186+0000 7f35caa00640 1 -- start start 2024-04-13T05:21:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35caa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 0x7f35c41a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35caa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35c410bdb0 0x7f35c41a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35caa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c41a8290 0x7f35c41aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35caa00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f35c4118420 con 0x7f35c4108fc0 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35caa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41182a0 con 0x7f35c41a8290 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35c9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 0x7f35c41a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35caa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35c41185a0 con 0x7f35c410bdb0 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c41a8290 0x7f35c41aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35c9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 0x7f35c41a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40528/0 (socket says 172.21.15.57:40528) 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c41a8290 0x7f35c41aa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55782/0 (socket says 172.21.15.57:55782) 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.187+0000 7f35c9600640 1 -- 172.21.15.57:0/3959378715 learned_addr learned my addr 172.21.15.57:0/3959378715 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:21.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.188+0000 7f35c8c00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35c410bdb0 0x7f35c41a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.188+0000 7f35c9600640 1 -- 172.21.15.57:0/3959378715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35c410bdb0 msgr2=0x7f35c41a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.188+0000 7f35c9600640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35c410bdb0 0x7f35c41a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.188+0000 7f35c9600640 1 -- 172.21.15.57:0/3959378715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c41a8290 msgr2=0x7f35c41aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.188+0000 7f35c9600640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c41a8290 0x7f35c41aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.188+0000 7f35c9600640 1 -- 172.21.15.57:0/3959378715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c41aad20 con 0x7f35c4108fc0 2024-04-13T05:21:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.189+0000 7f35c9600640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 0x7f35c41a2d80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00eab0 tx=0x7f35bc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.189+0000 7f35c2000640 1 -- 172.21.15.57:0/3959378715 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00ce40 con 0x7f35c4108fc0 2024-04-13T05:21:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.189+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c41aafb0 con 0x7f35c4108fc0 2024-04-13T05:21:21.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.190+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c41ab4c0 con 0x7f35c4108fc0 2024-04-13T05:21:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.190+0000 7f35c2000640 1 -- 172.21.15.57:0/3959378715 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35bc011920 con 0x7f35c4108fc0 2024-04-13T05:21:21.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.190+0000 7f35c2000640 1 -- 172.21.15.57:0/3959378715 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc010a00 con 0x7f35c4108fc0 2024-04-13T05:21:21.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.194+0000 7f35c2000640 1 -- 172.21.15.57:0/3959378715 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f35bc011ad0 con 0x7f35c4108fc0 2024-04-13T05:21:21.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.197+0000 7f35c2000640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f35ac0a3d00 0x7f35ac0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.197+0000 7f35cee58740 1 Processor -- start 2024-04-13T05:21:21.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.197+0000 7f35c2000640 1 -- 172.21.15.57:0/3959378715 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f35bc0d0640 con 0x7f35c4108fc0 2024-04-13T05:21:21.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.197+0000 7f35c8c00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f35ac0a3d00 0x7f35ac0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.197+0000 7f35cee58740 1 -- start start 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35cee58740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34e440 0x561d7c34e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35cee58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c34ee30 0x561d7c353930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35cee58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 0x561d7c356430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35cee58740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561d7c2b4020 con 0x561d7c353fa0 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35cee58740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d7c25a950 con 0x561d7c34e440 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35cee58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561d7c2affe0 con 0x561d7c34ee30 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35ca000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 0x561d7c356430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35c8c00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f35ac0a3d00 0x7f35ac0a61c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f35b0036040 tx=0x7f35b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35ca000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 0x561d7c356430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40544/0 (socket says 172.21.15.57:40544) 2024-04-13T05:21:21.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.198+0000 7f35ca000640 1 -- 172.21.15.57:0/239239253 learned_addr learned my addr 172.21.15.57:0/239239253 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:21.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.199+0000 7f35c8c00640 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c34ee30 0x561d7c353930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.199+0000 7f35ca000640 1 -- 172.21.15.57:0/239239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c34ee30 msgr2=0x561d7c353930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.199+0000 7f35ca000640 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c34ee30 0x561d7c353930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.199+0000 7f35ca000640 1 -- 172.21.15.57:0/239239253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34e440 msgr2=0x561d7c34e860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:21:21.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.199+0000 7f35ca000640 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34e440 0x561d7c34e860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.199+0000 7f35ca000640 1 -- 172.21.15.57:0/239239253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7bfc63f0 con 0x561d7c353fa0 2024-04-13T05:21:21.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.200+0000 7f35ca000640 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 0x561d7c356430 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f35b800bcc0 tx=0x7f35b800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98001fb80046baa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:21.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.200+0000 7f35b6600640 1 -- 172.21.15.57:0/239239253 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8015070 con 0x561d7c353fa0 2024-04-13T05:21:21.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.201+0000 7f35b6600640 1 -- 172.21.15.57:0/239239253 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b801a910 con 0x561d7c353fa0 2024-04-13T05:21:21.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.201+0000 7f35b6600640 1 -- 172.21.15.57:0/239239253 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8011810 con 0x561d7c353fa0 2024-04-13T05:21:21.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.201+0000 7f35cee58740 1 -- 172.21.15.57:0/239239253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 msgr2=0x561d7c356430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.201+0000 7f35cee58740 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 0x561d7c356430 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f35b800bcc0 tx=0x7f35b800f730 comp rx=0 tx=0).stop 2024-04-13T05:21:21.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 -- 172.21.15.57:0/239239253 shutdown_connections 2024-04-13T05:21:21.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c353fa0 0x561d7c356430 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c34ee30 0x561d7c353930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 --2- 172.21.15.57:0/239239253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34e440 0x561d7c34e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 -- 172.21.15.57:0/239239253 >> 172.21.15.57:0/239239253 conn(0x561d7c3499b0 msgr2=0x561d7c349db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:21.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 -- 172.21.15.57:0/239239253 shutdown_connections 2024-04-13T05:21:21.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 -- 172.21.15.57:0/239239253 wait complete. 2024-04-13T05:21:21.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.202+0000 7f35cee58740 1 -- start start 2024-04-13T05:21:21.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 Processor -- start 2024-04-13T05:21:21.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c358970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 0x561d7c358eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c35fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561d7c356f50 con 0x561d7c34e440 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d7c26dc80 con 0x561d7c34ee30 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561d7c3570d0 con 0x561d7c353fa0 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.203+0000 7f35cee58740 10 client.0 _ll_register_callbacks cb 0x561d7c26d670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 0x561d7c358eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c358970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 0x561d7c358eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55796/0 (socket says 172.21.15.57:55796) 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 -- 172.21.15.57:0/143317611 learned_addr learned my addr 172.21.15.57:0/143317611 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c9600640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c358970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40550/0 (socket says 172.21.15.57:40550) 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 -- 172.21.15.57:0/143317611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 msgr2=0x561d7c35fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c35fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 -- 172.21.15.57:0/143317611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 msgr2=0x561d7c358970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c358970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c8c00640 1 -- 172.21.15.57:0/143317611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7c0907d0 con 0x561d7c34ee30 2024-04-13T05:21:21.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.204+0000 7f35c9600640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c358970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:21.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.205+0000 7f35c8c00640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 0x561d7c358eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009230 tx=0x7f35b0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f87332f0c6bcc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:21.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.205+0000 7f35aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.205+0000 7f35aaa00640 1 -- 172.21.15.57:0/143317611 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0050030 con 0x561d7c34ee30 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.205+0000 7f35aaa00640 1 -- 172.21.15.57:0/143317611 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f35b003a040 con 0x561d7c34ee30 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.206+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b801a910 con 0x561d7c34ee30 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.206+0000 7f35aaa00640 1 -- 172.21.15.57:0/143317611 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35b0005650 con 0x561d7c34ee30 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.206+0000 7f35cee58740 10 client.35678 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.206+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d7c363300 con 0x561d7c34ee30 2024-04-13T05:21:21.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35aaa00640 1 -- 172.21.15.57:0/143317611 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b801a910 con 0x561d7c34ee30 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35aaa00640 1 -- 172.21.15.57:0/143317611 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f35b0003a00 con 0x561d7c34ee30 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35cee58740 10 client.35678 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35cee58740 10 client.35678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35cee58740 20 client.35678 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35cee58740 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c364aa0 0x561d7c366ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35cee58740 4 client.35678 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.207+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d7c253750 con 0x561d7c364aa0 2024-04-13T05:21:21.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.208+0000 7f35c9600640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c364aa0 0x561d7c366ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.209+0000 7f35c9600640 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c364aa0 0x561d7c366ee0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0e2a8e1bdff998 server_cookie=ae0930e982b25fb4 in_seq=0 out_seq=0 2024-04-13T05:21:21.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.209+0000 7f35aaa00640 10 client.35678 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.210+0000 7f35aaa00640 1 -- 172.21.15.57:0/143317611 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d7c253750 con 0x561d7c364aa0 2024-04-13T05:21:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.210+0000 7f35aaa00640 10 client.35678 handle_command_reply: tid=0 2024-04-13T05:21:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.210+0000 7f35cee58740 1 client.35678 shutdown 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.210+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 msgr2=0x561d7c358eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.210+0000 7f35cee58740 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 0x561d7c358eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009230 tx=0x7f35b0009260 comp rx=0 tx=0).stop 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 shutdown_connections 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c364aa0 0x561d7c366ee0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c34ee30 0x561d7c358eb0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009230 tx=0x7f35b0009260 comp rx=0 tx=0).stop 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c35fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 --2- 172.21.15.57:0/143317611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c358970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 >> 172.21.15.57:0/143317611 conn(0x561d7c3499b0 msgr2=0x561d7c348cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:21.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35aaa00640 0 client.35678 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:21.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 shutdown_connections 2024-04-13T05:21:21.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 1 -- 172.21.15.57:0/143317611 wait complete. 2024-04-13T05:21:21.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.211+0000 7f35cee58740 20 client.35678 trim_cache size 0 max 16384 2024-04-13T05:21:21.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 Processor -- start 2024-04-13T05:21:21.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 -- start start 2024-04-13T05:21:21.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c3499b0 0x561d7c357f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c348440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4012540 con 0x561d7c34e440 2024-04-13T05:21:21.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35a400fdb0 con 0x561d7c3499b0 2024-04-13T05:21:21.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.234+0000 7f35cee58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35a400dc00 con 0x561d7c353fa0 2024-04-13T05:21:21.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40562/0 (socket says 172.21.15.57:40562) 2024-04-13T05:21:21.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 -- 172.21.15.57:0/171421200 learned_addr learned my addr 172.21.15.57:0/171421200 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35ca000640 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c348440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c9600640 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c3499b0 0x561d7c357f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 -- 172.21.15.57:0/171421200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 msgr2=0x561d7c348440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c348440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 -- 172.21.15.57:0/171421200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c3499b0 msgr2=0x561d7c357f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c3499b0 0x561d7c357f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35c8c00640 1 -- 172.21.15.57:0/171421200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7c34a720 con 0x561d7c34e440 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.235+0000 7f35ca000640 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c348440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.236+0000 7f35c8c00640 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35feb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f35b0041550 tx=0x7f35b002c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9df514f813ffbf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.236+0000 7f35b5c00640 1 -- 172.21.15.57:0/171421200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0050070 con 0x561d7c34e440 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.236+0000 7f35b5c00640 1 -- 172.21.15.57:0/171421200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b0063030 con 0x561d7c34e440 2024-04-13T05:21:21.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.236+0000 7f35b5c00640 1 -- 172.21.15.57:0/171421200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0003a90 con 0x561d7c34e440 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 -- 172.21.15.57:0/171421200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 msgr2=0x561d7c35feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35feb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f35b0041550 tx=0x7f35b002c380 comp rx=0 tx=0).stop 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 -- 172.21.15.57:0/171421200 shutdown_connections 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c353fa0 0x561d7c348440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35feb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 --2- 172.21.15.57:0/171421200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c3499b0 0x561d7c357f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 -- 172.21.15.57:0/171421200 >> 172.21.15.57:0/171421200 conn(0x561d7c357b80 msgr2=0x561d7c358970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:21.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 -- 172.21.15.57:0/171421200 shutdown_connections 2024-04-13T05:21:21.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.237+0000 7f35cee58740 1 -- 172.21.15.57:0/171421200 wait complete. 2024-04-13T05:21:21.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.238+0000 7f35cee58740 1 -- start start 2024-04-13T05:21:21.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.238+0000 7f35cee58740 1 Processor -- start 2024-04-13T05:21:21.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.239+0000 7f35cee58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c3499b0 0x561d7c35d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35cee58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35cee58740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c353fa0 0x561d7c35ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35cee58740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f35a400e270 con 0x561d7c34e440 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35cee58740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35a400f6e0 con 0x561d7c353fa0 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35cee58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35a400e940 con 0x561d7c3499b0 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35cee58740 10 client.0 _ll_register_callbacks cb 0x7f35b0004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35c8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40576/0 (socket says 172.21.15.57:40576) 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35c8c00640 1 -- 172.21.15.57:0/1342417174 learned_addr learned my addr 172.21.15.57:0/1342417174 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35c9600640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c3499b0 0x561d7c35d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.240+0000 7f35ca000640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c353fa0 0x561d7c35ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c8c00640 1 -- 172.21.15.57:0/1342417174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c3499b0 msgr2=0x561d7c35d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c8c00640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c3499b0 0x561d7c35d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c8c00640 1 -- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c353fa0 msgr2=0x561d7c35ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c8c00640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c353fa0 0x561d7c35ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c8c00640 1 -- 172.21.15.57:0/1342417174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7c35f040 con 0x561d7c34e440 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c9600640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c3499b0 0x561d7c35d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35c8c00640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35da90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f35b0003f40 tx=0x7f35b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd96bc648ffd45ec server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.241+0000 7f35aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:21.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.242+0000 7f35aaa00640 1 -- 172.21.15.57:0/1342417174 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b006d060 con 0x561d7c34e440 2024-04-13T05:21:21.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.242+0000 7f35aaa00640 1 -- 172.21.15.57:0/1342417174 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f35b0004ad0 con 0x561d7c34e440 2024-04-13T05:21:21.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.242+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b0063030 con 0x561d7c34e440 2024-04-13T05:21:21.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.242+0000 7f35aaa00640 1 -- 172.21.15.57:0/1342417174 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35b0050070 con 0x561d7c34e440 2024-04-13T05:21:21.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.243+0000 7f35cee58740 10 client.16245 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:21.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.243+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d7c3d3870 con 0x561d7c34e440 2024-04-13T05:21:21.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.243+0000 7f35aaa00640 1 -- 172.21.15.57:0/1342417174 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b0063030 con 0x561d7c34e440 2024-04-13T05:21:21.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35aaa00640 1 -- 172.21.15.57:0/1342417174 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f35b004d450 con 0x561d7c34e440 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35cee58740 10 client.16245 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35cee58740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35cee58740 20 client.16245 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35cee58740 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c3d4090 0x561d7c3d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35cee58740 4 client.16245 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.244+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d7c358760 con 0x561d7c3d4090 2024-04-13T05:21:21.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.245+0000 7f35c9600640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c3d4090 0x561d7c3d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:21.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.246+0000 7f35c9600640 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c3d4090 0x561d7c3d64b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568fcae69a65f984 server_cookie=3ff12c2a655148ac in_seq=0 out_seq=0 2024-04-13T05:21:21.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.246+0000 7f35aaa00640 10 client.16245 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:21.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35aaa00640 1 -- 172.21.15.57:0/1342417174 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d7c358760 con 0x561d7c3d4090 2024-04-13T05:21:21.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35aaa00640 10 client.16245 handle_command_reply: tid=0 2024-04-13T05:21:21.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35cee58740 1 client.16245 shutdown 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 msgr2=0x561d7c35da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35cee58740 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35da90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f35b0003f40 tx=0x7f35b0005fd0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 shutdown_connections 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.247+0000 7f35cee58740 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x561d7c3d4090 0x561d7c3d64b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d7c353fa0 0x561d7c35ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d7c34e440 0x561d7c35da90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 1 --2- 172.21.15.57:0/1342417174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d7c3499b0 0x561d7c35d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 >> 172.21.15.57:0/1342417174 conn(0x561d7c357b80 msgr2=0x561d7c34cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:21.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35aaa00640 0 client.16245 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:21.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 shutdown_connections 2024-04-13T05:21:21.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 1 -- 172.21.15.57:0/1342417174 wait complete. 2024-04-13T05:21:21.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.248+0000 7f35cee58740 20 client.16245 trim_cache size 0 max 16384 2024-04-13T05:21:21.251 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f35ac0a3d00 msgr2=0x7f35ac0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f35ac0a3d00 0x7f35ac0a61c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f35b0036040 tx=0x7f35b0000bc0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 msgr2=0x7f35c41a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 0x7f35c41a2d80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00eab0 tx=0x7f35bc00ef70 comp rx=0 tx=0).stop 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 shutdown_connections 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f35ac0a3d00 0x7f35ac0a61c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c41a8290 0x7f35c41aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35c410bdb0 0x7f35c41a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 --2- 172.21.15.57:0/3959378715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35c4108fc0 0x7f35c41a2d80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:21.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.249+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 >> 172.21.15.57:0/3959378715 conn(0x7f35c406f610 msgr2=0x7f35c4070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:21.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.250+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 shutdown_connections 2024-04-13T05:21:21.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:21.250+0000 7f35caa00640 1 -- 172.21.15.57:0/3959378715 wait complete. 2024-04-13T05:21:21.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:20 smithi083 ceph-mon[35133]: pgmap v568: 129 pgs: 129 active+clean; 121 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-04-13T05:21:21.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:20 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/626846308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:21:21.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:20 smithi057 ceph-mon[32331]: pgmap v568: 129 pgs: 129 active+clean; 121 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-04-13T05:21:21.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:20 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/626846308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:21:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:20 smithi189 ceph-mon[35531]: pgmap v568: 129 pgs: 129 active+clean; 121 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-04-13T05:21:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:20 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/626846308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:21:22.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:21:22.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:21:22.274 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:21:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 -- 172.21.15.57:0/894514182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c1007a0 msgr2=0x7fd78c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 --2- 172.21.15.57:0/894514182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c1007a0 0x7fd78c100bc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd780009ca0 tx=0x7fd78002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 -- 172.21.15.57:0/894514182 shutdown_connections 2024-04-13T05:21:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 --2- 172.21.15.57:0/894514182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c101100 0x7fd78c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 --2- 172.21.15.57:0/894514182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78c101aa0 0x7fd78c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 --2- 172.21.15.57:0/894514182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c1007a0 0x7fd78c100bc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.529+0000 7fd792a00640 1 -- 172.21.15.57:0/894514182 >> 172.21.15.57:0/894514182 conn(0x7fd78c0fc360 msgr2=0x7fd78c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.530+0000 7fd792a00640 1 -- 172.21.15.57:0/894514182 shutdown_connections 2024-04-13T05:21:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.530+0000 7fd792a00640 1 -- 172.21.15.57:0/894514182 wait complete. 2024-04-13T05:21:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.531+0000 7fd792a00640 1 Processor -- start 2024-04-13T05:21:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.531+0000 7fd792a00640 1 -- start start 2024-04-13T05:21:22.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.531+0000 7fd792a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c101100 0x7fd78c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78c101aa0 0x7fd78c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 0x7fd78c1a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10fc60 con 0x7fd78c101100 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10fae0 con 0x7fd78c1a3ec0 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c10fde0 con 0x7fd78c101aa0 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd791600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c101100 0x7fd78c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 0x7fd78c1a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 0x7fd78c1a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55842/0 (socket says 172.21.15.57:55842) 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd792000640 1 -- 172.21.15.57:0/4027106093 learned_addr learned my addr 172.21.15.57:0/4027106093 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.532+0000 7fd790c00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78c101aa0 0x7fd78c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd791600640 1 -- 172.21.15.57:0/4027106093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78c101aa0 msgr2=0x7fd78c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd791600640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78c101aa0 0x7fd78c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd791600640 1 -- 172.21.15.57:0/4027106093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 msgr2=0x7fd78c1a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd791600640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 0x7fd78c1a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd791600640 1 -- 172.21.15.57:0/4027106093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c1a6920 con 0x7fd78c101100 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd792000640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 0x7fd78c1a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.533+0000 7fd791600640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c101100 0x7fd78c19e920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd780008000 tx=0x7fd7800328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.534+0000 7fd78a000640 1 -- 172.21.15.57:0/4027106093 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780039040 con 0x7fd78c101100 2024-04-13T05:21:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.534+0000 7fd78a000640 1 -- 172.21.15.57:0/4027106093 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd780003970 con 0x7fd78c101100 2024-04-13T05:21:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.534+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c1a6bb0 con 0x7fd78c101100 2024-04-13T05:21:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.534+0000 7fd78a000640 1 -- 172.21.15.57:0/4027106093 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780004330 con 0x7fd78c101100 2024-04-13T05:21:22.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.534+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c1a7070 con 0x7fd78c101100 2024-04-13T05:21:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.539+0000 7fd78a000640 1 -- 172.21.15.57:0/4027106093 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd780044030 con 0x7fd78c101100 2024-04-13T05:21:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.539+0000 7fd796fdb740 1 Processor -- start 2024-04-13T05:21:22.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 -- start start 2024-04-13T05:21:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8df420 0x563a8c8df840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563a8c845bc0 con 0x563a8c8dfe10 2024-04-13T05:21:22.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a8c7ec660 con 0x563a8c8e4f80 2024-04-13T05:21:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd796fdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a8c841b80 con 0x563a8c8df420 2024-04-13T05:21:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.540+0000 7fd791600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8e7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55846/0 (socket says 172.21.15.57:55846) 2024-04-13T05:21:22.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 -- 172.21.15.57:0/691056343 learned_addr learned my addr 172.21.15.57:0/691056343 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd792000640 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8df420 0x563a8c8df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd78a000640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd7740a3d00 0x7fd7740a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 -- 172.21.15.57:0/691056343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8df420 msgr2=0x563a8c8df840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8df420 0x563a8c8df840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 -- 172.21.15.57:0/691056343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 msgr2=0x563a8c8e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd78a000640 1 -- 172.21.15.57:0/4027106093 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd7800f1d20 con 0x7fd78c101100 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd790c00640 1 -- 172.21.15.57:0/691056343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8c80f220 con 0x563a8c8e4f80 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.541+0000 7fd791600640 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.542+0000 7fd790c00640 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8e7410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd77800b7f0 tx=0x7fd77800bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50d7dc01d424ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.542+0000 7fd792000640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd7740a3d00 0x7fd7740a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.542+0000 7fd77e600640 1 -- 172.21.15.57:0/691056343 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800d950 con 0x563a8c8e4f80 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.542+0000 7fd77e600640 1 -- 172.21.15.57:0/691056343 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd778018070 con 0x563a8c8e4f80 2024-04-13T05:21:22.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.542+0000 7fd77e600640 1 -- 172.21.15.57:0/691056343 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7780123e0 con 0x563a8c8e4f80 2024-04-13T05:21:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 -- 172.21.15.57:0/691056343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 msgr2=0x563a8c8e7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8e7410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd77800b7f0 tx=0x7fd77800bcb0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 -- 172.21.15.57:0/691056343 shutdown_connections 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8e7410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 --2- 172.21.15.57:0/691056343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8df420 0x563a8c8df840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.543+0000 7fd796fdb740 1 -- 172.21.15.57:0/691056343 >> 172.21.15.57:0/691056343 conn(0x563a8c8da990 msgr2=0x563a8c8dad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.544+0000 7fd792000640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd7740a3d00 0x7fd7740a61c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x563a8c8e0670 tx=0x7fd7840046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.544+0000 7fd796fdb740 1 -- 172.21.15.57:0/691056343 shutdown_connections 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.544+0000 7fd796fdb740 1 -- 172.21.15.57:0/691056343 wait complete. 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.544+0000 7fd796fdb740 1 -- start start 2024-04-13T05:21:22.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.545+0000 7fd796fdb740 1 Processor -- start 2024-04-13T05:21:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.546+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 0x563a8c8e9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.546+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.546+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.546+0000 7fd796fdb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563a8c8e7f30 con 0x563a8c8df420 2024-04-13T05:21:22.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd796fdb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a8c7ffa30 con 0x563a8c8e4f80 2024-04-13T05:21:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 0x563a8c8e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd790c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd796fdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a8c8e80b0 con 0x563a8c8dfe10 2024-04-13T05:21:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 0x563a8c8e9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40610/0 (socket says 172.21.15.57:40610) 2024-04-13T05:21:22.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 -- 172.21.15.57:0/3093708826 learned_addr learned my addr 172.21.15.57:0/3093708826 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd792000640 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd796fdb740 10 client.0 _ll_register_callbacks cb 0x563a8c557c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 -- 172.21.15.57:0/3093708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8dfe10 msgr2=0x563a8c8e9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 -- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 msgr2=0x563a8c8f0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.547+0000 7fd791600640 1 -- 172.21.15.57:0/3093708826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8c6207e0 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.548+0000 7fd791600640 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 0x563a8c8e9940 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd7800ef420 tx=0x7fd7800ef740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b28517c1b5ab169 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.548+0000 7fd772a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.548+0000 7fd772a00640 1 -- 172.21.15.57:0/3093708826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800ba490 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.548+0000 7fd772a00640 1 -- 172.21.15.57:0/3093708826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd7800de0b0 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.549+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd778018070 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.549+0000 7fd772a00640 1 -- 172.21.15.57:0/3093708826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7800098d0 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.549+0000 7fd796fdb740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.549+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563a8c680f70 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd772a00640 1 -- 172.21.15.57:0/3093708826 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd778018070 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd772a00640 1 -- 172.21.15.57:0/3093708826 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fd78003c070 con 0x563a8c8df420 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd796fdb740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd796fdb740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:22.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd796fdb740 20 client.16257 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd796fdb740 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c8f5780 0x563a8c8f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd796fdb740 4 client.16257 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.550+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a8c8535e0 con 0x563a8c8f5780 2024-04-13T05:21:22.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.551+0000 7fd790c00640 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c8f5780 0x563a8c8f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.553+0000 7fd790c00640 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c8f5780 0x563a8c8f7bf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65ef35a4dfd7683 server_cookie=1084cd25bb5dcfef in_seq=0 out_seq=0 2024-04-13T05:21:22.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.553+0000 7fd772a00640 10 client.16257 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.554+0000 7fd772a00640 1 -- 172.21.15.57:0/3093708826 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a8c8535e0 con 0x563a8c8f5780 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.554+0000 7fd772a00640 10 client.16257 handle_command_reply: tid=0 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.554+0000 7fd796fdb740 1 client.16257 shutdown 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 msgr2=0x563a8c8e9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 0x563a8c8e9940 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd7800ef420 tx=0x7fd7800ef740 comp rx=0 tx=0).stop 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 shutdown_connections 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c8f5780 0x563a8c8f7bf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 --2- 172.21.15.57:0/3093708826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8df420 0x563a8c8e9940 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 >> 172.21.15.57:0/3093708826 conn(0x563a8c8da990 msgr2=0x563a8c8d9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd772a00640 0 client.16257 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 shutdown_connections 2024-04-13T05:21:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 1 -- 172.21.15.57:0/3093708826 wait complete. 2024-04-13T05:21:22.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.555+0000 7fd796fdb740 20 client.16257 trim_cache size 0 max 16384 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.584+0000 7fd796fdb740 1 Processor -- start 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.584+0000 7fd796fdb740 1 -- start start 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8de300 0x563a8c8fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd796fdb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1b3da0 con 0x563a8c8e4f80 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd796fdb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1b1610 con 0x563a8c8dfe10 2024-04-13T05:21:22.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd796fdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1af460 con 0x563a8c8de300 2024-04-13T05:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd791600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd791600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55864/0 (socket says 172.21.15.57:55864) 2024-04-13T05:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd790c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd790c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40622/0 (socket says 172.21.15.57:40622) 2024-04-13T05:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd790c00640 1 -- 172.21.15.57:0/1961228439 learned_addr learned my addr 172.21.15.57:0/1961228439 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.585+0000 7fd792000640 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8de300 0x563a8c8fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.586+0000 7fd790c00640 1 -- 172.21.15.57:0/1961228439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8de300 msgr2=0x563a8c8fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.586+0000 7fd790c00640 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8de300 0x563a8c8fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.586+0000 7fd790c00640 1 -- 172.21.15.57:0/1961228439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8dfe10 msgr2=0x563a8c8e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.586+0000 7fd790c00640 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.586+0000 7fd790c00640 1 -- 172.21.15.57:0/1961228439 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8c8f7f60 con 0x563a8c8e4f80 2024-04-13T05:21:22.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.586+0000 7fd790c00640 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0e80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd778004810 tx=0x7fd778008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9fd9c7b0a816f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.587+0000 7fd77dc00640 1 -- 172.21.15.57:0/1961228439 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778027bd0 con 0x563a8c8e4f80 2024-04-13T05:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.587+0000 7fd77dc00640 1 -- 172.21.15.57:0/1961228439 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77802c070 con 0x563a8c8e4f80 2024-04-13T05:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.587+0000 7fd77dc00640 1 -- 172.21.15.57:0/1961228439 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778027620 con 0x563a8c8e4f80 2024-04-13T05:21:22.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 -- 172.21.15.57:0/1961228439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 msgr2=0x563a8c8f0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0e80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd778004810 tx=0x7fd778008040 comp rx=0 tx=0).stop 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 -- 172.21.15.57:0/1961228439 shutdown_connections 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8de300 0x563a8c8fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8e4f80 0x563a8c8f0e80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 --2- 172.21.15.57:0/1961228439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8dfe10 0x563a8c8e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 -- 172.21.15.57:0/1961228439 >> 172.21.15.57:0/1961228439 conn(0x563a8c8da990 msgr2=0x563a8c8ddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 -- 172.21.15.57:0/1961228439 shutdown_connections 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.588+0000 7fd796fdb740 1 -- 172.21.15.57:0/1961228439 wait complete. 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 -- start start 2024-04-13T05:21:22.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 Processor -- start 2024-04-13T05:21:22.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8de300 0x563a8c8f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8f1040 0x563a8c8ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1afad0 con 0x563a8c8dfe10 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1b0f40 con 0x563a8c8de300 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1b01a0 con 0x563a8c8f1040 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.589+0000 7fd796fdb740 10 client.0 _ll_register_callbacks cb 0x7fd780009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40628/0 (socket says 172.21.15.57:40628) 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 -- 172.21.15.57:0/2131600542 learned_addr learned my addr 172.21.15.57:0/2131600542 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd792000640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8f1040 0x563a8c8ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd791600640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8de300 0x563a8c8f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 -- 172.21.15.57:0/2131600542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8f1040 msgr2=0x563a8c8ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8f1040 0x563a8c8ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 -- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8de300 msgr2=0x563a8c8f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8de300 0x563a8c8f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.590+0000 7fd790c00640 1 -- 172.21.15.57:0/2131600542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8c518170 con 0x563a8c8dfe10 2024-04-13T05:21:22.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.591+0000 7fd790c00640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8ef0c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd778008010 tx=0x7fd778006bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71096476fb21ab97 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.591+0000 7fd772a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.591+0000 7fd772a00640 1 -- 172.21.15.57:0/2131600542 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778027bd0 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.591+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77802c070 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.592+0000 7fd772a00640 1 -- 172.21.15.57:0/2131600542 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fd77800db50 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.592+0000 7fd772a00640 1 -- 172.21.15.57:0/2131600542 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd77803f050 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.592+0000 7fd796fdb740 10 client.16263 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.592+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563a8c800be0 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.592+0000 7fd772a00640 1 -- 172.21.15.57:0/2131600542 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77802c070 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.592+0000 7fd772a00640 1 -- 172.21.15.57:0/2131600542 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fd778013ce0 con 0x563a8c8dfe10 2024-04-13T05:21:22.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.593+0000 7fd796fdb740 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.593+0000 7fd796fdb740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.593+0000 7fd796fdb740 20 client.16263 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.593+0000 7fd796fdb740 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c96e3c0 0x563a8c9707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.593+0000 7fd796fdb740 4 client.16263 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.593+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a8c84d450 con 0x563a8c96e3c0 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.594+0000 7fd791600640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c96e3c0 0x563a8c9707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.594+0000 7fd791600640 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c96e3c0 0x563a8c9707e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d13f704df651c8 server_cookie=2d0cbb3d097f0824 in_seq=0 out_seq=0 2024-04-13T05:21:22.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.594+0000 7fd772a00640 10 client.16263 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.597+0000 7fd772a00640 1 -- 172.21.15.57:0/2131600542 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a8c84d450 con 0x563a8c96e3c0 2024-04-13T05:21:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.597+0000 7fd772a00640 10 client.16263 handle_command_reply: tid=0 2024-04-13T05:21:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.597+0000 7fd796fdb740 1 client.16263 shutdown 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.597+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 msgr2=0x563a8c8ef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.597+0000 7fd796fdb740 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8ef0c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd778008010 tx=0x7fd778006bc0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 shutdown_connections 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563a8c96e3c0 0x563a8c9707e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a8c8f1040 0x563a8c8ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563a8c8dfe10 0x563a8c8ef0c0 secure :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd778008010 tx=0x7fd778006bc0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd772a00640 0 client.16263 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 --2- 172.21.15.57:0/2131600542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a8c8de300 0x563a8c8f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 >> 172.21.15.57:0/2131600542 conn(0x563a8c8da990 msgr2=0x563a8c8d9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 shutdown_connections 2024-04-13T05:21:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.598+0000 7fd796fdb740 1 -- 172.21.15.57:0/2131600542 wait complete. 2024-04-13T05:21:22.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.599+0000 7fd796fdb740 20 client.16263 trim_cache size 0 max 16384 2024-04-13T05:21:22.602 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:21:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:21:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "aeab05bc-3faa-4663-834d-1e3a5d76ced6", 2024-04-13T05:21:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:21:22.602 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.599+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd7740a3d00 msgr2=0x7fd7740a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.599+0000 7fd792a00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd7740a3d00 0x7fd7740a61c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x563a8c8e0670 tx=0x7fd7840046c0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.599+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c101100 msgr2=0x7fd78c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.599+0000 7fd792a00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c101100 0x7fd78c19e920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd780008000 tx=0x7fd7800328a0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 shutdown_connections 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fd7740a3d00 0x7fd7740a61c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd78c1a3ec0 0x7fd78c1a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78c101aa0 0x7fd78c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 --2- 172.21.15.57:0/4027106093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd78c101100 0x7fd78c19e920 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 >> 172.21.15.57:0/4027106093 conn(0x7fd78c0fc360 msgr2=0x7fd78c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 shutdown_connections 2024-04-13T05:21:22.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.600+0000 7fd792a00640 1 -- 172.21.15.57:0/4027106093 wait complete. 2024-04-13T05:21:22.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:21:22.624 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:21:22.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 -- 172.21.15.57:0/3196800128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 msgr2=0x7f1fb410a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3196800128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb410a450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009be0 tx=0x7f1fac02bef0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 -- 172.21.15.57:0/3196800128 shutdown_connections 2024-04-13T05:21:22.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3196800128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb410a450 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3196800128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb4103200 0x7f1fb4103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3196800128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 0x7f1fb4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.905+0000 7f1fbba00640 1 -- 172.21.15.57:0/3196800128 >> 172.21.15.57:0/3196800128 conn(0x7f1fb40fd5a0 msgr2=0x7f1fb40ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.906+0000 7f1fbba00640 1 -- 172.21.15.57:0/3196800128 shutdown_connections 2024-04-13T05:21:22.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.906+0000 7f1fbba00640 1 -- 172.21.15.57:0/3196800128 wait complete. 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 Processor -- start 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 -- start start 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 0x7f1fb419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb4103200 0x7f1fb419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb410dba0 con 0x7f1fb4103bc0 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb410da20 con 0x7f1fb4102010 2024-04-13T05:21:22.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.907+0000 7f1fbba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb410dd20 con 0x7f1fb4103200 2024-04-13T05:21:22.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb41a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40654/0 (socket says 172.21.15.57:40654) 2024-04-13T05:21:22.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9a00640 1 -- 172.21.15.57:0/3100424143 learned_addr learned my addr 172.21.15.57:0/3100424143 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 0x7f1fb419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb4103200 0x7f1fb419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9000640 1 -- 172.21.15.57:0/3100424143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb4103200 msgr2=0x7f1fb419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb4103200 0x7f1fb419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9000640 1 -- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 msgr2=0x7f1fb41a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb41a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9000640 1 -- 172.21.15.57:0/3100424143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41a2080 con 0x7f1fb4102010 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.908+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.909+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 0x7f1fb419a460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8005fd0 tx=0x7f1fa800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.909+0000 7f1fb1600640 1 -- 172.21.15.57:0/3100424143 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800ccc0 con 0x7f1fb4102010 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.909+0000 7f1fb1600640 1 -- 172.21.15.57:0/3100424143 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa800ee70 con 0x7f1fb4102010 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.909+0000 7f1fb1600640 1 -- 172.21.15.57:0/3100424143 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8003e10 con 0x7f1fb4102010 2024-04-13T05:21:22.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.909+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb41a2370 con 0x7f1fb4102010 2024-04-13T05:21:22.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.910+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41a2870 con 0x7f1fb4102010 2024-04-13T05:21:22.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.914+0000 7f1fb1600640 1 -- 172.21.15.57:0/3100424143 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1fa8004000 con 0x7f1fb4102010 2024-04-13T05:21:22.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.914+0000 7f1fbe990740 1 Processor -- start 2024-04-13T05:21:22.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.914+0000 7f1fbe990740 1 -- start start 2024-04-13T05:21:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e4d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e53040 0x55c0e0e554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fbe990740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0da3020 con 0x55c0e0e4ded0 2024-04-13T05:21:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fbe990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0d496d0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fbe990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0d9efe0 con 0x55c0e0e53040 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fb9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e4d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40656/0 (socket says 172.21.15.57:40656) 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fb9000640 1 -- 172.21.15.57:0/2852696040 learned_addr learned my addr 172.21.15.57:0/2852696040 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.915+0000 7f1fb1600640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f9c0a3c30 0x7f1f9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb1600640 1 -- 172.21.15.57:0/3100424143 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1fa80d09d0 con 0x7f1fb4102010 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f9c0a3c30 0x7f1f9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb9000640 1 -- 172.21.15.57:0/2852696040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e53040 msgr2=0x55c0e0e554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e53040 0x55c0e0e554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb9000640 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e53040 0x55c0e0e554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb9000640 1 -- 172.21.15.57:0/2852696040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 msgr2=0x55c0e0e4d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb9000640 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e4d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.916+0000 7f1fb9000640 1 -- 172.21.15.57:0/2852696040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e0b7f7d0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e4d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e53040 0x55c0e0e554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fb9000640 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e529d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8098d10 tx=0x7f1fa809c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99445375c477eade server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f9c0a3c30 0x7f1f9c0a60f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55c0e0e4e730 tx=0x7f1fac02bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fa6000640 1 -- 172.21.15.57:0/2852696040 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa809cfb0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fa6000640 1 -- 172.21.15.57:0/2852696040 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa809c8b0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.917+0000 7f1fa6000640 1 -- 172.21.15.57:0/2852696040 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8098f20 con 0x55c0e0e4ded0 2024-04-13T05:21:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 -- 172.21.15.57:0/2852696040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 msgr2=0x55c0e0e529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e529d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8098d10 tx=0x7f1fa809c680 comp rx=0 tx=0).stop 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 -- 172.21.15.57:0/2852696040 shutdown_connections 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e53040 0x55c0e0e554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e529d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 --2- 172.21.15.57:0/2852696040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e4d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 -- 172.21.15.57:0/2852696040 >> 172.21.15.57:0/2852696040 conn(0x55c0e0e37a40 msgr2=0x55c0e0e37e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 -- 172.21.15.57:0/2852696040 shutdown_connections 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.919+0000 7f1fbe990740 1 -- 172.21.15.57:0/2852696040 wait complete. 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.920+0000 7f1fbe990740 1 -- start start 2024-04-13T05:21:22.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.920+0000 7f1fbe990740 1 Processor -- start 2024-04-13T05:21:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.921+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.921+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.921+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e67580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fbe990740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0e55f90 con 0x55c0e0e53040 2024-04-13T05:21:22.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:55902/0 (socket says 172.21.15.57:55902) 2024-04-13T05:21:22.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fbe990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0d5ca90 con 0x55c0e0e4ded0 2024-04-13T05:21:22.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 -- 172.21.15.57:0/203185378 learned_addr learned my addr 172.21.15.57:0/203185378 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0e0e56110 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fbe990740 10 client.0 _ll_register_callbacks cb 0x55c0e0d5c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e67580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb9000640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 -- 172.21.15.57:0/203185378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4d4e0 msgr2=0x55c0e0e60000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e60000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 -- 172.21.15.57:0/203185378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 msgr2=0x55c0e0e67580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e67580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb3e00640 1 -- 172.21.15.57:0/203185378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e0ab53f0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.922+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e67580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.923+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00136e0 tx=0x7f1fa0013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75b8241f41eca0f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.923+0000 7f1f9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:21:22.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.923+0000 7f1f9aa00640 1 -- 172.21.15.57:0/203185378 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0004460 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.923+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa809c8b0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.923+0000 7f1f9aa00640 1 -- 172.21.15.57:0/203185378 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1fa00119c0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.924+0000 7f1f9aa00640 1 -- 172.21.15.57:0/203185378 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fa0011bc0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.924+0000 7f1fbe990740 10 client.35708 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.924+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0e0a750f0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.924+0000 7f1f9aa00640 1 -- 172.21.15.57:0/203185378 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa809c8b0 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1f9aa00640 1 -- 172.21.15.57:0/203185378 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1fa001a070 con 0x55c0e0e4ded0 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fbe990740 10 client.35708 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fbe990740 10 client.35708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fbe990740 20 client.35708 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fbe990740 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0e57b10 0x55c0e0e59f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fbe990740 4 client.35708 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:22.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e0d42600 con 0x55c0e0e57b10 2024-04-13T05:21:22.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.925+0000 7f1fb9000640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0e57b10 0x55c0e0e59f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.926+0000 7f1fb9000640 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0e57b10 0x55c0e0e59f90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe479267a09f6107 server_cookie=212eaa69f8b1e454 in_seq=0 out_seq=0 2024-04-13T05:21:22.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.926+0000 7f1f9aa00640 10 client.35708 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1f9aa00640 1 -- 172.21.15.57:0/203185378 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0e0d42600 con 0x55c0e0e57b10 2024-04-13T05:21:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1f9aa00640 10 client.35708 handle_command_reply: tid=0 2024-04-13T05:21:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 client.35708 shutdown 2024-04-13T05:21:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 msgr2=0x55c0e0e60540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00136e0 tx=0x7f1fa0013ba0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 shutdown_connections 2024-04-13T05:21:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0e57b10 0x55c0e0e59f90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e67580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 --2- 172.21.15.57:0/203185378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.928+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 >> 172.21.15.57:0/203185378 conn(0x55c0e0e37a40 msgr2=0x55c0e0e36b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.929+0000 7f1f9aa00640 0 client.35708 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.929+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 shutdown_connections 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.929+0000 7f1fbe990740 1 -- 172.21.15.57:0/203185378 wait complete. 2024-04-13T05:21:22.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.929+0000 7f1fbe990740 20 client.35708 trim_cache size 0 max 16384 2024-04-13T05:21:22.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.955+0000 7f1fbe990740 1 Processor -- start 2024-04-13T05:21:22.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 -- start start 2024-04-13T05:21:22.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e5e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e60540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41a2b80 con 0x55c0e0e53040 2024-04-13T05:21:22.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4071aa0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.956+0000 7f1fbe990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb406f950 con 0x55c0e0e4ded0 2024-04-13T05:21:22.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e60540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e60540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40666/0 (socket says 172.21.15.57:40666) 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9a00640 1 -- 172.21.15.57:0/3168349458 learned_addr learned my addr 172.21.15.57:0/3168349458 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e5e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9000640 1 -- 172.21.15.57:0/3168349458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 msgr2=0x55c0e0e60000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9000640 1 -- 172.21.15.57:0/3168349458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 msgr2=0x55c0e0e60540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e60540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9000640 1 -- 172.21.15.57:0/3168349458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e0e4e760 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.957+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e60540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.958+0000 7f1fb9000640 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e5e850 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8098d10 tx=0x7f1fa800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc231f4c5b2dffc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.958+0000 7f1fa5600640 1 -- 172.21.15.57:0/3168349458 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80d6020 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.958+0000 7f1fa5600640 1 -- 172.21.15.57:0/3168349458 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa809c8b0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.958+0000 7f1fa5600640 1 -- 172.21.15.57:0/3168349458 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80870b0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 -- 172.21.15.57:0/3168349458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 msgr2=0x55c0e0e5e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e5e850 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8098d10 tx=0x7f1fa800cf60 comp rx=0 tx=0).stop 2024-04-13T05:21:22.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 -- 172.21.15.57:0/3168349458 shutdown_connections 2024-04-13T05:21:22.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e53040 0x55c0e0e60540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 --2- 172.21.15.57:0/3168349458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e5e850 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.959+0000 7f1fbe990740 1 -- 172.21.15.57:0/3168349458 >> 172.21.15.57:0/3168349458 conn(0x55c0e0e37a40 msgr2=0x55c0e0e32a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.960+0000 7f1fbe990740 1 -- 172.21.15.57:0/3168349458 shutdown_connections 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.960+0000 7f1fbe990740 1 -- 172.21.15.57:0/3168349458 wait complete. 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.960+0000 7f1fbe990740 1 -- start start 2024-04-13T05:21:22.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.960+0000 7f1fbe990740 1 Processor -- start 2024-04-13T05:21:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e6fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e6e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e53040 0x55c0e0e6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb406ffa0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40713d0 con 0x55c0e0e53040 2024-04-13T05:21:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4070630 con 0x55c0e0e4ded0 2024-04-13T05:21:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fbe990740 10 client.0 _ll_register_callbacks cb 0x7f1fa0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.961+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e6fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e6fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40668/0 (socket says 172.21.15.57:40668) 2024-04-13T05:21:22.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 -- 172.21.15.57:0/1560985150 learned_addr learned my addr 172.21.15.57:0/1560985150 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e6e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e53040 0x55c0e0e6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 -- 172.21.15.57:0/1560985150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 msgr2=0x55c0e0e6e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e6e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 -- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e53040 msgr2=0x55c0e0e6ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e53040 0x55c0e0e6ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9000640 1 -- 172.21.15.57:0/1560985150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e0ec80b0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.962+0000 7f1fb9a00640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e53040 0x55c0e0e6ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.963+0000 7f1fb9000640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e6fde0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1fa809c840 tx=0x7f1fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72fc5ee261bafa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.963+0000 7f1f9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:22.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.964+0000 7f1f9aa00640 1 -- 172.21.15.57:0/1560985150 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80d6020 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.964+0000 7f1f9aa00640 1 -- 172.21.15.57:0/1560985150 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1fa80035f0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.964+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa809c8b0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.965+0000 7f1f9aa00640 1 -- 172.21.15.57:0/1560985150 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fa809abb0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.965+0000 7f1fbe990740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:22.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.965+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0e0ec9210 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.966+0000 7f1f9aa00640 1 -- 172.21.15.57:0/1560985150 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa809c8b0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.966+0000 7f1f9aa00640 1 -- 172.21.15.57:0/1560985150 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1fa809dbe0 con 0x55c0e0e4d4e0 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.966+0000 7f1fbe990740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.967+0000 7f1fbe990740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.967+0000 7f1fbe990740 20 client.16281 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.967+0000 7f1fbe990740 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0ec9650 0x55c0e0ecba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.967+0000 7f1fbe990740 4 client.16281 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:21:22.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.967+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0e0db0a40 con 0x55c0e0ec9650 2024-04-13T05:21:22.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.967+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0ec9650 0x55c0e0ecba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:22.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.968+0000 7f1fb3e00640 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0ec9650 0x55c0e0ecba40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9beeefc457ec426 server_cookie=d6421f96454a9145 in_seq=0 out_seq=0 2024-04-13T05:21:22.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.968+0000 7f1f9aa00640 10 client.16281 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.968+0000 7f1f9aa00640 1 -- 172.21.15.57:0/1560985150 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c0e0db0a40 con 0x55c0e0ec9650 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.968+0000 7f1f9aa00640 10 client.16281 handle_command_reply: tid=0 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.968+0000 7f1fbe990740 1 client.16281 shutdown 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 msgr2=0x55c0e0e6fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e6fde0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1fa809c840 tx=0x7f1fa80046c0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 shutdown_connections 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0e0ec9650 0x55c0e0ecba40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0e0e53040 0x55c0e0e6ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0e0e4ded0 0x55c0e0e6e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 --2- 172.21.15.57:0/1560985150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0e0e4d4e0 0x55c0e0e6fde0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 >> 172.21.15.57:0/1560985150 conn(0x55c0e0e37a40 msgr2=0x55c0e0e5e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1f9aa00640 0 client.16281 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 shutdown_connections 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 1 -- 172.21.15.57:0/1560985150 wait complete. 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.969+0000 7f1fbe990740 20 client.16281 trim_cache size 0 max 16384 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:21:22.973 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout: "aeab05bc-3faa-4663-834d-1e3a5d76ced6": { 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "aeab05bc-3faa-4663-834d-1e3a5d76ced6", 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:21:22.974 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.970+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f9c0a3c30 msgr2=0x7f1f9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.970+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f9c0a3c30 0x7f1f9c0a60f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55c0e0e4e730 tx=0x7f1fac02bef0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.970+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 msgr2=0x7f1fb419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.970+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 0x7f1fb419a460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8005fd0 tx=0x7f1fa800ec20 comp rx=0 tx=0).stop 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 shutdown_connections 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1f9c0a3c30 0x7f1f9c0a60f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fb4103bc0 0x7f1fb41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb4103200 0x7f1fb419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 --2- 172.21.15.57:0/3100424143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb4102010 0x7f1fb419a460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 >> 172.21.15.57:0/3100424143 conn(0x7f1fb40fd5a0 msgr2=0x7f1fb40741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 shutdown_connections 2024-04-13T05:21:22.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:22.971+0000 7f1fbba00640 1 -- 172.21.15.57:0/3100424143 wait complete. 2024-04-13T05:21:22.994 INFO:tasks.cephfs.filesystem:scrub status for tag:aeab05bc-3faa-4663-834d-1e3a5d76ced6 - {'path': '/', 'tag': 'aeab05bc-3faa-4663-834d-1e3a5d76ced6', 'options': 'recursive,force'} 2024-04-13T05:21:23.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:22 smithi083 ceph-mon[35133]: pgmap v569: 129 pgs: 129 active+clean; 123 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-04-13T05:21:23.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:22 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:23.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:22 smithi057 ceph-mon[32331]: pgmap v569: 129 pgs: 129 active+clean; 123 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-04-13T05:21:23.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:22 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:22 smithi189 ceph-mon[35531]: pgmap v569: 129 pgs: 129 active+clean; 123 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-04-13T05:21:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:22 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:24.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:21:24.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:24.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:24.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:24.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:23 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:23 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:24.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:23 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:24 smithi083 ceph-mon[35133]: pgmap v570: 129 pgs: 129 active+clean; 131 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-04-13T05:21:25.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:24 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:25.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:24 smithi057 ceph-mon[32331]: pgmap v570: 129 pgs: 129 active+clean; 131 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-04-13T05:21:25.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:24 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:24 smithi189 ceph-mon[35531]: pgmap v570: 129 pgs: 129 active+clean; 131 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-04-13T05:21:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:24 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:27.253 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:26 smithi083 ceph-mon[35133]: pgmap v571: 129 pgs: 129 active+clean; 135 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-04-13T05:21:27.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:26 smithi057 ceph-mon[32331]: pgmap v571: 129 pgs: 129 active+clean; 135 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-04-13T05:21:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:26 smithi189 ceph-mon[35531]: pgmap v571: 129 pgs: 129 active+clean; 135 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-04-13T05:21:29.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:28 smithi083 ceph-mon[35133]: pgmap v572: 129 pgs: 129 active+clean; 135 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-13T05:21:29.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:28 smithi057 ceph-mon[32331]: pgmap v572: 129 pgs: 129 active+clean; 135 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-13T05:21:29.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:28 smithi189 ceph-mon[35531]: pgmap v572: 129 pgs: 129 active+clean; 135 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-04-13T05:21:29.636 INFO:tasks.workunit.client.0.smithi057.stdout: 32 8401 189 5749 170 2635 140 2024-04-13T05:21:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:30 smithi083 ceph-mon[35133]: pgmap v573: 129 pgs: 129 active+clean; 142 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-04-13T05:21:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:30 smithi057 ceph-mon[32331]: pgmap v573: 129 pgs: 129 active+clean; 142 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-04-13T05:21:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:30 smithi189 ceph-mon[35531]: pgmap v573: 129 pgs: 129 active+clean; 142 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-04-13T05:21:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:32 smithi083 ceph-mon[35133]: pgmap v574: 129 pgs: 129 active+clean; 143 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-04-13T05:21:33.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:32 smithi057 ceph-mon[32331]: pgmap v574: 129 pgs: 129 active+clean; 143 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-04-13T05:21:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:32 smithi189 ceph-mon[35531]: pgmap v574: 129 pgs: 129 active+clean; 143 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-04-13T05:21:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:34 smithi083 ceph-mon[35133]: pgmap v575: 129 pgs: 129 active+clean; 148 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-04-13T05:21:35.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:34 smithi057 ceph-mon[32331]: pgmap v575: 129 pgs: 129 active+clean; 148 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-04-13T05:21:35.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:34 smithi189 ceph-mon[35531]: pgmap v575: 129 pgs: 129 active+clean; 148 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-04-13T05:21:37.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:36 smithi083 ceph-mon[35133]: pgmap v576: 129 pgs: 129 active+clean; 149 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-04-13T05:21:37.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:36 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:21:37.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:36 smithi057 ceph-mon[32331]: pgmap v576: 129 pgs: 129 active+clean; 149 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-04-13T05:21:37.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:36 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:21:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:36 smithi189 ceph-mon[35531]: pgmap v576: 129 pgs: 129 active+clean; 149 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-04-13T05:21:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:36 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:21:38.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:38 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:21:38.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:38 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:21:38.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:38 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:21:38.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:38 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:21:38.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:38 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:21:38.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:38 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:21:39.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:39 smithi057 ceph-mon[32331]: pgmap v577: 129 pgs: 129 active+clean; 149 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-04-13T05:21:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:39 smithi189 ceph-mon[35531]: pgmap v577: 129 pgs: 129 active+clean; 149 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-04-13T05:21:39.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:39 smithi083 ceph-mon[35133]: pgmap v577: 129 pgs: 129 active+clean; 149 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-04-13T05:21:39.636 INFO:tasks.workunit.client.0.smithi057.stdout: 32 8706 96 5403 89 2527 241 2024-04-13T05:21:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:41 smithi057 ceph-mon[32331]: pgmap v578: 129 pgs: 129 active+clean; 157 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-04-13T05:21:41.304 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:41 smithi083 ceph-mon[35133]: pgmap v578: 129 pgs: 129 active+clean; 157 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-04-13T05:21:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:41 smithi189 ceph-mon[35531]: pgmap v578: 129 pgs: 129 active+clean; 157 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-04-13T05:21:43.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:43 smithi057 ceph-mon[32331]: pgmap v579: 129 pgs: 129 active+clean; 158 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-04-13T05:21:43.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:43 smithi189 ceph-mon[35531]: pgmap v579: 129 pgs: 129 active+clean; 158 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-04-13T05:21:43.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:43 smithi083 ceph-mon[35133]: pgmap v579: 129 pgs: 129 active+clean; 158 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-04-13T05:21:45.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:45 smithi057 ceph-mon[32331]: pgmap v580: 129 pgs: 129 active+clean; 163 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-04-13T05:21:45.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:45 smithi189 ceph-mon[35531]: pgmap v580: 129 pgs: 129 active+clean; 163 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-04-13T05:21:45.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:45 smithi083 ceph-mon[35133]: pgmap v580: 129 pgs: 129 active+clean; 163 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-04-13T05:21:47.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:47 smithi189 ceph-mon[35531]: pgmap v581: 129 pgs: 129 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-04-13T05:21:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:47 smithi083 ceph-mon[35133]: pgmap v581: 129 pgs: 129 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-04-13T05:21:47.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:47 smithi057 ceph-mon[32331]: pgmap v581: 129 pgs: 129 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-04-13T05:21:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:49 smithi189 ceph-mon[35531]: pgmap v582: 129 pgs: 129 active+clean; 167 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-04-13T05:21:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:49 smithi083 ceph-mon[35133]: pgmap v582: 129 pgs: 129 active+clean; 167 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-04-13T05:21:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:49 smithi057 ceph-mon[32331]: pgmap v582: 129 pgs: 129 active+clean; 167 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-04-13T05:21:49.636 INFO:tasks.workunit.client.0.smithi057.stdout: 34 7485 132 4972 147 2113 200 2024-04-13T05:21:51.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:51 smithi189 ceph-mon[35531]: pgmap v583: 129 pgs: 129 active+clean; 172 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-04-13T05:21:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:51 smithi083 ceph-mon[35133]: pgmap v583: 129 pgs: 129 active+clean; 172 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-04-13T05:21:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:51 smithi057 ceph-mon[32331]: pgmap v583: 129 pgs: 129 active+clean; 172 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-04-13T05:21:52.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:21:52.995 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 -- 172.21.15.57:0/2029608456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8102010 msgr2=0x7fabd8102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 --2- 172.21.15.57:0/2029608456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8102010 0x7fabd8102430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009900 tx=0x7fabc802bc40 comp rx=0 tx=0).stop 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 -- 172.21.15.57:0/2029608456 shutdown_connections 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 --2- 172.21.15.57:0/2029608456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103200 0x7fabd8103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 --2- 172.21.15.57:0/2029608456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabd8103bc0 0x7fabd810a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 --2- 172.21.15.57:0/2029608456 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8102010 0x7fabd8102430 secure :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009900 tx=0x7fabc802bc40 comp rx=0 tx=0).stop 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 -- 172.21.15.57:0/2029608456 >> 172.21.15.57:0/2029608456 conn(0x7fabd80fd5a0 msgr2=0x7fabd80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 -- 172.21.15.57:0/2029608456 shutdown_connections 2024-04-13T05:21:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.253+0000 7fabdca00640 1 -- 172.21.15.57:0/2029608456 wait complete. 2024-04-13T05:21:53.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.254+0000 7fabdca00640 1 Processor -- start 2024-04-13T05:21:53.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 -- start start 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8103200 0x7fabd810e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103bc0 0x7fabd810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabd810b0c0 0x7fabd810b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fabd810fda0 con 0x7fabd8103200 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabd810fc20 con 0x7fabd8103bc0 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabdca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabd810ff20 con 0x7fabd810b0c0 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabd5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8103200 0x7fabd810e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabd5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8103200 0x7fabd810e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52920/0 (socket says 172.21.15.57:52920) 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabd5e00640 1 -- 172.21.15.57:0/3427229404 learned_addr learned my addr 172.21.15.57:0/3427229404 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:53.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.255+0000 7fabd5400640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103bc0 0x7fabd810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd6800640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabd810b0c0 0x7fabd810b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd5400640 1 -- 172.21.15.57:0/3427229404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabd810b0c0 msgr2=0x7fabd810b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd5400640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabd810b0c0 0x7fabd810b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd5400640 1 -- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8103200 msgr2=0x7fabd810e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd5400640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8103200 0x7fabd810e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd5400640 1 -- 172.21.15.57:0/3427229404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd81a6a40 con 0x7fabd8103bc0 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.256+0000 7fabd5400640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103bc0 0x7fabd810eb40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c970 tx=0x7fabc400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.257+0000 7fabcea00640 1 -- 172.21.15.57:0/3427229404 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4007bb0 con 0x7fabd8103bc0 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.257+0000 7fabcea00640 1 -- 172.21.15.57:0/3427229404 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc4007d50 con 0x7fabd8103bc0 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.257+0000 7fabcea00640 1 -- 172.21.15.57:0/3427229404 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4018780 con 0x7fabd8103bc0 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.257+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd81a6d30 con 0x7fabd8103bc0 2024-04-13T05:21:53.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.257+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd81a71a0 con 0x7fabd8103bc0 2024-04-13T05:21:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 Processor -- start 2024-04-13T05:21:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 -- start start 2024-04-13T05:21:53.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabcea00640 1 -- 172.21.15.57:0/3427229404 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fabc400f450 con 0x7fabd8103bc0 2024-04-13T05:21:53.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 0x5597733248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559773324ec0 0x5597733299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x55977332c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55977328b020 con 0x5597733244d0 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5597732316d0 con 0x559773324ec0 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.263+0000 7fabdf963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559773286fe0 con 0x55977332a030 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.264+0000 7fabd6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 0x5597733248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.264+0000 7fabd6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 0x5597733248f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52936/0 (socket says 172.21.15.57:52936) 2024-04-13T05:21:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.264+0000 7fabd6800640 1 -- 172.21.15.57:0/445548410 learned_addr learned my addr 172.21.15.57:0/445548410 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.264+0000 7fabd5e00640 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559773324ec0 0x5597733299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabd5400640 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x55977332c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabd6800640 1 -- 172.21.15.57:0/445548410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 msgr2=0x55977332c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabd6800640 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x55977332c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabd6800640 1 -- 172.21.15.57:0/445548410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559773324ec0 msgr2=0x5597733299c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabd6800640 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559773324ec0 0x5597733299c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabd6800640 1 -- 172.21.15.57:0/445548410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597730677d0 con 0x5597733244d0 2024-04-13T05:21:53.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabcea00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabbc0a3c80 0x7fabbc0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.265+0000 7fabcea00640 1 -- 172.21.15.57:0/3427229404 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fabc40d0820 con 0x7fabd8103bc0 2024-04-13T05:21:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.266+0000 7fabd6800640 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 0x5597733248f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009df0 tx=0x7fabd000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1259f552568a75d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.266+0000 7fabd5e00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabbc0a3c80 0x7fabbc0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.266+0000 7fabc3000640 1 -- 172.21.15.57:0/445548410 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0003970 con 0x5597733244d0 2024-04-13T05:21:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.266+0000 7fabc3000640 1 -- 172.21.15.57:0/445548410 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd0004030 con 0x5597733244d0 2024-04-13T05:21:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.266+0000 7fabc3000640 1 -- 172.21.15.57:0/445548410 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011770 con 0x5597733244d0 2024-04-13T05:21:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.267+0000 7fabdf963740 1 -- 172.21.15.57:0/445548410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 msgr2=0x5597733248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.267+0000 7fabdf963740 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 0x5597733248f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009df0 tx=0x7fabd000f730 comp rx=0 tx=0).stop 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.268+0000 7fabdf963740 1 -- 172.21.15.57:0/445548410 shutdown_connections 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.268+0000 7fabdf963740 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559773324ec0 0x5597733299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.268+0000 7fabdf963740 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x55977332c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.268+0000 7fabdf963740 1 --2- 172.21.15.57:0/445548410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5597733244d0 0x5597733248f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.268+0000 7fabdf963740 1 -- 172.21.15.57:0/445548410 >> 172.21.15.57:0/445548410 conn(0x55977331fa40 msgr2=0x55977331fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.268+0000 7fabd5e00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabbc0a3c80 0x7fabbc0a6140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fabc802c250 tx=0x7fabc8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.269+0000 7fabdf963740 1 -- 172.21.15.57:0/445548410 shutdown_connections 2024-04-13T05:21:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.269+0000 7fabdf963740 1 -- 172.21.15.57:0/445548410 wait complete. 2024-04-13T05:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.269+0000 7fabdf963740 1 -- start start 2024-04-13T05:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.269+0000 7fabdf963740 1 Processor -- start 2024-04-13T05:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.270+0000 7fabdf963740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x55977332e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.270+0000 7fabdf963740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x55977332ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.270+0000 7fabdf963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x559773335d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x55977332ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x55977332e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x55977332e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46532/0 (socket says 172.21.15.57:46532) 2024-04-13T05:21:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x55977332ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52940/0 (socket says 172.21.15.57:52940) 2024-04-13T05:21:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5e00640 1 -- 172.21.15.57:0/2439389904 learned_addr learned my addr 172.21.15.57:0/2439389904 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55977332cfe0 con 0x559773324ec0 2024-04-13T05:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559773244a90 con 0x5597733244d0 2024-04-13T05:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55977332d160 con 0x55977332a030 2024-04-13T05:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabdf963740 10 client.0 _ll_register_callbacks cb 0x559773244480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd6800640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x559773335d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5e00640 1 -- 172.21.15.57:0/2439389904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 msgr2=0x559773335d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.271+0000 7fabd5e00640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x559773335d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.272+0000 7fabd5e00640 1 -- 172.21.15.57:0/2439389904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 msgr2=0x55977332ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.272+0000 7fabd5e00640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x55977332ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.272+0000 7fabd5e00640 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559772f9d3f0 con 0x5597733244d0 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.272+0000 7fabd5400640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x55977332ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.272+0000 7fabd5e00640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x55977332e800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fabd8103060 tx=0x7fabc802bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3682e6e73dfff0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.272+0000 7fabb7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.273+0000 7fabb7400640 1 -- 172.21.15.57:0/2439389904 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8050030 con 0x5597733244d0 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.273+0000 7fabb7400640 1 -- 172.21.15.57:0/2439389904 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fabc803a040 con 0x5597733244d0 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.273+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd0004030 con 0x5597733244d0 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.273+0000 7fabb7400640 1 -- 172.21.15.57:0/2439389904 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fabc805d050 con 0x5597733244d0 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.273+0000 7fabdf963740 10 client.35723 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.273+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559772f5d0f0 con 0x5597733244d0 2024-04-13T05:21:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabb7400640 1 -- 172.21.15.57:0/2439389904 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabd0004030 con 0x5597733244d0 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabb7400640 1 -- 172.21.15.57:0/2439389904 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fabc80039c0 con 0x5597733244d0 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabdf963740 10 client.35723 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabdf963740 10 client.35723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabdf963740 20 client.35723 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabdf963740 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55977333a6c0 0x55977333cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabdf963740 4 client.35723 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.274+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55977322a600 con 0x55977333a6c0 2024-04-13T05:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.275+0000 7fabd5400640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55977333a6c0 0x55977333cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.275+0000 7fabd5400640 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55977333a6c0 0x55977333cb60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59848e40b52054c server_cookie=d556d05c17bab9ea in_seq=0 out_seq=0 2024-04-13T05:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.275+0000 7fabb7400640 10 client.35723 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabb7400640 1 -- 172.21.15.57:0/2439389904 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55977322a600 con 0x55977333a6c0 2024-04-13T05:21:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabb7400640 10 client.35723 handle_command_reply: tid=0 2024-04-13T05:21:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 client.35723 shutdown 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 msgr2=0x55977332e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x55977332e800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fabd8103060 tx=0x7fabc802bc40 comp rx=0 tx=0).stop 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 shutdown_connections 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55977333a6c0 0x55977333cb60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977332a030 0x559773335d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x55977332ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabb7400640 0 client.35723 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 --2- 172.21.15.57:0/2439389904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x55977332e800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.277+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 >> 172.21.15.57:0/2439389904 conn(0x55977331fa40 msgr2=0x55977331eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.278+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 shutdown_connections 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.278+0000 7fabdf963740 1 -- 172.21.15.57:0/2439389904 wait complete. 2024-04-13T05:21:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.278+0000 7fabdf963740 20 client.35723 trim_cache size 0 max 16384 2024-04-13T05:21:53.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.307+0000 7fabdf963740 1 Processor -- start 2024-04-13T05:21:53.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 -- start start 2024-04-13T05:21:53.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597733244d0 0x55977331aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x559773335d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977332a030 0x55977332eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81b40e0 con 0x559773324ec0 2024-04-13T05:21:53.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81b1950 con 0x55977332a030 2024-04-13T05:21:53.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabdf963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81af7a0 con 0x5597733244d0 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.308+0000 7fabd5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x559773335d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x559773335d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52948/0 (socket says 172.21.15.57:52948) 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 -- 172.21.15.57:0/3279024946 learned_addr learned my addr 172.21.15.57:0/3279024946 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd6800640 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977332a030 0x55977332eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5e00640 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597733244d0 0x55977331aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 -- 172.21.15.57:0/3279024946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597733244d0 msgr2=0x55977331aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597733244d0 0x55977331aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 -- 172.21.15.57:0/3279024946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977332a030 msgr2=0x55977332eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977332a030 0x55977332eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 -- 172.21.15.57:0/3279024946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597733209b0 con 0x559773324ec0 2024-04-13T05:21:53.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabd5400640 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x559773335d40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fabc40d62d0 tx=0x7fabc40cddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0e3dc703bd81b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.309+0000 7fabc2600640 1 -- 172.21.15.57:0/3279024946 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40a5370 con 0x559773324ec0 2024-04-13T05:21:53.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.310+0000 7fabc2600640 1 -- 172.21.15.57:0/3279024946 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc40870b0 con 0x559773324ec0 2024-04-13T05:21:53.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.310+0000 7fabc2600640 1 -- 172.21.15.57:0/3279024946 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4098d70 con 0x559773324ec0 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.310+0000 7fabdf963740 1 -- 172.21.15.57:0/3279024946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 msgr2=0x559773335d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.310+0000 7fabdf963740 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x559773335d40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fabc40d62d0 tx=0x7fabc40cddf0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 -- 172.21.15.57:0/3279024946 shutdown_connections 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55977332a030 0x55977332eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559773324ec0 0x559773335d40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 --2- 172.21.15.57:0/3279024946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597733244d0 0x55977331aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 -- 172.21.15.57:0/3279024946 >> 172.21.15.57:0/3279024946 conn(0x55977331fa40 msgr2=0x559773322ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 -- 172.21.15.57:0/3279024946 shutdown_connections 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 -- 172.21.15.57:0/3279024946 wait complete. 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 -- start start 2024-04-13T05:21:53.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.311+0000 7fabdf963740 1 Processor -- start 2024-04-13T05:21:53.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x559773333760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559773324ec0 0x559773333ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55977332a030 0x559773334cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81afe10 con 0x55977332a030 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81b1280 con 0x5597733244d0 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81b04e0 con 0x559773324ec0 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabdf963740 10 client.0 _ll_register_callbacks cb 0x7fabc80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabd6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55977332a030 0x559773334cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabd5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x559773333760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabd5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x559773333760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46550/0 (socket says 172.21.15.57:46550) 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.312+0000 7fabd5e00640 1 -- 172.21.15.57:0/1920921546 learned_addr learned my addr 172.21.15.57:0/1920921546 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd6800640 1 -- 172.21.15.57:0/1920921546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559773324ec0 msgr2=0x559773333ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd6800640 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559773324ec0 0x559773333ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd6800640 1 -- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 msgr2=0x559773333760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd6800640 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x559773333760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd6800640 1 -- 172.21.15.57:0/1920921546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597733352a0 con 0x55977332a030 2024-04-13T05:21:53.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd5e00640 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x559773333760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:53.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabd6800640 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55977332a030 0x559773334cb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009dc0 tx=0x7fabd0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92700111a5216bc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.313+0000 7fabb7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:53.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabb7400640 1 -- 172.21.15.57:0/1920921546 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd002c060 con 0x55977332a030 2024-04-13T05:21:53.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabb7400640 1 -- 172.21.15.57:0/1920921546 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fabd0003df0 con 0x55977332a030 2024-04-13T05:21:53.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc40870b0 con 0x55977332a030 2024-04-13T05:21:53.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabb7400640 1 -- 172.21.15.57:0/1920921546 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fabd0033050 con 0x55977332a030 2024-04-13T05:21:53.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabdf963740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597733a0f80 con 0x55977332a030 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.314+0000 7fabb7400640 1 -- 172.21.15.57:0/1920921546 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabc40870b0 con 0x55977332a030 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabb7400640 1 -- 172.21.15.57:0/1920921546 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fabd0026070 con 0x55977332a030 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabdf963740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabdf963740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabdf963740 20 client.16299 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabdf963740 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5597733a1540 0x5597733a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabdf963740 4 client.16299 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559773298a40 con 0x5597733a1540 2024-04-13T05:21:53.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.315+0000 7fabd5e00640 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5597733a1540 0x5597733a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.316+0000 7fabd5e00640 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5597733a1540 0x5597733a3930 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7248f0725c21d612 server_cookie=a1220cb2005973ed in_seq=0 out_seq=0 2024-04-13T05:21:53.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.316+0000 7fabb7400640 10 client.16299 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabb7400640 1 -- 172.21.15.57:0/1920921546 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559773298a40 con 0x5597733a1540 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabb7400640 10 client.16299 handle_command_reply: tid=0 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 client.16299 shutdown 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55977332a030 msgr2=0x559773334cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55977332a030 0x559773334cb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009dc0 tx=0x7fabd0003970 comp rx=0 tx=0).stop 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 shutdown_connections 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5597733a1540 0x5597733a3930 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55977332a030 0x559773334cb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559773324ec0 0x559773333ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 --2- 172.21.15.57:0/1920921546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5597733244d0 0x559773333760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 >> 172.21.15.57:0/1920921546 conn(0x55977331fa40 msgr2=0x559773323530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabb7400640 0 client.16299 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 shutdown_connections 2024-04-13T05:21:53.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.317+0000 7fabdf963740 1 -- 172.21.15.57:0/1920921546 wait complete. 2024-04-13T05:21:53.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.318+0000 7fabdf963740 20 client.16299 trim_cache size 0 max 16384 2024-04-13T05:21:53.322 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:21:53.322 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:21:53.322 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:21:53.322 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabbc0a3c80 msgr2=0x7fabbc0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabbc0a3c80 0x7fabbc0a6140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fabc802c250 tx=0x7fabc8000bc0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103bc0 msgr2=0x7fabd810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103bc0 0x7fabd810eb40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c970 tx=0x7fabc400ce30 comp rx=0 tx=0).stop 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 shutdown_connections 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabbc0a3c80 0x7fabbc0a6140 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabd8103bc0 0x7fabd810eb40 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c970 tx=0x7fabc400ce30 comp rx=0 tx=0).stop 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabd810b0c0 0x7fabd810b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 --2- 172.21.15.57:0/3427229404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fabd8103200 0x7fabd810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 >> 172.21.15.57:0/3427229404 conn(0x7fabd80fd5a0 msgr2=0x7fabd8106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 shutdown_connections 2024-04-13T05:21:53.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.319+0000 7fabdca00640 1 -- 172.21.15.57:0/3427229404 wait complete. 2024-04-13T05:21:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:53 smithi189 ceph-mon[35531]: pgmap v584: 129 pgs: 129 active+clean; 176 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-04-13T05:21:53.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:53 smithi057 ceph-mon[32331]: pgmap v584: 129 pgs: 129 active+clean; 176 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-04-13T05:21:53.343 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:21:53.343 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:21:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:53 smithi083 ceph-mon[35133]: pgmap v584: 129 pgs: 129 active+clean; 176 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 -- 172.21.15.57:0/147900173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 msgr2=0x7f6c44078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 --2- 172.21.15.57:0/147900173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c44078020 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009860 tx=0x7f6c3002bb70 comp rx=0 tx=0).stop 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 -- 172.21.15.57:0/147900173 shutdown_connections 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 --2- 172.21.15.57:0/147900173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c44078020 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 --2- 172.21.15.57:0/147900173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c44078560 0x7f6c44078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 --2- 172.21.15.57:0/147900173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44076980 0x7f6c44076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 -- 172.21.15.57:0/147900173 >> 172.21.15.57:0/147900173 conn(0x7f6c440ff7b0 msgr2=0x7f6c44101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:53.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.610+0000 7f6c4a600640 1 -- 172.21.15.57:0/147900173 shutdown_connections 2024-04-13T05:21:53.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.611+0000 7f6c4a600640 1 -- 172.21.15.57:0/147900173 wait complete. 2024-04-13T05:21:53.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 Processor -- start 2024-04-13T05:21:53.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 -- start start 2024-04-13T05:21:53.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c44076980 0x7f6c4419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c4419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44078560 0x7f6c441a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44114140 con 0x7f6c44078560 2024-04-13T05:21:53.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44113fc0 con 0x7f6c44077bc0 2024-04-13T05:21:53.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.612+0000 7f6c4a600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441142c0 con 0x7f6c44076980 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.613+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44078560 0x7f6c441a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.613+0000 7f6c42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c4419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.613+0000 7f6c43400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c44076980 0x7f6c4419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.613+0000 7f6c42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c4419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46568/0 (socket says 172.21.15.57:46568) 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.613+0000 7f6c42a00640 1 -- 172.21.15.57:0/2826760499 learned_addr learned my addr 172.21.15.57:0/2826760499 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c43e00640 1 -- 172.21.15.57:0/2826760499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c44076980 msgr2=0x7f6c4419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c43e00640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c44076980 0x7f6c4419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c43e00640 1 -- 172.21.15.57:0/2826760499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 msgr2=0x7f6c4419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:53.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c43e00640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c4419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:53.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c43e00640 1 -- 172.21.15.57:0/2826760499 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441a6470 con 0x7f6c44078560 2024-04-13T05:21:53.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c42a00640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c4419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:53.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.614+0000 7f6c43e00640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44078560 0x7f6c441a5e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e940 tx=0x7f6c3c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.615+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004d20 con 0x7f6c44078560 2024-04-13T05:21:53.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.615+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c3c005020 con 0x7f6c44078560 2024-04-13T05:21:53.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.615+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0116d0 con 0x7f6c44078560 2024-04-13T05:21:53.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.615+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a6760 con 0x7f6c44078560 2024-04-13T05:21:53.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.615+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a6c60 con 0x7f6c44078560 2024-04-13T05:21:53.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.616+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44110df0 con 0x7f6c44078560 2024-04-13T05:21:53.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.620+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c3c00cb20 con 0x7f6c44078560 2024-04-13T05:21:53.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.622+0000 7f6c3be00640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c280a3c30 0x7f6c280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:53.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.622+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6c3c0d07e0 con 0x7f6c44078560 2024-04-13T05:21:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.623+0000 7f6c43400640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c280a3c30 0x7f6c280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.624+0000 7f6c43400640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c280a3c30 0x7f6c280a60f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009530 tx=0x7f6c34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:53.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:53.630+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c3c016030 con 0x7f6c44078560 2024-04-13T05:21:54.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.049+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4402d5e0 con 0x7f6c44078560 2024-04-13T05:21:54.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.050+0000 7f6c3be00640 1 -- 172.21.15.57:0/2826760499 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11859 (secure 0 0 0) 0x7f6c3c09c770 con 0x7f6c44078560 2024-04-13T05:21:54.053 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:21:54.053 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:20:25.642360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:21:54.053 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 28 2024-04-13T05:21:54.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c280a3c30 msgr2=0x7f6c280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c280a3c30 0x7f6c280a60f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009530 tx=0x7f6c34009260 comp rx=0 tx=0).stop 2024-04-13T05:21:54.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44078560 msgr2=0x7f6c441a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44078560 0x7f6c441a5e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e940 tx=0x7f6c3c00ee00 comp rx=0 tx=0).stop 2024-04-13T05:21:54.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 shutdown_connections 2024-04-13T05:21:54.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6c280a3c30 0x7f6c280a60f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c44077bc0 0x7f6c4419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c44078560 0x7f6c441a5e00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 --2- 172.21.15.57:0/2826760499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c44076980 0x7f6c4419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.054+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 >> 172.21.15.57:0/2826760499 conn(0x7f6c440ff7b0 msgr2=0x7f6c44100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.055+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 shutdown_connections 2024-04-13T05:21:54.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.055+0000 7f6c4a600640 1 -- 172.21.15.57:0/2826760499 wait complete. 2024-04-13T05:21:54.078 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:21:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 -- 172.21.15.57:0/147115419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5178104100 msgr2=0x7f5178104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 --2- 172.21.15.57:0/147115419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5178104100 0x7f5178104520 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f516c009900 tx=0x7f516c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 -- 172.21.15.57:0/147115419 shutdown_connections 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 --2- 172.21.15.57:0/147115419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 0x7f5178105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 --2- 172.21.15.57:0/147115419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5178105cb0 0x7f517810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 --2- 172.21.15.57:0/147115419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5178104100 0x7f5178104520 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 -- 172.21.15.57:0/147115419 >> 172.21.15.57:0/147115419 conn(0x7f51780ff670 msgr2=0x7f5178101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 -- 172.21.15.57:0/147115419 shutdown_connections 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.338+0000 7f517e600640 1 -- 172.21.15.57:0/147115419 wait complete. 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.339+0000 7f517e600640 1 Processor -- start 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.340+0000 7f517e600640 1 -- start start 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.340+0000 7f517e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 0x7f5178070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.340+0000 7f517e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5178105cb0 0x7f5178071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51781a8370 0x7f51781aa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517e600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f517810fc90 con 0x7f51781a8370 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 0x7f5178070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 0x7f5178070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46584/0 (socket says 172.21.15.57:46584) 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517d200640 1 -- 172.21.15.57:0/2255474146 learned_addr learned my addr 172.21.15.57:0/2255474146 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517dc00640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51781a8370 0x7f51781aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f5177e00640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5178105cb0 0x7f5178071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.341+0000 7f517e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f517810fb10 con 0x7f51781052f0 2024-04-13T05:21:54.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.342+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f517810fe10 con 0x7f5178105cb0 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.342+0000 7f517dc00640 1 -- 172.21.15.57:0/2255474146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5178105cb0 msgr2=0x7f5178071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.342+0000 7f517dc00640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5178105cb0 0x7f5178071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.342+0000 7f517dc00640 1 -- 172.21.15.57:0/2255474146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 msgr2=0x7f5178070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.342+0000 7f517dc00640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 0x7f5178070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.342+0000 7f517dc00640 1 -- 172.21.15.57:0/2255474146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51781aad50 con 0x7f51781a8370 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.343+0000 7f517dc00640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51781a8370 0x7f51781aa760 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5170009fd0 tx=0x7f517000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.343+0000 7f5175600640 1 -- 172.21.15.57:0/2255474146 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517000ce20 con 0x7f51781a8370 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.343+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51781aafe0 con 0x7f51781a8370 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.343+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781ab550 con 0x7f51781a8370 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.343+0000 7f5175600640 1 -- 172.21.15.57:0/2255474146 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5170005020 con 0x7f51781a8370 2024-04-13T05:21:54.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.344+0000 7f5175600640 1 -- 172.21.15.57:0/2255474146 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170010690 con 0x7f51781a8370 2024-04-13T05:21:54.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.347+0000 7f5182b6f740 1 Processor -- start 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.348+0000 7f5175600640 1 -- 172.21.15.57:0/2255474146 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5170004a40 con 0x7f51781a8370 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.348+0000 7f5182b6f740 1 -- start start 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.348+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a320555070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.348+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a32055cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5182b6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a3204b2020 con 0x55a320554c50 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5182b6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a320458950 con 0x55a32055a7b0 2024-04-13T05:21:54.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5182b6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a3204adfe0 con 0x55a320555640 2024-04-13T05:21:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5177e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a320555070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5177e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a320555070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53000/0 (socket says 172.21.15.57:53000) 2024-04-13T05:21:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5177e00640 1 -- 172.21.15.57:0/3226834733 learned_addr learned my addr 172.21.15.57:0/3226834733 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f517d200640 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.349+0000 7f5177e00640 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a32055cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5177e00640 1 -- 172.21.15.57:0/3226834733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 msgr2=0x55a32055a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5177e00640 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5177e00640 1 -- 172.21.15.57:0/3226834733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 msgr2=0x55a32055cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5177e00640 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a32055cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5177e00640 1 -- 172.21.15.57:0/3226834733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3201c43f0 con 0x55a320554c50 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5175600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51600a3c30 0x7f51600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5175600640 1 -- 172.21.15.57:0/2255474146 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f51700d0c80 con 0x7f51781a8370 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f5177e00640 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a320555070 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f51640125f0 tx=0x7f516400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8355a832cdc42c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.350+0000 7f517d200640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51600a3c30 0x7f51600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.351+0000 7f516a000640 1 -- 172.21.15.57:0/3226834733 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164004020 con 0x55a320554c50 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.351+0000 7f516a000640 1 -- 172.21.15.57:0/3226834733 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5164004c50 con 0x55a320554c50 2024-04-13T05:21:54.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.351+0000 7f516a000640 1 -- 172.21.15.57:0/3226834733 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51640052f0 con 0x55a320554c50 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 -- 172.21.15.57:0/3226834733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 msgr2=0x55a320555070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f517d200640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51600a3c30 0x7f51600a60f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f516c02c6a0 tx=0x7f516c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a320555070 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f51640125f0 tx=0x7f516400ba40 comp rx=0 tx=0).stop 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 -- 172.21.15.57:0/3226834733 shutdown_connections 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a32055cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 --2- 172.21.15.57:0/3226834733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a320555070 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 -- 172.21.15.57:0/3226834733 >> 172.21.15.57:0/3226834733 conn(0x55a3205479b0 msgr2=0x55a320547db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 -- 172.21.15.57:0/3226834733 shutdown_connections 2024-04-13T05:21:54.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.352+0000 7f5182b6f740 1 -- 172.21.15.57:0/3226834733 wait complete. 2024-04-13T05:21:54.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.353+0000 7f5182b6f740 1 -- start start 2024-04-13T05:21:54.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.353+0000 7f5182b6f740 1 Processor -- start 2024-04-13T05:21:54.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.353+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a32055f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f5182b6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a32055d760 con 0x55a320554c50 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f5182b6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a32046bc80 con 0x55a32055a7b0 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f5182b6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a32055d8e0 con 0x55a320555640 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f5182b6f740 10 client.0 _ll_register_callbacks cb 0x55a32046b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f517d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a32055f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f517d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a32055f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53014/0 (socket says 172.21.15.57:53014) 2024-04-13T05:21:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.354+0000 7f517d200640 1 -- 172.21.15.57:0/1261385457 learned_addr learned my addr 172.21.15.57:0/1261385457 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517dc00640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f5177e00640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517d200640 1 -- 172.21.15.57:0/1261385457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 msgr2=0x55a32055f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517d200640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517d200640 1 -- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 msgr2=0x55a3205666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517d200640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517d200640 1 -- 172.21.15.57:0/1261385457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32028e7d0 con 0x55a320554c50 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f5177e00640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517dc00640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f517d200640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a32055f170 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5178105150 tx=0x7f516c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9121c1ddb9f8c2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.355+0000 7f515ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.356+0000 7f515ea00640 1 -- 172.21.15.57:0/1261385457 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c050030 con 0x55a320554c50 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.356+0000 7f515ea00640 1 -- 172.21.15.57:0/1261385457 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f516c03a040 con 0x55a320554c50 2024-04-13T05:21:54.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.356+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5164004c50 con 0x55a320554c50 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.357+0000 7f515ea00640 1 -- 172.21.15.57:0/1261385457 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f516c0046c0 con 0x55a320554c50 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.357+0000 7f5182b6f740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.357+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a320569b00 con 0x55a320554c50 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.357+0000 7f515ea00640 1 -- 172.21.15.57:0/1261385457 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5164004c50 con 0x55a320554c50 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f515ea00640 1 -- 172.21.15.57:0/1261385457 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f516c0039a0 con 0x55a320554c50 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f5182b6f740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f5182b6f740 10 client.16317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:21:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f5182b6f740 20 client.16317 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f5182b6f740 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a32056b6e0 0x55a32056dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f5182b6f740 4 client.16317 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.358+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a320451750 con 0x55a32056b6e0 2024-04-13T05:21:54.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.359+0000 7f5177e00640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a32056b6e0 0x55a32056dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.359+0000 7f5177e00640 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a32056b6e0 0x55a32056dba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4434ae19ac52ac6e server_cookie=1046affd607dea20 in_seq=0 out_seq=0 2024-04-13T05:21:54.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.359+0000 7f515ea00640 10 client.16317 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.360+0000 7f515ea00640 1 -- 172.21.15.57:0/1261385457 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a320451750 con 0x55a32056b6e0 2024-04-13T05:21:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.360+0000 7f515ea00640 10 client.16317 handle_command_reply: tid=0 2024-04-13T05:21:54.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 client.16317 shutdown 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 msgr2=0x55a32055f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a32055f170 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5178105150 tx=0x7f516c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 shutdown_connections 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a32056b6e0 0x55a32056dba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 --2- 172.21.15.57:0/1261385457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320554c50 0x55a32055f170 secure :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5178105150 tx=0x7f516c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:21:54.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.361+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 >> 172.21.15.57:0/1261385457 conn(0x55a3205479b0 msgr2=0x55a320546cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.362+0000 7f515ea00640 0 client.16317 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.362+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 shutdown_connections 2024-04-13T05:21:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.362+0000 7f5182b6f740 1 -- 172.21.15.57:0/1261385457 wait complete. 2024-04-13T05:21:54.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.362+0000 7f5182b6f740 20 client.16317 trim_cache size 0 max 16384 2024-04-13T05:21:54.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.387+0000 7f5182b6f740 1 Processor -- start 2024-04-13T05:21:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.387+0000 7f5182b6f740 1 -- start start 2024-04-13T05:21:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.387+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 0x55a3205468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f5182b6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f51580124e0 con 0x55a320546440 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f5182b6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f515800fd50 con 0x55a32055a7b0 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f5182b6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f515800dba0 con 0x55a320555640 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f517dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 0x55a3205468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f517dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 0x55a3205468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53028/0 (socket says 172.21.15.57:53028) 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f517dc00640 1 -- 172.21.15.57:0/246922617 learned_addr learned my addr 172.21.15.57:0/246922617 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f5177e00640 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.388+0000 7f517d200640 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.389+0000 7f517dc00640 1 -- 172.21.15.57:0/246922617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 msgr2=0x55a32055f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.389+0000 7f517dc00640 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.389+0000 7f517dc00640 1 -- 172.21.15.57:0/246922617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 msgr2=0x55a3205666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.389+0000 7f517dc00640 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.389+0000 7f517dc00640 1 -- 172.21.15.57:0/246922617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a320559df0 con 0x55a320546440 2024-04-13T05:21:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.389+0000 7f517dc00640 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 0x55a3205468f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f517009c8c0 tx=0x7f517009cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff42971edcc230a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.390+0000 7f5169600640 1 -- 172.21.15.57:0/246922617 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51700b5020 con 0x55a320546440 2024-04-13T05:21:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.390+0000 7f5169600640 1 -- 172.21.15.57:0/246922617 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f517009aec0 con 0x55a320546440 2024-04-13T05:21:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.390+0000 7f5169600640 1 -- 172.21.15.57:0/246922617 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51700a5600 con 0x55a320546440 2024-04-13T05:21:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 -- 172.21.15.57:0/246922617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 msgr2=0x55a3205468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 0x55a3205468f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f517009c8c0 tx=0x7f517009cd00 comp rx=0 tx=0).stop 2024-04-13T05:21:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 -- 172.21.15.57:0/246922617 shutdown_connections 2024-04-13T05:21:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a32055a7b0 0x55a3205666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320546440 0x55a3205468f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 --2- 172.21.15.57:0/246922617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a320555640 0x55a32055f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 -- 172.21.15.57:0/246922617 >> 172.21.15.57:0/246922617 conn(0x55a3205479b0 msgr2=0x55a32055ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 -- 172.21.15.57:0/246922617 shutdown_connections 2024-04-13T05:21:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.391+0000 7f5182b6f740 1 -- 172.21.15.57:0/246922617 wait complete. 2024-04-13T05:21:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.392+0000 7f5182b6f740 1 -- start start 2024-04-13T05:21:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.392+0000 7f5182b6f740 1 Processor -- start 2024-04-13T05:21:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 0x55a320563e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320555640 0x55a3205643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f5182b6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a32055a7b0 0x55a3205653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f517d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 0x55a320563e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f517d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 0x55a320563e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46610/0 (socket says 172.21.15.57:46610) 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f5182b6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f515800e210 con 0x55a320555640 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f5182b6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f515800f680 con 0x55a320546440 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f5182b6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f515800e8e0 con 0x55a32055a7b0 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.393+0000 7f517d200640 1 -- 172.21.15.57:0/117965168 learned_addr learned my addr 172.21.15.57:0/117965168 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f5182b6f740 10 client.0 _ll_register_callbacks cb 0x7f516c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:54.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f5177e00640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320555640 0x55a3205643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f517dc00640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a32055a7b0 0x55a3205653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f517d200640 1 -- 172.21.15.57:0/117965168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a32055a7b0 msgr2=0x55a3205653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f517d200640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a32055a7b0 0x55a3205653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f517d200640 1 -- 172.21.15.57:0/117965168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320555640 msgr2=0x55a3205643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f517d200640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320555640 0x55a3205643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.394+0000 7f517d200640 1 -- 172.21.15.57:0/117965168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3205659b0 con 0x55a320546440 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.395+0000 7f517dc00640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a32055a7b0 0x55a3205653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.395+0000 7f517d200640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 0x55a320563e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f516c003e60 tx=0x7f516c004ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fa2f07ce65c51a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.395+0000 7f515ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:21:54.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.396+0000 7f515ea00640 1 -- 172.21.15.57:0/117965168 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c06c060 con 0x55a320546440 2024-04-13T05:21:54.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.396+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517009aec0 con 0x55a320546440 2024-04-13T05:21:54.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.396+0000 7f515ea00640 1 -- 172.21.15.57:0/117965168 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f516c003a00 con 0x55a320546440 2024-04-13T05:21:54.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.396+0000 7f515ea00640 1 -- 172.21.15.57:0/117965168 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f516c073050 con 0x55a320546440 2024-04-13T05:21:54.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.396+0000 7f5182b6f740 10 client.35753 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:54.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.396+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3205d1e50 con 0x55a320546440 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f515ea00640 1 -- 172.21.15.57:0/117965168 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f517009aec0 con 0x55a320546440 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f515ea00640 1 -- 172.21.15.57:0/117965168 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f516c04cb70 con 0x55a320546440 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f5182b6f740 10 client.35753 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f5182b6f740 10 client.35753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f5182b6f740 20 client.35753 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f5182b6f740 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a3205d27f0 0x55a3205d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f5182b6f740 4 client.35753 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:21:54.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.398+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3204bfa40 con 0x55a3205d27f0 2024-04-13T05:21:54.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.399+0000 7f5177e00640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a3205d27f0 0x55a3205d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.400+0000 7f5177e00640 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a3205d27f0 0x55a3205d4c10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f53017d77c1df86e server_cookie=65083553e7aad483 in_seq=0 out_seq=0 2024-04-13T05:21:54.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.400+0000 7f515ea00640 10 client.35753 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:54.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f515ea00640 1 -- 172.21.15.57:0/117965168 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3204bfa40 con 0x55a3205d27f0 2024-04-13T05:21:54.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f515ea00640 10 client.35753 handle_command_reply: tid=0 2024-04-13T05:21:54.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 client.35753 shutdown 2024-04-13T05:21:54.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 msgr2=0x55a320563e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 0x55a320563e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f516c003e60 tx=0x7f516c004ed0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 shutdown_connections 2024-04-13T05:21:54.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55a3205d27f0 0x55a3205d4c10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a32055a7b0 0x55a3205653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f515ea00640 0 client.35753 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:21:54.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a320555640 0x55a3205643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 --2- 172.21.15.57:0/117965168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a320546440 0x55a320563e70 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f516c003e60 tx=0x7f516c004ed0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.401+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 >> 172.21.15.57:0/117965168 conn(0x55a3205479b0 msgr2=0x55a32055ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.402+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 shutdown_connections 2024-04-13T05:21:54.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.402+0000 7f5182b6f740 1 -- 172.21.15.57:0/117965168 wait complete. 2024-04-13T05:21:54.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.402+0000 7f5182b6f740 20 client.35753 trim_cache size 0 max 16384 2024-04-13T05:21:54.405 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:21:54.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.403+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51600a3c30 msgr2=0x7f51600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.403+0000 7f517e600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51600a3c30 0x7f51600a60f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f516c02c6a0 tx=0x7f516c02c7e0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.403+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51781a8370 msgr2=0x7f51781aa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.403+0000 7f517e600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51781a8370 0x7f51781aa760 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5170009fd0 tx=0x7f517000ef10 comp rx=0 tx=0).stop 2024-04-13T05:21:54.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 shutdown_connections 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51600a3c30 0x7f51600a60f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51781a8370 0x7f51781aa760 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5178105cb0 0x7f5178071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 --2- 172.21.15.57:0/2255474146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51781052f0 0x7f5178070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 >> 172.21.15.57:0/2255474146 conn(0x7f51780ff670 msgr2=0x7f5178108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 shutdown_connections 2024-04-13T05:21:54.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.404+0000 7f517e600640 1 -- 172.21.15.57:0/2255474146 wait complete. 2024-04-13T05:21:54.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:54 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2826760499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:21:54.427 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:21:54.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:54 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2826760499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:21:54.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:54 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2826760499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:21:54.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.696+0000 7fb2f5000640 1 -- 172.21.15.57:0/262717184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0075fa0 msgr2=0x7fb2f00743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.696+0000 7fb2f5000640 1 --2- 172.21.15.57:0/262717184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0075fa0 0x7fb2f00743f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 -- 172.21.15.57:0/262717184 shutdown_connections 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 --2- 172.21.15.57:0/262717184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f00749c0 0x7fb2f0074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 --2- 172.21.15.57:0/262717184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f0109d80 0x7fb2f010c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 --2- 172.21.15.57:0/262717184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0075fa0 0x7fb2f00743f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 -- 172.21.15.57:0/262717184 >> 172.21.15.57:0/262717184 conn(0x7fb2f00fd460 msgr2=0x7fb2f00ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 -- 172.21.15.57:0/262717184 shutdown_connections 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.697+0000 7fb2f5000640 1 -- 172.21.15.57:0/262717184 wait complete. 2024-04-13T05:21:54.700 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.698+0000 7fb2f5000640 1 Processor -- start 2024-04-13T05:21:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.698+0000 7fb2f5000640 1 -- start start 2024-04-13T05:21:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f00749c0 0x7fb2f0105270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 0x7fb2f01057b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2f5000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0109d80 0x7fb2f0101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2f5000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010fc30 con 0x7fb2f00749c0 2024-04-13T05:21:54.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2f5000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010fab0 con 0x7fb2f0075fa0 2024-04-13T05:21:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2f5000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010fdb0 con 0x7fb2f0109d80 2024-04-13T05:21:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 0x7fb2f01057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 0x7fb2f01057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46638/0 (socket says 172.21.15.57:46638) 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2eea00640 1 -- 172.21.15.57:0/3617361520 learned_addr learned my addr 172.21.15.57:0/3617361520 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2ef400640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f00749c0 0x7fb2f0105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.699+0000 7fb2efe00640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0109d80 0x7fb2f0101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2eea00640 1 -- 172.21.15.57:0/3617361520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0109d80 msgr2=0x7fb2f0101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2eea00640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0109d80 0x7fb2f0101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2eea00640 1 -- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f00749c0 msgr2=0x7fb2f0105270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2eea00640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f00749c0 0x7fb2f0105270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2eea00640 1 -- 172.21.15.57:0/3617361520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f01026a0 con 0x7fb2f0075fa0 2024-04-13T05:21:54.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2ef400640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f00749c0 0x7fb2f0105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.700+0000 7fb2eea00640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 0x7fb2f01057b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00d950 tx=0x7fb2dc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.701+0000 7fb2e7e00640 1 -- 172.21.15.57:0/3617361520 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc017070 con 0x7fb2f0075fa0 2024-04-13T05:21:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.701+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f0199ed0 con 0x7fb2f0075fa0 2024-04-13T05:21:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.701+0000 7fb2e7e00640 1 -- 172.21.15.57:0/3617361520 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc009c30 con 0x7fb2f0075fa0 2024-04-13T05:21:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.701+0000 7fb2e7e00640 1 -- 172.21.15.57:0/3617361520 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00f620 con 0x7fb2f0075fa0 2024-04-13T05:21:54.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.701+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f019a470 con 0x7fb2f0075fa0 2024-04-13T05:21:54.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.705+0000 7fb2f95a2740 1 Processor -- start 2024-04-13T05:21:54.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.705+0000 7fb2f95a2740 1 -- start start 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f535870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535e40 0x55f81f53a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f53d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55f81f48a020 con 0x55f81f53afb0 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f81f430950 con 0x55f81f535450 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f81f485fe0 con 0x55f81f535e40 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f535870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f535870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46648/0 (socket says 172.21.15.57:46648) 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.706+0000 7fb2efe00640 1 -- 172.21.15.57:0/2423249546 learned_addr learned my addr 172.21.15.57:0/2423249546 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.707+0000 7fb2ef400640 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535e40 0x55f81f53a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.707+0000 7fb2e7e00640 1 -- 172.21.15.57:0/3617361520 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb2dc0109a0 con 0x7fb2f0075fa0 2024-04-13T05:21:54.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.707+0000 7fb2efe00640 1 -- 172.21.15.57:0/2423249546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535e40 msgr2=0x55f81f53a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.707+0000 7fb2efe00640 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535e40 0x55f81f53a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.707+0000 7fb2efe00640 1 -- 172.21.15.57:0/2423249546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 msgr2=0x55f81f53d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.707+0000 7fb2eea00640 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f53d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.708+0000 7fb2efe00640 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f53d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.708+0000 7fb2efe00640 1 -- 172.21.15.57:0/2423249546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81f19c3f0 con 0x55f81f535450 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.708+0000 7fb2eea00640 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f53d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:54.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.708+0000 7fb2efe00640 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f535870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8007e50 tx=0x7fb2e8030730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed1c27737257473d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.709+0000 7fb2e4c00640 1 -- 172.21.15.57:0/2423249546 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8036070 con 0x55f81f535450 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.709+0000 7fb2e4c00640 1 -- 172.21.15.57:0/2423249546 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2e8004d20 con 0x55f81f535450 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.709+0000 7fb2e4c00640 1 -- 172.21.15.57:0/2423249546 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8005480 con 0x55f81f535450 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.709+0000 7fb2e7e00640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb2d40a3bb0 0x7fb2d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.709+0000 7fb2e7e00640 1 -- 172.21.15.57:0/3617361520 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb2dc0d04e0 con 0x7fb2f0075fa0 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.709+0000 7fb2ef400640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb2d40a3bb0 0x7fb2d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 -- 172.21.15.57:0/2423249546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 msgr2=0x55f81f535870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f535870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8007e50 tx=0x7fb2e8030730 comp rx=0 tx=0).stop 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 -- 172.21.15.57:0/2423249546 shutdown_connections 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f53d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535e40 0x55f81f53a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/2423249546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f535870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 -- 172.21.15.57:0/2423249546 >> 172.21.15.57:0/2423249546 conn(0x55f81f51f9b0 msgr2=0x55f81f51fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2ef400640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb2d40a3bb0 0x7fb2d40a6070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e00058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.710+0000 7fb2f95a2740 1 -- 172.21.15.57:0/2423249546 shutdown_connections 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.711+0000 7fb2f95a2740 1 -- 172.21.15.57:0/2423249546 wait complete. 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.711+0000 7fb2f95a2740 1 -- start start 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.711+0000 7fb2f95a2740 1 Processor -- start 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 0x55f81f525e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f5263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 0x55f81f52d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55f81f524340 con 0x55f81f535450 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f81f443c80 con 0x55f81f535e40 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f81f5244c0 con 0x55f81f53afb0 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2f95a2740 10 client.0 _ll_register_callbacks cb 0x55f81f443670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2ef400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 0x55f81f525e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2ef400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 0x55f81f525e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53078/0 (socket says 172.21.15.57:53078) 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.712+0000 7fb2ef400640 1 -- 172.21.15.57:0/1449681901 learned_addr learned my addr 172.21.15.57:0/1449681901 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2eea00640 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f5263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2ef400640 1 -- 172.21.15.57:0/1449681901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 msgr2=0x55f81f52d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2ef400640 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 0x55f81f52d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2ef400640 1 -- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 msgr2=0x55f81f5263a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2ef400640 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f5263a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2ef400640 1 -- 172.21.15.57:0/1449681901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81f2667d0 con 0x55f81f535450 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2ef400640 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 0x55f81f525e60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3909fb4c99ec09 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.713+0000 7fb2d9600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.714+0000 7fb2d9600640 1 -- 172.21.15.57:0/1449681901 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e004e020 con 0x55f81f535450 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.714+0000 7fb2d9600640 1 -- 172.21.15.57:0/1449681901 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb2e00046c0 con 0x55f81f535450 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.714+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e8004d20 con 0x55f81f535450 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.714+0000 7fb2d9600640 1 -- 172.21.15.57:0/1449681901 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2e0059050 con 0x55f81f535450 2024-04-13T05:21:54.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.714+0000 7fb2f95a2740 10 client.16329 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.714+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f81f530730 con 0x55f81f535450 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2d9600640 1 -- 172.21.15.57:0/1449681901 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2e8004d20 con 0x55f81f535450 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2d9600640 1 -- 172.21.15.57:0/1449681901 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fb2e0003b60 con 0x55f81f535450 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2f95a2740 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2f95a2740 10 client.16329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2f95a2740 20 client.16329 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f531ec0 0x55f81f534360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2f95a2740 4 client.16329 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.715+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f81f429750 con 0x55f81f531ec0 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.716+0000 7fb2eea00640 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f531ec0 0x55f81f534360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.716+0000 7fb2eea00640 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f531ec0 0x55f81f534360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=368695c0a189865f server_cookie=dc6933268b98ef34 in_seq=0 out_seq=0 2024-04-13T05:21:54.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.716+0000 7fb2d9600640 10 client.16329 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.719+0000 7fb2d9600640 1 -- 172.21.15.57:0/1449681901 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f81f429750 con 0x55f81f531ec0 2024-04-13T05:21:54.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.719+0000 7fb2d9600640 10 client.16329 handle_command_reply: tid=0 2024-04-13T05:21:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.719+0000 7fb2f95a2740 1 client.16329 shutdown 2024-04-13T05:21:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.719+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 msgr2=0x55f81f525e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.719+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 0x55f81f525e60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 shutdown_connections 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f531ec0 0x55f81f534360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f5263a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 0x55f81f52d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/1449681901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535450 0x55f81f525e60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2d9600640 0 client.16329 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 >> 172.21.15.57:0/1449681901 conn(0x55f81f51f9b0 msgr2=0x55f81f51ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 shutdown_connections 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 1 -- 172.21.15.57:0/1449681901 wait complete. 2024-04-13T05:21:54.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.720+0000 7fb2f95a2740 20 client.16329 trim_cache size 0 max 16384 2024-04-13T05:21:54.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.746+0000 7fb2f95a2740 1 Processor -- start 2024-04-13T05:21:54.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.747+0000 7fb2f95a2740 1 -- start start 2024-04-13T05:21:54.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.747+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f5368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.747+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 0x55f81f52d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 0x55f81f525a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0070380 con 0x55f81f535e40 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006dbf0 con 0x55f81f535450 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006ba60 con 0x55f81f53afb0 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 0x55f81f52d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 0x55f81f52d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53092/0 (socket says 172.21.15.57:53092) 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 -- 172.21.15.57:0/707295308 learned_addr learned my addr 172.21.15.57:0/707295308 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2ef400640 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f5368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 -- 172.21.15.57:0/707295308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 msgr2=0x55f81f525a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 0x55f81f525a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 -- 172.21.15.57:0/707295308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 msgr2=0x55f81f5368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f5368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.748+0000 7fb2eea00640 1 -- 172.21.15.57:0/707295308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81f520400 con 0x55f81f535e40 2024-04-13T05:21:54.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.749+0000 7fb2eea00640 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 0x55f81f52d3a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc098820 tx=0x7fb2dc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d25226c8d5b3110 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.749+0000 7fb2dbe00640 1 -- 172.21.15.57:0/707295308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0b0e60 con 0x55f81f535e40 2024-04-13T05:21:54.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.749+0000 7fb2dbe00640 1 -- 172.21.15.57:0/707295308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc0a0770 con 0x55f81f535e40 2024-04-13T05:21:54.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.749+0000 7fb2dbe00640 1 -- 172.21.15.57:0/707295308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc020040 con 0x55f81f535e40 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 -- 172.21.15.57:0/707295308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 msgr2=0x55f81f52d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 0x55f81f52d3a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc098820 tx=0x7fb2dc00bf60 comp rx=0 tx=0).stop 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 -- 172.21.15.57:0/707295308 shutdown_connections 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f53afb0 0x55f81f525a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f535e40 0x55f81f52d3a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/707295308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535450 0x55f81f5368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 -- 172.21.15.57:0/707295308 >> 172.21.15.57:0/707295308 conn(0x55f81f51f9b0 msgr2=0x55f81f51ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 -- 172.21.15.57:0/707295308 shutdown_connections 2024-04-13T05:21:54.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.750+0000 7fb2f95a2740 1 -- 172.21.15.57:0/707295308 wait complete. 2024-04-13T05:21:54.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.751+0000 7fb2f95a2740 1 -- start start 2024-04-13T05:21:54.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.751+0000 7fb2f95a2740 1 Processor -- start 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535450 0x55f81f529ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f52a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f59f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006c0b0 con 0x55f81f53afb0 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006d520 con 0x55f81f535e40 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006c780 con 0x55f81f535450 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2f95a2740 10 client.0 _ll_register_callbacks cb 0x7fb2e002de20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f59f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f59f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53100/0 (socket says 172.21.15.57:53100) 2024-04-13T05:21:54.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2efe00640 1 -- 172.21.15.57:0/618319309 learned_addr learned my addr 172.21.15.57:0/618319309 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2eea00640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f52a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.752+0000 7fb2ef400640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535450 0x55f81f529ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2efe00640 1 -- 172.21.15.57:0/618319309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535450 msgr2=0x55f81f529ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2efe00640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535450 0x55f81f529ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2efe00640 1 -- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 msgr2=0x55f81f52a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2efe00640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f52a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2efe00640 1 -- 172.21.15.57:0/618319309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81f52d5b0 con 0x55f81f53afb0 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2efe00640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f59f5b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8033da0 tx=0x7fb2e8033dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55869c731af97091 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:54.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.753+0000 7fb2d9600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:54.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.754+0000 7fb2d9600640 1 -- 172.21.15.57:0/618319309 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8048060 con 0x55f81f53afb0 2024-04-13T05:21:54.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.754+0000 7fb2d9600640 1 -- 172.21.15.57:0/618319309 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb2e8007480 con 0x55f81f53afb0 2024-04-13T05:21:54.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.754+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc0a0770 con 0x55f81f53afb0 2024-04-13T05:21:54.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.754+0000 7fb2d9600640 1 -- 172.21.15.57:0/618319309 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2e804f050 con 0x55f81f53afb0 2024-04-13T05:21:54.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.755+0000 7fb2f95a2740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:54.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.755+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f81f5a0a70 con 0x55f81f53afb0 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.755+0000 7fb2d9600640 1 -- 172.21.15.57:0/618319309 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc0a0770 con 0x55f81f53afb0 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2d9600640 1 -- 172.21.15.57:0/618319309 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fb2e80053f0 con 0x55f81f53afb0 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2f95a2740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2f95a2740 10 client.16335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2f95a2740 20 client.16335 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f5a1110 0x55f81f5a3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2f95a2740 4 client.16335 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:21:54.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f81f497a40 con 0x55f81f5a1110 2024-04-13T05:21:54.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.756+0000 7fb2ef400640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f5a1110 0x55f81f5a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:54.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.757+0000 7fb2ef400640 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f5a1110 0x55f81f5a3500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb8697b3b48c15a4 server_cookie=ce314844d23d96f0 in_seq=0 out_seq=0 2024-04-13T05:21:54.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.757+0000 7fb2d9600640 10 client.16335 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:54.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2d9600640 1 -- 172.21.15.57:0/618319309 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f81f497a40 con 0x55f81f5a1110 2024-04-13T05:21:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2d9600640 10 client.16335 handle_command_reply: tid=0 2024-04-13T05:21:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 client.16335 shutdown 2024-04-13T05:21:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 msgr2=0x55f81f59f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f59f5b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8033da0 tx=0x7fb2e8033dd0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 shutdown_connections 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55f81f5a1110 0x55f81f5a3500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f81f535e40 0x55f81f52a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f81f53afb0 0x55f81f59f5b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 --2- 172.21.15.57:0/618319309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81f535450 0x55f81f529ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 >> 172.21.15.57:0/618319309 conn(0x55f81f51f9b0 msgr2=0x55f81f536840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.758+0000 7fb2d9600640 0 client.16335 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.759+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 shutdown_connections 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.759+0000 7fb2f95a2740 1 -- 172.21.15.57:0/618319309 wait complete. 2024-04-13T05:21:54.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.759+0000 7fb2f95a2740 20 client.16335 trim_cache size 0 max 16384 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb2d40a3bb0 msgr2=0x7fb2d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb2d40a3bb0 0x7fb2d40a6070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e00058b0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 msgr2=0x7fb2f01057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 0x7fb2f01057b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00d950 tx=0x7fb2dc00de10 comp rx=0 tx=0).stop 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 shutdown_connections 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb2d40a3bb0 0x7fb2d40a6070 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e00058b0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f0075fa0 0x7fb2f01057b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f0109d80 0x7fb2f0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 --2- 172.21.15.57:0/3617361520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2f00749c0 0x7fb2f0105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 >> 172.21.15.57:0/3617361520 conn(0x7fb2f00fd460 msgr2=0x7fb2f0105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 shutdown_connections 2024-04-13T05:21:54.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:54.760+0000 7fb2f5000640 1 -- 172.21.15.57:0/3617361520 wait complete. 2024-04-13T05:21:54.783 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:21:55.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 -- 172.21.15.57:0/3430583445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810104240 msgr2=0x7f8810104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 --2- 172.21.15.57:0/3430583445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810104240 0x7f8810104660 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8804009900 tx=0x7f880402bc30 comp rx=0 tx=0).stop 2024-04-13T05:21:55.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 -- 172.21.15.57:0/3430583445 shutdown_connections 2024-04-13T05:21:55.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 --2- 172.21.15.57:0/3430583445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810105df0 0x7f881010c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 --2- 172.21.15.57:0/3430583445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8810105430 0x7f88101058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 --2- 172.21.15.57:0/3430583445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810104240 0x7f8810104660 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.043+0000 7f8817c00640 1 -- 172.21.15.57:0/3430583445 >> 172.21.15.57:0/3430583445 conn(0x7f88100ff7b0 msgr2=0x7f8810101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:55.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.044+0000 7f8817c00640 1 -- 172.21.15.57:0/3430583445 shutdown_connections 2024-04-13T05:21:55.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.044+0000 7f8817c00640 1 -- 172.21.15.57:0/3430583445 wait complete. 2024-04-13T05:21:55.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.045+0000 7f8817c00640 1 Processor -- start 2024-04-13T05:21:55.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.046+0000 7f8817c00640 1 -- start start 2024-04-13T05:21:55.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.046+0000 7f8817c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810104240 0x7f88100709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.046+0000 7f8817c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8810105430 0x7f8810070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f8817c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810105df0 0x7f88101aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f8817c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f881010fdd0 con 0x7f8810105df0 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f8815200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810104240 0x7f88100709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f8815200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810104240 0x7f88100709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47234/0 (socket says 172.21.15.57:47234) 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f8815200640 1 -- 172.21.15.57:0/600279624 learned_addr learned my addr 172.21.15.57:0/600279624 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f8815c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810105df0 0x7f88101aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.047+0000 7f880fe00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8810105430 0x7f8810070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f881010fc50 con 0x7f8810104240 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f881010ff50 con 0x7f8810105430 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8815c00640 1 -- 172.21.15.57:0/600279624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8810105430 msgr2=0x7f8810070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8815c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8810105430 0x7f8810070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8815c00640 1 -- 172.21.15.57:0/600279624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810104240 msgr2=0x7f88100709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8815c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810104240 0x7f88100709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.048+0000 7f8815c00640 1 -- 172.21.15.57:0/600279624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88101aaa00 con 0x7f8810105df0 2024-04-13T05:21:55.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.049+0000 7f8815c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810105df0 0x7f88101aa4c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f880800e940 tx=0x7f880800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:55.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.049+0000 7f880d600640 1 -- 172.21.15.57:0/600279624 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808004d20 con 0x7f8810105df0 2024-04-13T05:21:55.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.049+0000 7f880d600640 1 -- 172.21.15.57:0/600279624 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8808005020 con 0x7f8810105df0 2024-04-13T05:21:55.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.049+0000 7f880d600640 1 -- 172.21.15.57:0/600279624 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080116d0 con 0x7f8810105df0 2024-04-13T05:21:55.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.049+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88101aabd0 con 0x7f8810105df0 2024-04-13T05:21:55.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.050+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101ab0a0 con 0x7f8810105df0 2024-04-13T05:21:55.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.054+0000 7f881ab84740 1 Processor -- start 2024-04-13T05:21:55.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.055+0000 7f880d600640 1 -- 172.21.15.57:0/600279624 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f880800cb20 con 0x7f8810105df0 2024-04-13T05:21:55.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 -- start start 2024-04-13T05:21:55.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6af5c0 0x55d99b6b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d99b607020 con 0x55d99b6aa450 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d99b5ad950 con 0x55d99b6a9a60 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f881ab84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d99b602fe0 con 0x55d99b6af5c0 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f8815200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f880fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f8815200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48886/0 (socket says 172.21.15.57:48886) 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.056+0000 7f8815200640 1 -- 172.21.15.57:0/3510597314 learned_addr learned my addr 172.21.15.57:0/3510597314 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.057+0000 7f880fe00640 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6af5c0 0x55d99b6b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.057+0000 7f880fe00640 1 -- 172.21.15.57:0/3510597314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6af5c0 msgr2=0x55d99b6b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.057+0000 7f880fe00640 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6af5c0 0x55d99b6b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.057+0000 7f880fe00640 1 -- 172.21.15.57:0/3510597314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 msgr2=0x55d99b6aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.057+0000 7f880fe00640 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.057+0000 7f880fe00640 1 -- 172.21.15.57:0/3510597314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99b3193f0 con 0x55d99b6a9a60 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.058+0000 7f880fe00640 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6a9e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00cc00 tx=0x7f87fc00c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6d5d92ec7f69d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.058+0000 7f880d600640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f80a3d30 0x7f87f80a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.058+0000 7f880d600640 1 -- 172.21.15.57:0/600279624 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f88080d07f0 con 0x7f8810105df0 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.058+0000 7f8802000640 1 -- 172.21.15.57:0/3510597314 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00f680 con 0x55d99b6a9a60 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.058+0000 7f8802000640 1 -- 172.21.15.57:0/3510597314 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87fc00fcd0 con 0x55d99b6a9a60 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.058+0000 7f8802000640 1 -- 172.21.15.57:0/3510597314 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc011cb0 con 0x55d99b6a9a60 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f8815200640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f80a3d30 0x7f87f80a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 -- 172.21.15.57:0/3510597314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 msgr2=0x55d99b6a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6a9e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00cc00 tx=0x7f87fc00c610 comp rx=0 tx=0).stop 2024-04-13T05:21:55.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 -- 172.21.15.57:0/3510597314 shutdown_connections 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6af5c0 0x55d99b6b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 --2- 172.21.15.57:0/3510597314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6a9e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.059+0000 7f881ab84740 1 -- 172.21.15.57:0/3510597314 >> 172.21.15.57:0/3510597314 conn(0x55d99b6a4fd0 msgr2=0x55d99b6a53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.060+0000 7f8815200640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f80a3d30 0x7f87f80a61f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8804005a90 tx=0x7f8804005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.061+0000 7f881ab84740 1 -- 172.21.15.57:0/3510597314 shutdown_connections 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.061+0000 7f881ab84740 1 -- 172.21.15.57:0/3510597314 wait complete. 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.061+0000 7f881ab84740 1 -- start start 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.061+0000 7f881ab84740 1 Processor -- start 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6af5c0 0x55d99b6bb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d99b6b2570 con 0x55d99b6aa450 2024-04-13T05:21:55.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d99b5c0c80 con 0x55d99b6af5c0 2024-04-13T05:21:55.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d99b6b26f0 con 0x55d99b6a9a60 2024-04-13T05:21:55.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f881ab84740 10 client.0 _ll_register_callbacks cb 0x55d99b5c0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.062+0000 7f880fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48898/0 (socket says 172.21.15.57:48898) 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f8815200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 -- 172.21.15.57:0/542778601 learned_addr learned my addr 172.21.15.57:0/542778601 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f8815c00640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6af5c0 0x55d99b6bb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 -- 172.21.15.57:0/542778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 msgr2=0x55d99b6b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 -- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6af5c0 msgr2=0x55d99b6bb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6af5c0 0x55d99b6bb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 -- 172.21.15.57:0/542778601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99b3e37d0 con 0x55d99b6aa450 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f8815c00640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6af5c0 0x55d99b6bb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f880fe00640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6b4570 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f87fc005d00 tx=0x7f87fc00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958e2260f6aff7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.063+0000 7f87f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.064+0000 7f87f6a00640 1 -- 172.21.15.57:0/542778601 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc005050 con 0x55d99b6aa450 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.064+0000 7f87f6a00640 1 -- 172.21.15.57:0/542778601 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f87fc0051f0 con 0x55d99b6aa450 2024-04-13T05:21:55.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.064+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87fc00fcd0 con 0x55d99b6aa450 2024-04-13T05:21:55.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.064+0000 7f87f6a00640 1 -- 172.21.15.57:0/542778601 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87fc0053f0 con 0x55d99b6aa450 2024-04-13T05:21:55.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.064+0000 7f881ab84740 10 client.16347 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.064+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d99b6be970 con 0x55d99b6aa450 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f87f6a00640 1 -- 172.21.15.57:0/542778601 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87fc00fcd0 con 0x55d99b6aa450 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f87f6a00640 1 -- 172.21.15.57:0/542778601 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f87fc004850 con 0x55d99b6aa450 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f881ab84740 10 client.16347 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f881ab84740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f881ab84740 20 client.16347 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f881ab84740 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b6bfdd0 0x55d99b6c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f881ab84740 4 client.16347 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.065+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d99b5a6750 con 0x55d99b6bfdd0 2024-04-13T05:21:55.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.066+0000 7f8815200640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b6bfdd0 0x55d99b6c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.067+0000 7f87f6a00640 10 client.16347 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:55.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.067+0000 7f8815200640 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b6bfdd0 0x55d99b6c21c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f266bed8a7306a server_cookie=70f3b3855b50f14 in_seq=0 out_seq=0 2024-04-13T05:21:55.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f87f6a00640 1 -- 172.21.15.57:0/542778601 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d99b5a6750 con 0x55d99b6bfdd0 2024-04-13T05:21:55.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f87f6a00640 10 client.16347 handle_command_reply: tid=0 2024-04-13T05:21:55.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 client.16347 shutdown 2024-04-13T05:21:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 msgr2=0x55d99b6b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6b4570 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f87fc005d00 tx=0x7f87fc00ff20 comp rx=0 tx=0).stop 2024-04-13T05:21:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 shutdown_connections 2024-04-13T05:21:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b6bfdd0 0x55d99b6c21c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6af5c0 0x55d99b6bb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6aa450 0x55d99b6b4570 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 --2- 172.21.15.57:0/542778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 >> 172.21.15.57:0/542778601 conn(0x55d99b6a4fd0 msgr2=0x55d99b69ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:55.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.070+0000 7f87f6a00640 0 client.16347 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:55.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.071+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 shutdown_connections 2024-04-13T05:21:55.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.071+0000 7f881ab84740 1 -- 172.21.15.57:0/542778601 wait complete. 2024-04-13T05:21:55.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.071+0000 7f881ab84740 20 client.16347 trim_cache size 0 max 16384 2024-04-13T05:21:55.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 Processor -- start 2024-04-13T05:21:55.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 -- start start 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6aa450 0x55d99b6bb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b69b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f881019dd20 con 0x55d99b6af5c0 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f881019b590 con 0x55d99b6a9a60 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.095+0000 7f881ab84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8810199400 con 0x55d99b6aa450 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.096+0000 7f8815200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.096+0000 7f8815200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47268/0 (socket says 172.21.15.57:47268) 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.096+0000 7f8815200640 1 -- 172.21.15.57:0/4007859895 learned_addr learned my addr 172.21.15.57:0/4007859895 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.096+0000 7f880fe00640 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6aa450 0x55d99b6bb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.096+0000 7f8815c00640 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b69b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.097+0000 7f8815200640 1 -- 172.21.15.57:0/4007859895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6aa450 msgr2=0x55d99b6bb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.097+0000 7f8815200640 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6aa450 0x55d99b6bb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.097+0000 7f8815200640 1 -- 172.21.15.57:0/4007859895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 msgr2=0x55d99b69b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.097+0000 7f8815200640 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b69b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.097+0000 7f8815200640 1 -- 172.21.15.57:0/4007859895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99b6bca30 con 0x55d99b6a9a60 2024-04-13T05:21:55.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.097+0000 7f8815200640 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6b3650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f88040096c0 tx=0x7f8804044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99a614e466e69d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:55.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.098+0000 7f8801600640 1 -- 172.21.15.57:0/4007859895 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004380 con 0x55d99b6a9a60 2024-04-13T05:21:55.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.098+0000 7f8801600640 1 -- 172.21.15.57:0/4007859895 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8804055860 con 0x55d99b6a9a60 2024-04-13T05:21:55.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.098+0000 7f8801600640 1 -- 172.21.15.57:0/4007859895 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804005670 con 0x55d99b6a9a60 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 -- 172.21.15.57:0/4007859895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 msgr2=0x55d99b6b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6b3650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f88040096c0 tx=0x7f8804044000 comp rx=0 tx=0).stop 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 -- 172.21.15.57:0/4007859895 shutdown_connections 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b69b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6aa450 0x55d99b6bb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 --2- 172.21.15.57:0/4007859895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6a9a60 0x55d99b6b3650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 -- 172.21.15.57:0/4007859895 >> 172.21.15.57:0/4007859895 conn(0x55d99b6a4fd0 msgr2=0x55d99b6b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 -- 172.21.15.57:0/4007859895 shutdown_connections 2024-04-13T05:21:55.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.099+0000 7f881ab84740 1 -- 172.21.15.57:0/4007859895 wait complete. 2024-04-13T05:21:55.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.100+0000 7f881ab84740 1 -- start start 2024-04-13T05:21:55.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.100+0000 7f881ab84740 1 Processor -- start 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6aa450 0x55d99b6b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b6ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8810199a50 con 0x55d99b6af5c0 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f881019aec0 con 0x55d99b6aa450 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f881019a120 con 0x55d99b6a9a60 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f881ab84740 10 client.0 _ll_register_callbacks cb 0x7f87fc011e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:55.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.101+0000 7f8815c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b6ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f880fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6aa450 0x55d99b6b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b6ba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48914/0 (socket says 172.21.15.57:48914) 2024-04-13T05:21:55.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 -- 172.21.15.57:0/2114819968 learned_addr learned my addr 172.21.15.57:0/2114819968 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:55.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815200640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 -- 172.21.15.57:0/2114819968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 msgr2=0x55d99b6b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 -- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6aa450 msgr2=0x55d99b6b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:55.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6aa450 0x55d99b6b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 -- 172.21.15.57:0/2114819968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99b6bacd0 con 0x55d99b6af5c0 2024-04-13T05:21:55.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815200640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:21:55.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f8815c00640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b6ba6e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f88080096e0 tx=0x7f880809c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0d9620c5c2b321 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:55.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.102+0000 7f87f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:55.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.103+0000 7f87f6a00640 1 -- 172.21.15.57:0/2114819968 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080b2030 con 0x55d99b6af5c0 2024-04-13T05:21:55.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.103+0000 7f87f6a00640 1 -- 172.21.15.57:0/2114819968 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f88080054d0 con 0x55d99b6af5c0 2024-04-13T05:21:55.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.103+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8804055860 con 0x55d99b6af5c0 2024-04-13T05:21:55.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.104+0000 7f87f6a00640 1 -- 172.21.15.57:0/2114819968 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f880801f070 con 0x55d99b6af5c0 2024-04-13T05:21:55.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.104+0000 7f881ab84740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:55.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.104+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d99b7267c0 con 0x55d99b6af5c0 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.104+0000 7f87f6a00640 1 -- 172.21.15.57:0/2114819968 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8804055860 con 0x55d99b6af5c0 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.104+0000 7f87f6a00640 1 -- 172.21.15.57:0/2114819968 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f880800f040 con 0x55d99b6af5c0 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.105+0000 7f881ab84740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.105+0000 7f881ab84740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.105+0000 7f881ab84740 20 client.16353 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.105+0000 7f881ab84740 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b727170 0x55d99b729590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.105+0000 7f881ab84740 4 client.16353 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.105+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d99b6b3e20 con 0x55d99b727170 2024-04-13T05:21:55.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.106+0000 7f8815200640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b727170 0x55d99b729590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:55.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.107+0000 7f8815200640 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b727170 0x55d99b729590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ea5747746a9131 server_cookie=826c3699933aa632 in_seq=0 out_seq=0 2024-04-13T05:21:55.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.107+0000 7f87f6a00640 10 client.16353 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:55.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.107+0000 7f87f6a00640 1 -- 172.21.15.57:0/2114819968 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d99b6b3e20 con 0x55d99b727170 2024-04-13T05:21:55.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.107+0000 7f87f6a00640 10 client.16353 handle_command_reply: tid=0 2024-04-13T05:21:55.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.107+0000 7f881ab84740 1 client.16353 shutdown 2024-04-13T05:21:55.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 msgr2=0x55d99b6ba6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b6ba6e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f88080096e0 tx=0x7f880809c0f0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 shutdown_connections 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d99b727170 0x55d99b729590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d99b6aa450 0x55d99b6b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d99b6af5c0 0x55d99b6ba6e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 --2- 172.21.15.57:0/2114819968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d99b6a9a60 0x55d99b6b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 >> 172.21.15.57:0/2114819968 conn(0x55d99b6a4fd0 msgr2=0x55d99b6a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.108+0000 7f87f6a00640 0 client.16353 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.109+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 shutdown_connections 2024-04-13T05:21:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.109+0000 7f881ab84740 1 -- 172.21.15.57:0/2114819968 wait complete. 2024-04-13T05:21:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.109+0000 7f881ab84740 20 client.16353 trim_cache size 0 max 16384 2024-04-13T05:21:55.112 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:21:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.110+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f80a3d30 msgr2=0x7f87f80a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.110+0000 7f8817c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f80a3d30 0x7f87f80a61f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8804005a90 tx=0x7f8804005a20 comp rx=0 tx=0).stop 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.110+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810105df0 msgr2=0x7f88101aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.110+0000 7f8817c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810105df0 0x7f88101aa4c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f880800e940 tx=0x7f880800ee00 comp rx=0 tx=0).stop 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 shutdown_connections 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f80a3d30 0x7f87f80a61f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8810105df0 0x7f88101aa4c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8810105430 0x7f8810070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 --2- 172.21.15.57:0/600279624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8810104240 0x7f88100709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 >> 172.21.15.57:0/600279624 conn(0x7f88100ff7b0 msgr2=0x7f88101087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 shutdown_connections 2024-04-13T05:21:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:55.111+0000 7f8817c00640 1 -- 172.21.15.57:0/600279624 wait complete. 2024-04-13T05:21:55.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:55 smithi083 ceph-mon[35133]: pgmap v585: 129 pgs: 129 active+clean; 181 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 229 op/s 2024-04-13T05:21:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:55 smithi057 ceph-mon[32331]: pgmap v585: 129 pgs: 129 active+clean; 181 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 229 op/s 2024-04-13T05:21:55.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:55 smithi189 ceph-mon[35531]: pgmap v585: 129 pgs: 129 active+clean; 181 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 229 op/s 2024-04-13T05:21:56.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:21:56.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:21:56.137 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 -- 172.21.15.57:0/2677655925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c109e90 msgr2=0x7f9a4c10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 --2- 172.21.15.57:0/2677655925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c109e90 0x7f9a4c10c330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ca0 tx=0x7f9a4402c180 comp rx=0 tx=0).stop 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 -- 172.21.15.57:0/2677655925 shutdown_connections 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 --2- 172.21.15.57:0/2677655925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c109e90 0x7f9a4c10c330 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 --2- 172.21.15.57:0/2677655925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 --2- 172.21.15.57:0/2677655925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4c075fa0 0x7f9a4c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 -- 172.21.15.57:0/2677655925 >> 172.21.15.57:0/2677655925 conn(0x7f9a4c0fd570 msgr2=0x7f9a4c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.411+0000 7f9a52c00640 1 -- 172.21.15.57:0/2677655925 shutdown_connections 2024-04-13T05:21:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.412+0000 7f9a52c00640 1 -- 172.21.15.57:0/2677655925 wait complete. 2024-04-13T05:21:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 Processor -- start 2024-04-13T05:21:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 -- start start 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a4c075fa0 0x7f9a4c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4c109e90 0x7f9a4c1a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c10fd40 con 0x7f9a4c0749c0 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c10fbc0 con 0x7f9a4c109e90 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.413+0000 7f9a52c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c10fec0 con 0x7f9a4c075fa0 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4c109e90 0x7f9a4c1a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48936/0 (socket says 172.21.15.57:48936) 2024-04-13T05:21:56.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 -- 172.21.15.57:0/1797534249 learned_addr learned my addr 172.21.15.57:0/1797534249 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 -- 172.21.15.57:0/1797534249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a4c075fa0 msgr2=0x7f9a4c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a4c075fa0 0x7f9a4c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 -- 172.21.15.57:0/1797534249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4c109e90 msgr2=0x7f9a4c1a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4c109e90 0x7f9a4c1a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.414+0000 7f9a4be00640 1 -- 172.21.15.57:0/1797534249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c105b60 con 0x7f9a4c0749c0 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.415+0000 7f9a4be00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c19a420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ea60 tx=0x7f9a3c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.415+0000 7f9a48c00640 1 -- 172.21.15.57:0/1797534249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00cd60 con 0x7f9a4c0749c0 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.415+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c1a6760 con 0x7f9a4c0749c0 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.415+0000 7f9a48c00640 1 -- 172.21.15.57:0/1797534249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a3c0035f0 con 0x7f9a4c0749c0 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.415+0000 7f9a48c00640 1 -- 172.21.15.57:0/1797534249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c010690 con 0x7f9a4c0749c0 2024-04-13T05:21:56.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.416+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a6cd0 con 0x7f9a4c0749c0 2024-04-13T05:21:56.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.420+0000 7f9a55b9c740 1 Processor -- start 2024-04-13T05:21:56.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.420+0000 7f9a55b9c740 1 -- start start 2024-04-13T05:21:56.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41a6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b41a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e9b4107bc0 con 0x55e9b41a1280 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9b40ae660 con 0x55e9b41a6de0 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9b4103b80 con 0x55e9b41a1c70 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.421+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.422+0000 7f9a4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b41a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.422+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48944/0 (socket says 172.21.15.57:48944) 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.422+0000 7f9a4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b41a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47310/0 (socket says 172.21.15.57:47310) 2024-04-13T05:21:56.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.422+0000 7f9a50c00640 1 -- 172.21.15.57:0/1688450310 learned_addr learned my addr 172.21.15.57:0/1688450310 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.422+0000 7f9a48c00640 1 -- 172.21.15.57:0/1797534249 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a3c004000 con 0x7f9a4c0749c0 2024-04-13T05:21:56.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a50c00640 1 -- 172.21.15.57:0/1688450310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 msgr2=0x55e9b41a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a4be00640 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a50c00640 1 -- 172.21.15.57:0/1688450310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 msgr2=0x55e9b41a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b41a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a50c00640 1 -- 172.21.15.57:0/1688450310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b40d1220 con 0x55e9b41a1280 2024-04-13T05:21:56.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a4b400640 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b41a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:56.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.423+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41a16a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9a4402c790 tx=0x7f9a44032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e881773775742db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.424+0000 7f9a48c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a300a3c00 0x7f9a300a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.424+0000 7f9a48c00640 1 -- 172.21.15.57:0/1797534249 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9a3c0d0b60 con 0x7f9a4c0749c0 2024-04-13T05:21:56.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.424+0000 7f9a41200640 1 -- 172.21.15.57:0/1688450310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44043070 con 0x55e9b41a1280 2024-04-13T05:21:56.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.424+0000 7f9a41200640 1 -- 172.21.15.57:0/1688450310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a44003e20 con 0x55e9b41a1280 2024-04-13T05:21:56.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.424+0000 7f9a41200640 1 -- 172.21.15.57:0/1688450310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44039040 con 0x55e9b41a1280 2024-04-13T05:21:56.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.425+0000 7f9a4be00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a300a3c00 0x7f9a300a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.425+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1688450310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 msgr2=0x55e9b41a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.425+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41a16a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9a4402c790 tx=0x7f9a44032440 comp rx=0 tx=0).stop 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1688450310 shutdown_connections 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a4be00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a300a3c00 0x7f9a300a60c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00f010 tx=0x7f9a3c003aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b41a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1688450310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41a16a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1688450310 >> 172.21.15.57:0/1688450310 conn(0x55e9b419c7f0 msgr2=0x55e9b419cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1688450310 shutdown_connections 2024-04-13T05:21:56.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.426+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1688450310 wait complete. 2024-04-13T05:21:56.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.427+0000 7f9a55b9c740 1 -- start start 2024-04-13T05:21:56.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.427+0000 7f9a55b9c740 1 Processor -- start 2024-04-13T05:21:56.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 0x55e9b41ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a6de0 0x55e9b41b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e9b41a9d30 con 0x55e9b41a6de0 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9b40c1a30 con 0x55e9b41a1280 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9b41a9eb0 con 0x55e9b41a1c70 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a55b9c740 10 client.0 _ll_register_callbacks cb 0x55e9b3e19c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.428+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 0x55e9b41ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.429+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 0x55e9b41ab7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47320/0 (socket says 172.21.15.57:47320) 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.429+0000 7f9a50c00640 1 -- 172.21.15.57:0/4266283552 learned_addr learned my addr 172.21.15.57:0/4266283552 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.429+0000 7f9a50c00640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a6de0 0x55e9b41b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.429+0000 7f9a4b400640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.430+0000 7f9a4b400640 1 -- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 msgr2=0x55e9b41ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.430+0000 7f9a4b400640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 0x55e9b41ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.430+0000 7f9a4b400640 1 -- 172.21.15.57:0/4266283552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a6de0 msgr2=0x55e9b41b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.430+0000 7f9a4b400640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a6de0 0x55e9b41b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.430+0000 7f9a4b400640 1 -- 172.21.15.57:0/4266283552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b3ee27e0 con 0x55e9b41a1c70 2024-04-13T05:21:56.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.430+0000 7f9a50c00640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 0x55e9b41ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:56.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.431+0000 7f9a4b400640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41abd30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c19b930 tx=0x7f9a38010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685969b4eaf1e09a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.431+0000 7f9a35600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:21:56.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.431+0000 7f9a35600640 1 -- 172.21.15.57:0/4266283552 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38012860 con 0x55e9b41a1c70 2024-04-13T05:21:56.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.431+0000 7f9a35600640 1 -- 172.21.15.57:0/4266283552 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9a38012a00 con 0x55e9b41a1c70 2024-04-13T05:21:56.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.432+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a44003e20 con 0x55e9b41a1c70 2024-04-13T05:21:56.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.432+0000 7f9a35600640 1 -- 172.21.15.57:0/4266283552 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a38012c00 con 0x55e9b41a1c70 2024-04-13T05:21:56.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.432+0000 7f9a55b9c740 10 client.25720 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:56.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.432+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9b3f42f70 con 0x55e9b41a1c70 2024-04-13T05:21:56.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a35600640 1 -- 172.21.15.57:0/4266283552 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a44003e20 con 0x55e9b41a1c70 2024-04-13T05:21:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a35600640 1 -- 172.21.15.57:0/4266283552 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f9a3800f040 con 0x55e9b41a1c70 2024-04-13T05:21:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a55b9c740 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a55b9c740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a55b9c740 20 client.25720 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:56.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b41b7610 0x55e9b41b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a55b9c740 4 client.25720 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.433+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b41155e0 con 0x55e9b41b7610 2024-04-13T05:21:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.434+0000 7f9a50c00640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b41b7610 0x55e9b41b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.435+0000 7f9a35600640 10 client.25720 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.435+0000 7f9a50c00640 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b41b7610 0x55e9b41b9a50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c5e0bc15358335 server_cookie=4f3316dd89794c5d in_seq=0 out_seq=0 2024-04-13T05:21:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.436+0000 7f9a35600640 1 -- 172.21.15.57:0/4266283552 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9b41155e0 con 0x55e9b41b7610 2024-04-13T05:21:56.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.436+0000 7f9a35600640 10 client.25720 handle_command_reply: tid=0 2024-04-13T05:21:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.436+0000 7f9a55b9c740 1 client.25720 shutdown 2024-04-13T05:21:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 msgr2=0x55e9b41abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41abd30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c19b930 tx=0x7f9a38010800 comp rx=0 tx=0).stop 2024-04-13T05:21:56.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 shutdown_connections 2024-04-13T05:21:56.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b41b7610 0x55e9b41b9a50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a6de0 0x55e9b41b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41abd30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/4266283552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1280 0x55e9b41ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 >> 172.21.15.57:0/4266283552 conn(0x55e9b419c7f0 msgr2=0x55e9b419b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a35600640 0 client.25720 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 shutdown_connections 2024-04-13T05:21:56.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.437+0000 7f9a55b9c740 1 -- 172.21.15.57:0/4266283552 wait complete. 2024-04-13T05:21:56.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.438+0000 7f9a55b9c740 20 client.25720 trim_cache size 0 max 16384 2024-04-13T05:21:56.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.464+0000 7f9a55b9c740 1 Processor -- start 2024-04-13T05:21:56.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 -- start start 2024-04-13T05:21:56.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b419b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b3b40 con 0x55e9b41a1280 2024-04-13T05:21:56.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b13b0 con 0x55e9b41a6de0 2024-04-13T05:21:56.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.465+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1af200 con 0x55e9b41a1c70 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b419b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48958/0 (socket says 172.21.15.57:48958) 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 -- 172.21.15.57:0/3425437296 learned_addr learned my addr 172.21.15.57:0/3425437296 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a4b400640 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 -- 172.21.15.57:0/3425437296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 msgr2=0x55e9b41b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 -- 172.21.15.57:0/3425437296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 msgr2=0x55e9b419b6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b419b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.466+0000 7f9a50c00640 1 -- 172.21.15.57:0/3425437296 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b3dda170 con 0x55e9b41a1280 2024-04-13T05:21:56.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.467+0000 7f9a50c00640 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b419b160 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a4402c790 tx=0x7f9a4402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a252856f27ef13 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.468+0000 7f9a37e00640 1 -- 172.21.15.57:0/3425437296 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44039040 con 0x55e9b41a1280 2024-04-13T05:21:56.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.468+0000 7f9a37e00640 1 -- 172.21.15.57:0/3425437296 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a44032b60 con 0x55e9b41a1280 2024-04-13T05:21:56.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.468+0000 7f9a37e00640 1 -- 172.21.15.57:0/3425437296 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44043040 con 0x55e9b41a1280 2024-04-13T05:21:56.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.468+0000 7f9a55b9c740 1 -- 172.21.15.57:0/3425437296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 msgr2=0x55e9b419b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.468+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b419b160 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a4402c790 tx=0x7f9a4402e040 comp rx=0 tx=0).stop 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 -- 172.21.15.57:0/3425437296 shutdown_connections 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a6de0 0x55e9b419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a1c70 0x55e9b41b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/3425437296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b419b160 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 -- 172.21.15.57:0/3425437296 >> 172.21.15.57:0/3425437296 conn(0x55e9b419c7f0 msgr2=0x55e9b41aa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 -- 172.21.15.57:0/3425437296 shutdown_connections 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.469+0000 7f9a55b9c740 1 -- 172.21.15.57:0/3425437296 wait complete. 2024-04-13T05:21:56.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.470+0000 7f9a55b9c740 1 -- start start 2024-04-13T05:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.470+0000 7f9a55b9c740 1 Processor -- start 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1c70 0x55e9b41aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a6de0 0x55e9b41b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1af870 con 0x55e9b41a1280 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1b0ce0 con 0x55e9b41a1c70 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1aff40 con 0x55e9b41a6de0 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.471+0000 7f9a55b9c740 10 client.0 _ll_register_callbacks cb 0x7f9a38005ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.472+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.472+0000 7f9a50c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48964/0 (socket says 172.21.15.57:48964) 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.472+0000 7f9a50c00640 1 -- 172.21.15.57:0/1247151532 learned_addr learned my addr 172.21.15.57:0/1247151532 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.472+0000 7f9a4b400640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1c70 0x55e9b41aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.472+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a6de0 0x55e9b41b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a50c00640 1 -- 172.21.15.57:0/1247151532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a6de0 msgr2=0x55e9b41b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a6de0 0x55e9b41b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a50c00640 1 -- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1c70 msgr2=0x55e9b41aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1c70 0x55e9b41aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a50c00640 1 -- 172.21.15.57:0/1247151532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b40c2be0 con 0x55e9b41a1280 2024-04-13T05:21:56.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a50c00640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41ae6c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a4402e010 tx=0x7f9a44000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb81963103534ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.473+0000 7f9a35600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:56.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.474+0000 7f9a35600640 1 -- 172.21.15.57:0/1247151532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44039070 con 0x55e9b41a1280 2024-04-13T05:21:56.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.474+0000 7f9a35600640 1 -- 172.21.15.57:0/1247151532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9a440560b0 con 0x55e9b41a1280 2024-04-13T05:21:56.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.475+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a44032b60 con 0x55e9b41a1280 2024-04-13T05:21:56.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.475+0000 7f9a35600640 1 -- 172.21.15.57:0/1247151532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a44065050 con 0x55e9b41a1280 2024-04-13T05:21:56.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.475+0000 7f9a55b9c740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:56.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.475+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9b422ecd0 con 0x55e9b41a1280 2024-04-13T05:21:56.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.475+0000 7f9a35600640 1 -- 172.21.15.57:0/1247151532 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a44032b60 con 0x55e9b41a1280 2024-04-13T05:21:56.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a35600640 1 -- 172.21.15.57:0/1247151532 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f9a44056ce0 con 0x55e9b41a1280 2024-04-13T05:21:56.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a55b9c740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:56.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a55b9c740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:56.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a55b9c740 20 client.16371 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:56.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b422f0c0 0x55e9b42314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a55b9c740 4 client.16371 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:21:56.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e9b410f450 con 0x55e9b422f0c0 2024-04-13T05:21:56.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.477+0000 7f9a4b400640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b422f0c0 0x55e9b42314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.478+0000 7f9a35600640 10 client.16371 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.478+0000 7f9a4b400640 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b422f0c0 0x55e9b42314b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d070003d90a695c server_cookie=1b4904984075bb39 in_seq=0 out_seq=0 2024-04-13T05:21:56.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.480+0000 7f9a35600640 1 -- 172.21.15.57:0/1247151532 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e9b410f450 con 0x55e9b422f0c0 2024-04-13T05:21:56.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.480+0000 7f9a35600640 10 client.16371 handle_command_reply: tid=0 2024-04-13T05:21:56.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.480+0000 7f9a55b9c740 1 client.16371 shutdown 2024-04-13T05:21:56.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.480+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 msgr2=0x55e9b41ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.480+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41ae6c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a4402e010 tx=0x7f9a44000f40 comp rx=0 tx=0).stop 2024-04-13T05:21:56.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.480+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 shutdown_connections 2024-04-13T05:21:56.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9b422f0c0 0x55e9b42314b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a35600640 0 client.16371 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b41a1c70 0x55e9b41aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b41a6de0 0x55e9b41b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a55b9c740 1 --2- 172.21.15.57:0/1247151532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b41a1280 0x55e9b41ae6c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 >> 172.21.15.57:0/1247151532 conn(0x55e9b419c7f0 msgr2=0x55e9b419fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.481+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 shutdown_connections 2024-04-13T05:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.482+0000 7f9a55b9c740 1 -- 172.21.15.57:0/1247151532 wait complete. 2024-04-13T05:21:56.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.482+0000 7f9a55b9c740 20 client.16371 trim_cache size 0 max 16384 2024-04-13T05:21:56.485 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:21:56.485 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:21:56.485 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "9bb81912-1843-4cb4-9b87-f3dfd64e0675", 2024-04-13T05:21:56.485 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:21:56.485 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a300a3c00 msgr2=0x7f9a300a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a300a3c00 0x7f9a300a60c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00f010 tx=0x7f9a3c003aa0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 msgr2=0x7f9a4c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c19a420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ea60 tx=0x7f9a3c00ef20 comp rx=0 tx=0).stop 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 shutdown_connections 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a300a3c00 0x7f9a300a60c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a4c109e90 0x7f9a4c1a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a4c075fa0 0x7f9a4c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 --2- 172.21.15.57:0/1797534249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a4c0749c0 0x7f9a4c19a420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.483+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 >> 172.21.15.57:0/1797534249 conn(0x7f9a4c0fd570 msgr2=0x7f9a4c105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.484+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 shutdown_connections 2024-04-13T05:21:56.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.484+0000 7f9a52c00640 1 -- 172.21.15.57:0/1797534249 wait complete. 2024-04-13T05:21:56.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:21:56.510 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:21:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.813+0000 7f1169200640 1 -- 172.21.15.57:0/1494267703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164105dd0 msgr2=0x7f116410c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.813+0000 7f1169200640 1 --2- 172.21.15.57:0/1494267703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164105dd0 0x7f116410c660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1154009900 tx=0x7f115402bc30 comp rx=0 tx=0).stop 2024-04-13T05:21:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.814+0000 7f1169200640 1 -- 172.21.15.57:0/1494267703 shutdown_connections 2024-04-13T05:21:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.814+0000 7f1169200640 1 --2- 172.21.15.57:0/1494267703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164105dd0 0x7f116410c660 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.814+0000 7f1169200640 1 --2- 172.21.15.57:0/1494267703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 0x7f1164105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.814+0000 7f1169200640 1 --2- 172.21.15.57:0/1494267703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104220 0x7f1164104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.814+0000 7f1169200640 1 -- 172.21.15.57:0/1494267703 >> 172.21.15.57:0/1494267703 conn(0x7f11640ff770 msgr2=0x7f1164101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.815+0000 7f1169200640 1 -- 172.21.15.57:0/1494267703 shutdown_connections 2024-04-13T05:21:56.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.815+0000 7f1169200640 1 -- 172.21.15.57:0/1494267703 wait complete. 2024-04-13T05:21:56.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.816+0000 7f1169200640 1 Processor -- start 2024-04-13T05:21:56.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.816+0000 7f1169200640 1 -- start start 2024-04-13T05:21:56.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.817+0000 7f1169200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 0x7f11640709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.817+0000 7f1169200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 0x7f1164070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1169200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164105dd0 0x7f11641aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1169200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f116410fd60 con 0x7f1164105410 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1169200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f116410fbe0 con 0x7f1164104220 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1169200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f116410fee0 con 0x7f1164105dd0 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1161e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 0x7f11640709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1161e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 0x7f11640709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47334/0 (socket says 172.21.15.57:47334) 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1161e00640 1 -- 172.21.15.57:0/1368624164 learned_addr learned my addr 172.21.15.57:0/1368624164 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1161400640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 0x7f1164070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.818+0000 7f1162800640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164105dd0 0x7f11641aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.820+0000 7f1161e00640 1 -- 172.21.15.57:0/1368624164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164105dd0 msgr2=0x7f11641aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.820+0000 7f1161e00640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164105dd0 0x7f11641aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.820+0000 7f1161e00640 1 -- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 msgr2=0x7f1164070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.820+0000 7f1161e00640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 0x7f1164070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.820+0000 7f1161e00640 1 -- 172.21.15.57:0/1368624164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11641aa990 con 0x7f1164104220 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.820+0000 7f1161400640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 0x7f1164070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.821+0000 7f1161e00640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 0x7f11640709a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f115c00eaa0 tx=0x7f115c00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.821+0000 7f115aa00640 1 -- 172.21.15.57:0/1368624164 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c016070 con 0x7f1164104220 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.821+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11641aab60 con 0x7f1164104220 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.821+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11641ab050 con 0x7f1164104220 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.821+0000 7f115aa00640 1 -- 172.21.15.57:0/1368624164 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115c003e20 con 0x7f1164104220 2024-04-13T05:21:56.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.821+0000 7f115aa00640 1 -- 172.21.15.57:0/1368624164 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0116d0 con 0x7f1164104220 2024-04-13T05:21:56.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.827+0000 7f116c06e740 1 Processor -- start 2024-04-13T05:21:56.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.827+0000 7f116c06e740 1 -- start start 2024-04-13T05:21:56.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.828+0000 7f116c06e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.828+0000 7f116c06e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 0x5561cdd379c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.828+0000 7f116c06e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd38030 0x5561cdd3a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.828+0000 7f116c06e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5561cdc99020 con 0x5561cdd38030 2024-04-13T05:21:56.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.828+0000 7f116c06e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561cdc3f6d0 con 0x5561cdd32ec0 2024-04-13T05:21:56.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.828+0000 7f116c06e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561cdc94fe0 con 0x5561cdd324d0 2024-04-13T05:21:56.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f1162800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f1162800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd328f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:52310/0 (socket says 172.21.15.57:52310) 2024-04-13T05:21:56.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f1162800640 1 -- 172.21.15.57:0/4008806659 learned_addr learned my addr 172.21.15.57:0/4008806659 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f115aa00640 1 -- 172.21.15.57:0/1368624164 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f115c019070 con 0x7f1164104220 2024-04-13T05:21:56.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f1162800640 1 -- 172.21.15.57:0/4008806659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 msgr2=0x5561cdd379c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:21:56.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f1162800640 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 0x5561cdd379c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.829+0000 7f1162800640 1 -- 172.21.15.57:0/4008806659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd38030 msgr2=0x5561cdd3a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:21:56.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.830+0000 7f1161400640 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd38030 0x5561cdd3a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.830+0000 7f1162800640 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd38030 0x5561cdd3a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.830+0000 7f1162800640 1 -- 172.21.15.57:0/4008806659 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cda757d0 con 0x5561cdd324d0 2024-04-13T05:21:56.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.830+0000 7f1161400640 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd38030 0x5561cdd3a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:21:56.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.831+0000 7f1162800640 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd328f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f115402c240 tx=0x7f115402c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e041ec2a901f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.832+0000 7f115aa00640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11480a3d00 0x7f11480a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.832+0000 7f1153400640 1 -- 172.21.15.57:0/4008806659 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154045020 con 0x5561cdd324d0 2024-04-13T05:21:56.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.832+0000 7f1153400640 1 -- 172.21.15.57:0/4008806659 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115402ca50 con 0x5561cdd324d0 2024-04-13T05:21:56.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.832+0000 7f1153400640 1 -- 172.21.15.57:0/4008806659 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154004920 con 0x5561cdd324d0 2024-04-13T05:21:56.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.832+0000 7f115aa00640 1 -- 172.21.15.57:0/1368624164 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f115c0d0700 con 0x7f1164104220 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 -- 172.21.15.57:0/4008806659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 msgr2=0x5561cdd328f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd328f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f115402c240 tx=0x7f115402c8e0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f1161e00640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11480a3d00 0x7f11480a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 -- 172.21.15.57:0/4008806659 shutdown_connections 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 0x5561cdd379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd38030 0x5561cdd3a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 --2- 172.21.15.57:0/4008806659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd328f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.833+0000 7f116c06e740 1 -- 172.21.15.57:0/4008806659 >> 172.21.15.57:0/4008806659 conn(0x5561cdd2da40 msgr2=0x5561cdd2de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.834+0000 7f116c06e740 1 -- 172.21.15.57:0/4008806659 shutdown_connections 2024-04-13T05:21:56.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.834+0000 7f1161e00640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11480a3d00 0x7f11480a61c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f115c09c620 tx=0x7f115c0989c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.834+0000 7f116c06e740 1 -- 172.21.15.57:0/4008806659 wait complete. 2024-04-13T05:21:56.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.835+0000 7f116c06e740 1 -- start start 2024-04-13T05:21:56.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.835+0000 7f116c06e740 1 Processor -- start 2024-04-13T05:21:56.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.835+0000 7f116c06e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 0x5561cdd3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.836+0000 7f116c06e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd32ec0 0x5561cdd3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.836+0000 7f116c06e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd38030 0x5561cdd43d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.836+0000 7f116c06e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5561cdd3af80 con 0x5561cdd324d0 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 0x5561cdd3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f116c06e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561cdc52a90 con 0x5561cdd38030 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f116c06e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561cdd3b100 con 0x5561cdd32ec0 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 0x5561cdd3c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48998/0 (socket says 172.21.15.57:48998) 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 -- 172.21.15.57:0/3463494248 learned_addr learned my addr 172.21.15.57:0/3463494248 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f116c06e740 10 client.0 _ll_register_callbacks cb 0x5561cdc52480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 -- 172.21.15.57:0/3463494248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd32ec0 msgr2=0x5561cdd3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd32ec0 0x5561cdd3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 -- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd38030 msgr2=0x5561cdd43d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd38030 0x5561cdd43d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.837+0000 7f1162800640 1 -- 172.21.15.57:0/3463494248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cd9ab3f0 con 0x5561cdd324d0 2024-04-13T05:21:56.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.838+0000 7f1162800640 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 0x5561cdd3c7a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1154036040 tx=0x7f1154005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a00ec979d5698ad server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.838+0000 7f1147e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.838+0000 7f1147e00640 1 -- 172.21.15.57:0/3463494248 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154050070 con 0x5561cdd324d0 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.838+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115402ca50 con 0x5561cdd324d0 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.838+0000 7f1147e00640 1 -- 172.21.15.57:0/3463494248 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1154003b40 con 0x5561cdd324d0 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f1147e00640 1 -- 172.21.15.57:0/3463494248 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1154045030 con 0x5561cdd324d0 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 10 client.16380 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561cd96b0f0 con 0x5561cdd324d0 2024-04-13T05:21:56.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f1147e00640 1 -- 172.21.15.57:0/3463494248 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1154003d40 con 0x5561cdd324d0 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f1147e00640 1 -- 172.21.15.57:0/3463494248 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f1154053020 con 0x5561cdd324d0 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 10 client.16380 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 20 client.16380 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd481b0 0x5561cdd4a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 4 client.16380 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.839+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f115402ca50 con 0x5561cdd481b0 2024-04-13T05:21:56.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.840+0000 7f1161400640 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd481b0 0x5561cdd4a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.841+0000 7f1161400640 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd481b0 0x5561cdd4a5a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=661aafa42b996262 server_cookie=adf89c5b81e2c497 in_seq=0 out_seq=0 2024-04-13T05:21:56.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.841+0000 7f1147e00640 10 client.16380 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.844+0000 7f1147e00640 1 -- 172.21.15.57:0/3463494248 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f115402ca50 con 0x5561cdd481b0 2024-04-13T05:21:56.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.844+0000 7f1147e00640 10 client.16380 handle_command_reply: tid=0 2024-04-13T05:21:56.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.844+0000 7f116c06e740 1 client.16380 shutdown 2024-04-13T05:21:56.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 msgr2=0x5561cdd3c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 0x5561cdd3c7a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1154036040 tx=0x7f1154005650 comp rx=0 tx=0).stop 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 shutdown_connections 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd481b0 0x5561cdd4a5a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd38030 0x5561cdd43d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd32ec0 0x5561cdd3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 --2- 172.21.15.57:0/3463494248 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd324d0 0x5561cdd3c7a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 >> 172.21.15.57:0/3463494248 conn(0x5561cdd2da40 msgr2=0x5561cdd2cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f1147e00640 0 client.16380 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 shutdown_connections 2024-04-13T05:21:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.845+0000 7f116c06e740 1 -- 172.21.15.57:0/3463494248 wait complete. 2024-04-13T05:21:56.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.846+0000 7f116c06e740 20 client.16380 trim_cache size 0 max 16384 2024-04-13T05:21:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.879+0000 7f116c06e740 1 Processor -- start 2024-04-13T05:21:56.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 -- start start 2024-04-13T05:21:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 0x5561cdd3cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 0x5561cdd2c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11400124f0 con 0x5561cdd2da40 2024-04-13T05:21:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f114000fd60 con 0x5561cdd32ec0 2024-04-13T05:21:56.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.880+0000 7f116c06e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f114000dbb0 con 0x5561cdd324d0 2024-04-13T05:21:56.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.881+0000 7f1162800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 0x5561cdd3cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.881+0000 7f1162800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 0x5561cdd3cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49008/0 (socket says 172.21.15.57:49008) 2024-04-13T05:21:56.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.881+0000 7f1162800640 1 -- 172.21.15.57:0/291717926 learned_addr learned my addr 172.21.15.57:0/291717926 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.881+0000 7f1161400640 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.882+0000 7f1162800640 1 -- 172.21.15.57:0/291717926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 msgr2=0x5561cdd43ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.882+0000 7f1162800640 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd43ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.882+0000 7f1162800640 1 -- 172.21.15.57:0/291717926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 msgr2=0x5561cdd2c2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:21:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.882+0000 7f1162800640 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 0x5561cdd2c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.882+0000 7f1162800640 1 -- 172.21.15.57:0/291717926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cdd3b860 con 0x5561cdd2da40 2024-04-13T05:21:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.883+0000 7f1162800640 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 0x5561cdd3cac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f11540096c0 tx=0x7f115402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf827b6dc108c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.883+0000 7f1152a00640 1 -- 172.21.15.57:0/291717926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154050040 con 0x5561cdd2da40 2024-04-13T05:21:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.883+0000 7f1152a00640 1 -- 172.21.15.57:0/291717926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115404aab0 con 0x5561cdd2da40 2024-04-13T05:21:56.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.883+0000 7f1152a00640 1 -- 172.21.15.57:0/291717926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154041ac0 con 0x5561cdd2da40 2024-04-13T05:21:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 -- 172.21.15.57:0/291717926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 msgr2=0x5561cdd3cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 0x5561cdd3cac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f11540096c0 tx=0x7f115402c8e0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 -- 172.21.15.57:0/291717926 shutdown_connections 2024-04-13T05:21:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd32ec0 0x5561cdd2c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd43ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 --2- 172.21.15.57:0/291717926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd2da40 0x5561cdd3cac0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.885+0000 7f116c06e740 1 -- 172.21.15.57:0/291717926 >> 172.21.15.57:0/291717926 conn(0x5561cdd37b00 msgr2=0x5561cdd3c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.886+0000 7f116c06e740 1 -- 172.21.15.57:0/291717926 shutdown_connections 2024-04-13T05:21:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.887+0000 7f116c06e740 1 -- 172.21.15.57:0/291717926 wait complete. 2024-04-13T05:21:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.887+0000 7f116c06e740 1 -- start start 2024-04-13T05:21:56.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.888+0000 7f116c06e740 1 Processor -- start 2024-04-13T05:21:56.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.889+0000 7f116c06e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd2da40 0x5561cdd40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.889+0000 7f116c06e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f1162800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd2da40 0x5561cdd40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f1162800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd2da40 0x5561cdd40f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47360/0 (socket says 172.21.15.57:47360) 2024-04-13T05:21:56.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f116c06e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd32ec0 0x5561cdd42490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f116c06e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f114000e220 con 0x5561cdd32ec0 2024-04-13T05:21:56.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f114000f690 con 0x5561cdd2da40 2024-04-13T05:21:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f1162800640 1 -- 172.21.15.57:0/3527086575 learned_addr learned my addr 172.21.15.57:0/3527086575 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:21:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f114000e8f0 con 0x5561cdd324d0 2024-04-13T05:21:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f116c06e740 10 client.0 _ll_register_callbacks cb 0x7f1154004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:21:56.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.890+0000 7f1161400640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.891+0000 7f1162800640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd32ec0 0x5561cdd42490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.892+0000 7f1161400640 1 -- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd2da40 msgr2=0x5561cdd40f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.892+0000 7f1161400640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd2da40 0x5561cdd40f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.892+0000 7f1161400640 1 -- 172.21.15.57:0/3527086575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd32ec0 msgr2=0x5561cdd42490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:21:56.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.892+0000 7f1161400640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd32ec0 0x5561cdd42490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.892+0000 7f1161400640 1 -- 172.21.15.57:0/3527086575 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cdd42a80 con 0x5561cdd324d0 2024-04-13T05:21:56.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.893+0000 7f1161400640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd41480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5561cdd4ada0 tx=0x7f114c013410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61a64626502d814 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:21:56.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.893+0000 7f1147e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:21:56.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.894+0000 7f1147e00640 1 -- 172.21.15.57:0/3527086575 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c004250 con 0x5561cdd324d0 2024-04-13T05:21:56.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.894+0000 7f1147e00640 1 -- 172.21.15.57:0/3527086575 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f114c0043f0 con 0x5561cdd324d0 2024-04-13T05:21:56.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.895+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115404aab0 con 0x5561cdd324d0 2024-04-13T05:21:56.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.895+0000 7f1147e00640 1 -- 172.21.15.57:0/3527086575 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f114c013620 con 0x5561cdd324d0 2024-04-13T05:21:56.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.895+0000 7f116c06e740 10 client.25738 fetch_fsmap learned FSMap version 28 2024-04-13T05:21:56.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.895+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561cdd43e60 con 0x5561cdd324d0 2024-04-13T05:21:56.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f1147e00640 1 -- 172.21.15.57:0/3527086575 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115404aab0 con 0x5561cdd324d0 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f1147e00640 1 -- 172.21.15.57:0/3527086575 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f114c022030 con 0x5561cdd324d0 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f116c06e740 10 client.25738 fetch_fsmap finished waiting for FSMap version 28 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f116c06e740 10 client.25738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f116c06e740 20 client.25738 populate_metadata read hostname 'smithi057' 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f116c06e740 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd38030 0x5561cddb0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f116c06e740 4 client.25738 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:21:56.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.896+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561cdc38600 con 0x5561cdd38030 2024-04-13T05:21:56.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.897+0000 7f1162800640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd38030 0x5561cddb0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:21:56.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.898+0000 7f1147e00640 10 client.25738 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.898+0000 7f1162800640 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd38030 0x5561cddb0c60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be6170eee510e42 server_cookie=9c855093f6c772e0 in_seq=0 out_seq=0 2024-04-13T05:21:56.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.899+0000 7f1147e00640 1 -- 172.21.15.57:0/3527086575 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5561cdc38600 con 0x5561cdd38030 2024-04-13T05:21:56.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.899+0000 7f1147e00640 10 client.25738 handle_command_reply: tid=0 2024-04-13T05:21:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.899+0000 7f116c06e740 1 client.25738 shutdown 2024-04-13T05:21:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 msgr2=0x5561cdd41480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd41480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5561cdd4ada0 tx=0x7f114c013410 comp rx=0 tx=0).stop 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 shutdown_connections 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5561cdd38030 0x5561cddb0c60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5561cdd32ec0 0x5561cdd42490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561cdd324d0 0x5561cdd41480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 --2- 172.21.15.57:0/3527086575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561cdd2da40 0x5561cdd40f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 >> 172.21.15.57:0/3527086575 conn(0x5561cdd37b00 msgr2=0x5561cdd30ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f1147e00640 0 client.25738 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:21:56.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 shutdown_connections 2024-04-13T05:21:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.900+0000 7f116c06e740 1 -- 172.21.15.57:0/3527086575 wait complete. 2024-04-13T05:21:56.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.901+0000 7f116c06e740 20 client.25738 trim_cache size 0 max 16384 2024-04-13T05:21:56.905 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:21:56.905 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (52 inodes in the stack)", 2024-04-13T05:21:56.905 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:21:56.905 INFO:teuthology.orchestra.run.smithi057.stdout: "9bb81912-1843-4cb4-9b87-f3dfd64e0675": { 2024-04-13T05:21:56.905 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:21:56.906 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "9bb81912-1843-4cb4-9b87-f3dfd64e0675", 2024-04-13T05:21:56.906 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:21:56.906 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:21:56.906 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:21:56.906 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.903+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11480a3d00 msgr2=0x7f11480a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.903+0000 7f1169200640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11480a3d00 0x7f11480a61c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f115c09c620 tx=0x7f115c0989c0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.903+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 msgr2=0x7f11640709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.903+0000 7f1169200640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 0x7f11640709a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f115c00eaa0 tx=0x7f115c00ef60 comp rx=0 tx=0).stop 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.904+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 shutdown_connections 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.904+0000 7f1169200640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11480a3d00 0x7f11480a61c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.904+0000 7f1169200640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164105dd0 0x7f11641aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.904+0000 7f1169200640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1164105410 0x7f1164070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.904+0000 7f1169200640 1 --2- 172.21.15.57:0/1368624164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104220 0x7f11640709a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:21:56.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.904+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 >> 172.21.15.57:0/1368624164 conn(0x7f11640ff770 msgr2=0x7f1164108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:21:56.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.905+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 shutdown_connections 2024-04-13T05:21:56.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:21:56.905+0000 7f1169200640 1 -- 172.21.15.57:0/1368624164 wait complete. 2024-04-13T05:21:56.935 INFO:tasks.cephfs.filesystem:scrub status for tag:9bb81912-1843-4cb4-9b87-f3dfd64e0675 - {'path': '/', 'tag': '9bb81912-1843-4cb4-9b87-f3dfd64e0675', 'options': 'recursive,force'} 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: pgmap v586: 129 pgs: 129 active+clean; 189 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:57 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:21:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: pgmap v586: 129 pgs: 129 active+clean; 189 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2024-04-13T05:21:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:57 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:21:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: pgmap v586: 129 pgs: 129 active+clean; 189 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2024-04-13T05:21:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:21:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:57.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:57.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:57.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:57 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:21:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:21:58.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:58 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:21:58.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:58 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:21:58.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:58 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:59.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:59 smithi189 ceph-mon[35531]: pgmap v587: 129 pgs: 129 active+clean; 189 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-04-13T05:21:59.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:59 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:59.486 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:21:59.486 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:21:59.486 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:21:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:21:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:59 smithi083 ceph-mon[35133]: pgmap v587: 129 pgs: 129 active+clean; 189 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-04-13T05:21:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:59 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:21:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:21:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:21:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:21:59.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:59 smithi057 ceph-mon[32331]: pgmap v587: 129 pgs: 129 active+clean; 189 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-04-13T05:21:59.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:59 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:21:59.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:21:59.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:21:59.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:21:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:21:59.636 INFO:tasks.workunit.client.0.smithi057.stdout: 38 7250 161 4960 177 2358 186 2024-04-13T05:22:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:01 smithi083 ceph-mon[35133]: pgmap v588: 129 pgs: 129 active+clean; 194 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-13T05:22:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:01 smithi057 ceph-mon[32331]: pgmap v588: 129 pgs: 129 active+clean; 194 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-13T05:22:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:01 smithi189 ceph-mon[35531]: pgmap v588: 129 pgs: 129 active+clean; 194 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-13T05:22:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:03 smithi083 ceph-mon[35133]: pgmap v589: 129 pgs: 129 active+clean; 197 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-04-13T05:22:03.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:03 smithi057 ceph-mon[32331]: pgmap v589: 129 pgs: 129 active+clean; 197 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-04-13T05:22:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:03 smithi189 ceph-mon[35531]: pgmap v589: 129 pgs: 129 active+clean; 197 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-04-13T05:22:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:05 smithi083 ceph-mon[35133]: pgmap v590: 129 pgs: 129 active+clean; 201 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-04-13T05:22:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:05 smithi057 ceph-mon[32331]: pgmap v590: 129 pgs: 129 active+clean; 201 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-04-13T05:22:05.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:05 smithi189 ceph-mon[35531]: pgmap v590: 129 pgs: 129 active+clean; 201 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-04-13T05:22:07.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:07 smithi083 ceph-mon[35133]: pgmap v591: 129 pgs: 129 active+clean; 206 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-13T05:22:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:07 smithi057 ceph-mon[32331]: pgmap v591: 129 pgs: 129 active+clean; 206 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-13T05:22:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:07 smithi189 ceph-mon[35531]: pgmap v591: 129 pgs: 129 active+clean; 206 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-13T05:22:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:09 smithi083 ceph-mon[35133]: pgmap v592: 129 pgs: 129 active+clean; 206 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-04-13T05:22:09.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:09 smithi057 ceph-mon[32331]: pgmap v592: 129 pgs: 129 active+clean; 206 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-04-13T05:22:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:09 smithi189 ceph-mon[35531]: pgmap v592: 129 pgs: 129 active+clean; 206 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-04-13T05:22:09.636 INFO:tasks.workunit.client.0.smithi057.stdout:write(): Bad file descriptor 40 8962 166 5863 152 1853 140 2024-04-13T05:22:11.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:11 smithi083 ceph-mon[35133]: pgmap v593: 129 pgs: 129 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 148 op/s 2024-04-13T05:22:11.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:11 smithi057 ceph-mon[32331]: pgmap v593: 129 pgs: 129 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 148 op/s 2024-04-13T05:22:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:11 smithi189 ceph-mon[35531]: pgmap v593: 129 pgs: 129 active+clean; 213 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 148 op/s 2024-04-13T05:22:12.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:12 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:22:12.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:12 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:22:12.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:12 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:22:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:13 smithi083 ceph-mon[35133]: pgmap v594: 129 pgs: 129 active+clean; 214 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-04-13T05:22:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:13 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:22:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:13 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:22:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:13 smithi057 ceph-mon[32331]: pgmap v594: 129 pgs: 129 active+clean; 214 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-04-13T05:22:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:13 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:22:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:13 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:22:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:13 smithi189 ceph-mon[35531]: pgmap v594: 129 pgs: 129 active+clean; 214 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-04-13T05:22:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:13 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:22:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:13 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:22:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:15 smithi083 ceph-mon[35133]: pgmap v595: 129 pgs: 129 active+clean; 220 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-04-13T05:22:15.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:15 smithi057 ceph-mon[32331]: pgmap v595: 129 pgs: 129 active+clean; 220 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-04-13T05:22:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:15 smithi189 ceph-mon[35531]: pgmap v595: 129 pgs: 129 active+clean; 220 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-04-13T05:22:17.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:17 smithi083 ceph-mon[35133]: pgmap v596: 129 pgs: 129 active+clean; 224 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-04-13T05:22:17.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:17 smithi057 ceph-mon[32331]: pgmap v596: 129 pgs: 129 active+clean; 224 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-04-13T05:22:17.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:17 smithi189 ceph-mon[35531]: pgmap v596: 129 pgs: 129 active+clean; 224 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-04-13T05:22:18.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:18 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:22:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:18 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:22:18.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:18 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:22:19.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:19 smithi083 ceph-mon[35133]: pgmap v597: 129 pgs: 129 active+clean; 224 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-13T05:22:19.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:19 smithi057 ceph-mon[32331]: pgmap v597: 129 pgs: 129 active+clean; 224 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-13T05:22:19.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:19 smithi189 ceph-mon[35531]: pgmap v597: 129 pgs: 129 active+clean; 224 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-13T05:22:19.636 INFO:tasks.workunit.client.0.smithi057.stdout: 41 7917 147 5689 38 2116 111 2024-04-13T05:22:21.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:21 smithi083 ceph-mon[35133]: pgmap v598: 129 pgs: 129 active+clean; 225 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-04-13T05:22:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:21 smithi057 ceph-mon[32331]: pgmap v598: 129 pgs: 129 active+clean; 225 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-04-13T05:22:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:21 smithi189 ceph-mon[35531]: pgmap v598: 129 pgs: 129 active+clean; 225 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-04-13T05:22:23.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:23 smithi083 ceph-mon[35133]: pgmap v599: 129 pgs: 129 active+clean; 229 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-13T05:22:23.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:23 smithi057 ceph-mon[32331]: pgmap v599: 129 pgs: 129 active+clean; 229 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-13T05:22:23.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:23 smithi189 ceph-mon[35531]: pgmap v599: 129 pgs: 129 active+clean; 229 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-13T05:22:25.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:25 smithi057 ceph-mon[32331]: pgmap v600: 129 pgs: 129 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-13T05:22:25.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:25 smithi189 ceph-mon[35531]: pgmap v600: 129 pgs: 129 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-13T05:22:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:25 smithi083 ceph-mon[35133]: pgmap v600: 129 pgs: 129 active+clean; 231 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-13T05:22:26.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:22:26.937 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:22:27.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.202+0000 7fe32d600640 1 -- 172.21.15.57:0/3121072592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe328073400 msgr2=0x7fe328073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.202+0000 7fe32d600640 1 --2- 172.21.15.57:0/3121072592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe328073400 0x7fe328073820 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe318009900 tx=0x7fe31802bc50 comp rx=0 tx=0).stop 2024-04-13T05:22:27.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.202+0000 7fe32d600640 1 -- 172.21.15.57:0/3121072592 shutdown_connections 2024-04-13T05:22:27.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.202+0000 7fe32d600640 1 --2- 172.21.15.57:0/3121072592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe32810a040 0x7fe32810a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.202+0000 7fe32d600640 1 --2- 172.21.15.57:0/3121072592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3281096a0 0x7fe328109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.203+0000 7fe32d600640 1 --2- 172.21.15.57:0/3121072592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe328073400 0x7fe328073820 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.203+0000 7fe32d600640 1 -- 172.21.15.57:0/3121072592 >> 172.21.15.57:0/3121072592 conn(0x7fe32806eed0 msgr2=0x7fe328071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.203+0000 7fe32d600640 1 -- 172.21.15.57:0/3121072592 shutdown_connections 2024-04-13T05:22:27.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.203+0000 7fe32d600640 1 -- 172.21.15.57:0/3121072592 wait complete. 2024-04-13T05:22:27.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.204+0000 7fe32d600640 1 Processor -- start 2024-04-13T05:22:27.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.204+0000 7fe32d600640 1 -- start start 2024-04-13T05:22:27.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328073400 0x7fe3281a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 0x7fe3281a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe32810a040 0x7fe3281aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32d600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe328118440 con 0x7fe3281096a0 2024-04-13T05:22:27.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281182c0 con 0x7fe32810a040 2024-04-13T05:22:27.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32d600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281185c0 con 0x7fe328073400 2024-04-13T05:22:27.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe327400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 0x7fe3281a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe327400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 0x7fe3281a3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54738/0 (socket says 172.21.15.57:54738) 2024-04-13T05:22:27.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe327400640 1 -- 172.21.15.57:0/4114995850 learned_addr learned my addr 172.21.15.57:0/4114995850 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:27.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe32cc00640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe32810a040 0x7fe3281aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.205+0000 7fe327e00640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328073400 0x7fe3281a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe327400640 1 -- 172.21.15.57:0/4114995850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328073400 msgr2=0x7fe3281a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe327400640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328073400 0x7fe3281a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe327400640 1 -- 172.21.15.57:0/4114995850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe32810a040 msgr2=0x7fe3281aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe327400640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe32810a040 0x7fe3281aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe327400640 1 -- 172.21.15.57:0/4114995850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281aa710 con 0x7fe3281096a0 2024-04-13T05:22:27.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe327400640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 0x7fe3281a3030 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e9f0 tx=0x7fe31400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.206+0000 7fe324c00640 1 -- 172.21.15.57:0/4114995850 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31400cd80 con 0x7fe3281096a0 2024-04-13T05:22:27.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.207+0000 7fe324c00640 1 -- 172.21.15.57:0/4114995850 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3140054d0 con 0x7fe3281096a0 2024-04-13T05:22:27.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.207+0000 7fe324c00640 1 -- 172.21.15.57:0/4114995850 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314010490 con 0x7fe3281096a0 2024-04-13T05:22:27.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.207+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281aaa00 con 0x7fe3281096a0 2024-04-13T05:22:27.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.207+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281aaf50 con 0x7fe3281096a0 2024-04-13T05:22:27.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.210+0000 7fe324c00640 1 -- 172.21.15.57:0/4114995850 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe314005020 con 0x7fe3281096a0 2024-04-13T05:22:27.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 Processor -- start 2024-04-13T05:22:27.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 -- start start 2024-04-13T05:22:27.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5583c410f020 con 0x5583c41a90e0 2024-04-13T05:22:27.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5583c40b56d0 con 0x5583c41a86f0 2024-04-13T05:22:27.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.211+0000 7fe331bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583c410afe0 con 0x5583c41ae250 2024-04-13T05:22:27.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.212+0000 7fe32cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.212+0000 7fe327e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.212+0000 7fe32cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46384/0 (socket says 172.21.15.57:46384) 2024-04-13T05:22:27.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.212+0000 7fe32cc00640 1 -- 172.21.15.57:0/2467686940 learned_addr learned my addr 172.21.15.57:0/2467686940 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:27.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.212+0000 7fe327400640 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.212+0000 7fe324c00640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe30c0a3d00 0x7fe30c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe324c00640 1 -- 172.21.15.57:0/4114995850 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe3140d0b40 con 0x7fe3281096a0 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe327e00640 1 -- 172.21.15.57:0/2467686940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 msgr2=0x5583c41b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe327e00640 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe327e00640 1 -- 172.21.15.57:0/2467686940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 msgr2=0x5583c41a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe327e00640 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe327e00640 1 -- 172.21.15.57:0/2467686940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c3eeb7d0 con 0x5583c41a90e0 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe327400640 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe32cc00640 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.213+0000 7fe32cc00640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe30c0a3d00 0x7fe30c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.214+0000 7fe327e00640 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41adbe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe318036040 tx=0x7fe318009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba3f25e226b6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.214+0000 7fe31d200640 1 -- 172.21.15.57:0/2467686940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318044020 con 0x5583c41a90e0 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.214+0000 7fe31d200640 1 -- 172.21.15.57:0/2467686940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31802cb20 con 0x5583c41a90e0 2024-04-13T05:22:27.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.214+0000 7fe31d200640 1 -- 172.21.15.57:0/2467686940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318005650 con 0x5583c41a90e0 2024-04-13T05:22:27.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.215+0000 7fe331bc9740 1 -- 172.21.15.57:0/2467686940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 msgr2=0x5583c41adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.215+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41adbe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe318036040 tx=0x7fe318009550 comp rx=0 tx=0).stop 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.215+0000 7fe32cc00640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe30c0a3d00 0x7fe30c0a61c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5583c41a9940 tx=0x7fe320004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 -- 172.21.15.57:0/2467686940 shutdown_connections 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41adbe0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2467686940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 -- 172.21.15.57:0/2467686940 >> 172.21.15.57:0/2467686940 conn(0x5583c41a3c60 msgr2=0x5583c41a4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 -- 172.21.15.57:0/2467686940 shutdown_connections 2024-04-13T05:22:27.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 -- 172.21.15.57:0/2467686940 wait complete. 2024-04-13T05:22:27.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 -- start start 2024-04-13T05:22:27.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.216+0000 7fe331bc9740 1 Processor -- start 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 0x5583c41bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a90e0 0x5583c41bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5583c41b11a0 con 0x5583c41a86f0 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5583c40c8a90 con 0x5583c41a90e0 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583c41b1320 con 0x5583c41ae250 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.217+0000 7fe331bc9740 10 client.0 _ll_register_callbacks cb 0x5583c40c8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 0x5583c41bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 0x5583c41bb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54754/0 (socket says 172.21.15.57:54754) 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 -- 172.21.15.57:0/661235474 learned_addr learned my addr 172.21.15.57:0/661235474 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327400640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a90e0 0x5583c41bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe32cc00640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 -- 172.21.15.57:0/661235474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 msgr2=0x5583c41c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 -- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a90e0 msgr2=0x5583c41bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a90e0 0x5583c41bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327e00640 1 -- 172.21.15.57:0/661235474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c3e213f0 con 0x5583c41a86f0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe327400640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a90e0 0x5583c41bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.218+0000 7fe32cc00640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe327e00640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 0x5583c41bb170 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe31802c260 tx=0x7fe3180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4288cb9ebbbe4229 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe311600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe311600640 1 -- 172.21.15.57:0/661235474 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31804f030 con 0x5583c41a86f0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe311600640 1 -- 172.21.15.57:0/661235474 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe31802c920 con 0x5583c41a86f0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31802cb20 con 0x5583c41a86f0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe311600640 1 -- 172.21.15.57:0/661235474 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe318059050 con 0x5583c41a86f0 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe331bc9740 10 client.16398 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:27.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.219+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583c3de10f0 con 0x5583c41a86f0 2024-04-13T05:22:27.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.220+0000 7fe311600640 1 -- 172.21.15.57:0/661235474 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31802cb20 con 0x5583c41a86f0 2024-04-13T05:22:27.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.220+0000 7fe311600640 1 -- 172.21.15.57:0/661235474 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe318040b20 con 0x5583c41a86f0 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.220+0000 7fe331bc9740 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.220+0000 7fe331bc9740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.220+0000 7fe331bc9740 20 client.16398 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.221+0000 7fe331bc9740 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c41c6be0 0x5583c41c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.221+0000 7fe331bc9740 4 client.16398 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.221+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583c40ae600 con 0x5583c41c6be0 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.221+0000 7fe327400640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c41c6be0 0x5583c41c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.222+0000 7fe327400640 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c41c6be0 0x5583c41c8fd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9936eda565b4a54 server_cookie=6b257ae326baad1 in_seq=0 out_seq=0 2024-04-13T05:22:27.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.222+0000 7fe311600640 10 client.16398 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:27.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.223+0000 7fe311600640 1 -- 172.21.15.57:0/661235474 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583c40ae600 con 0x5583c41c6be0 2024-04-13T05:22:27.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.223+0000 7fe311600640 10 client.16398 handle_command_reply: tid=0 2024-04-13T05:22:27.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.223+0000 7fe331bc9740 1 client.16398 shutdown 2024-04-13T05:22:27.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 msgr2=0x5583c41bb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 0x5583c41bb170 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe31802c260 tx=0x7fe3180038a0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 shutdown_connections 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c41c6be0 0x5583c41c8fd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a90e0 0x5583c41bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41ae250 0x5583c41c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 --2- 172.21.15.57:0/661235474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a86f0 0x5583c41bb170 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 >> 172.21.15.57:0/661235474 conn(0x5583c41a3c60 msgr2=0x5583c41a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe311600640 0 client.16398 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.224+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 shutdown_connections 2024-04-13T05:22:27.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.225+0000 7fe331bc9740 1 -- 172.21.15.57:0/661235474 wait complete. 2024-04-13T05:22:27.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.225+0000 7fe331bc9740 20 client.16398 trim_cache size 0 max 16384 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 Processor -- start 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 -- start start 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 0x5583c41cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41ae250 0x5583c41cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b3800 con 0x5583c41a90e0 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b1070 con 0x5583c41ae250 2024-04-13T05:22:27.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.253+0000 7fe331bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281aeec0 con 0x5583c41a86f0 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.254+0000 7fe327e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 0x5583c41cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.254+0000 7fe327e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 0x5583c41cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:52764/0 (socket says 172.21.15.57:52764) 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.254+0000 7fe327e00640 1 -- 172.21.15.57:0/2163900279 learned_addr learned my addr 172.21.15.57:0/2163900279 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.254+0000 7fe327400640 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.254+0000 7fe32cc00640 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41ae250 0x5583c41cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.255+0000 7fe327e00640 1 -- 172.21.15.57:0/2163900279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41ae250 msgr2=0x5583c41cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.255+0000 7fe327e00640 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41ae250 0x5583c41cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.255+0000 7fe327e00640 1 -- 172.21.15.57:0/2163900279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 msgr2=0x5583c41c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.255+0000 7fe327e00640 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.255+0000 7fe327e00640 1 -- 172.21.15.57:0/2163900279 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c41a49d0 con 0x5583c41a86f0 2024-04-13T05:22:27.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.255+0000 7fe327400640 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:27.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.256+0000 7fe327e00640 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 0x5583c41cd7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe318006010 tx=0x7fe318009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9a3f0e9edc4adf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.256+0000 7fe313e00640 1 -- 172.21.15.57:0/2163900279 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31804f020 con 0x5583c41a86f0 2024-04-13T05:22:27.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.256+0000 7fe313e00640 1 -- 172.21.15.57:0/2163900279 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3180044d0 con 0x5583c41a86f0 2024-04-13T05:22:27.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.256+0000 7fe313e00640 1 -- 172.21.15.57:0/2163900279 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318044070 con 0x5583c41a86f0 2024-04-13T05:22:27.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 -- 172.21.15.57:0/2163900279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 msgr2=0x5583c41cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 0x5583c41cd7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe318006010 tx=0x7fe318009550 comp rx=0 tx=0).stop 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 -- 172.21.15.57:0/2163900279 shutdown_connections 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41ae250 0x5583c41cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41a90e0 0x5583c41c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 --2- 172.21.15.57:0/2163900279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a86f0 0x5583c41cd7c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 -- 172.21.15.57:0/2163900279 >> 172.21.15.57:0/2163900279 conn(0x5583c41a3c60 msgr2=0x5583c41a7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.257+0000 7fe331bc9740 1 -- 172.21.15.57:0/2163900279 shutdown_connections 2024-04-13T05:22:27.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.258+0000 7fe331bc9740 1 -- 172.21.15.57:0/2163900279 wait complete. 2024-04-13T05:22:27.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.258+0000 7fe331bc9740 1 -- start start 2024-04-13T05:22:27.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.258+0000 7fe331bc9740 1 Processor -- start 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a90e0 0x5583c41bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 0x5583c422b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281af530 con 0x5583c41ae250 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281b09a0 con 0x5583c41a86f0 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281afc00 con 0x5583c41a90e0 2024-04-13T05:22:27.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.259+0000 7fe331bc9740 10 client.0 _ll_register_callbacks cb 0x7fe318004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.260+0000 7fe327e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41bf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.260+0000 7fe32cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 0x5583c422b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.260+0000 7fe327400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a90e0 0x5583c41bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.260+0000 7fe32cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 0x5583c422b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54772/0 (socket says 172.21.15.57:54772) 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.260+0000 7fe32cc00640 1 -- 172.21.15.57:0/4185191267 learned_addr learned my addr 172.21.15.57:0/4185191267 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe32cc00640 1 -- 172.21.15.57:0/4185191267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a90e0 msgr2=0x5583c41bfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe32cc00640 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a90e0 0x5583c41bfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe32cc00640 1 -- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 msgr2=0x5583c41bf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:27.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe32cc00640 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41bf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe32cc00640 1 -- 172.21.15.57:0/4185191267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c422b5a0 con 0x5583c41ae250 2024-04-13T05:22:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe32cc00640 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 0x5583c422b030 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe320013b60 tx=0x7fe320004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accb3835f087b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.261+0000 7fe311600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.262+0000 7fe311600640 1 -- 172.21.15.57:0/4185191267 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32002e060 con 0x5583c41ae250 2024-04-13T05:22:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.262+0000 7fe311600640 1 -- 172.21.15.57:0/4185191267 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe32001c420 con 0x5583c41ae250 2024-04-13T05:22:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.262+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3180044d0 con 0x5583c41ae250 2024-04-13T05:22:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.263+0000 7fe311600640 1 -- 172.21.15.57:0/4185191267 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe32001c620 con 0x5583c41ae250 2024-04-13T05:22:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.263+0000 7fe331bc9740 10 client.16404 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:27.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.263+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583c422c960 con 0x5583c41ae250 2024-04-13T05:22:27.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.263+0000 7fe311600640 1 -- 172.21.15.57:0/4185191267 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3180044d0 con 0x5583c41ae250 2024-04-13T05:22:27.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.264+0000 7fe311600640 1 -- 172.21.15.57:0/4185191267 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe320028070 con 0x5583c41ae250 2024-04-13T05:22:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe331bc9740 10 client.16404 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe331bc9740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe331bc9740 20 client.16404 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe331bc9740 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c422d290 0x5583c422f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe331bc9740 4 client.16404 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:22:27.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583c411ca40 con 0x5583c422d290 2024-04-13T05:22:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.265+0000 7fe327e00640 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c422d290 0x5583c422f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.266+0000 7fe327e00640 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c422d290 0x5583c422f6b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bb939242be11e0 server_cookie=e2e26eaaf418f7a5 in_seq=0 out_seq=0 2024-04-13T05:22:27.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.266+0000 7fe311600640 10 client.16404 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:27.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.267+0000 7fe311600640 1 -- 172.21.15.57:0/4185191267 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5583c411ca40 con 0x5583c422d290 2024-04-13T05:22:27.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.267+0000 7fe311600640 10 client.16404 handle_command_reply: tid=0 2024-04-13T05:22:27.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.267+0000 7fe331bc9740 1 client.16404 shutdown 2024-04-13T05:22:27.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.267+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 msgr2=0x5583c422b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.267+0000 7fe331bc9740 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 0x5583c422b030 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe320013b60 tx=0x7fe320004ab0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 shutdown_connections 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5583c422d290 0x5583c422f6b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5583c41ae250 0x5583c422b030 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583c41a90e0 0x5583c41bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 --2- 172.21.15.57:0/4185191267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583c41a86f0 0x5583c41bf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 >> 172.21.15.57:0/4185191267 conn(0x5583c41a3c60 msgr2=0x5583c41a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe311600640 0 client.16404 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:27.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 shutdown_connections 2024-04-13T05:22:27.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 1 -- 172.21.15.57:0/4185191267 wait complete. 2024-04-13T05:22:27.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.268+0000 7fe331bc9740 20 client.16404 trim_cache size 0 max 16384 2024-04-13T05:22:27.272 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:22:27.272 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:22:27.272 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:22:27.272 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:22:27.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.269+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe30c0a3d00 msgr2=0x7fe30c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.269+0000 7fe32d600640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe30c0a3d00 0x7fe30c0a61c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5583c41a9940 tx=0x7fe320004a40 comp rx=0 tx=0).stop 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.269+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 msgr2=0x7fe3281a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.269+0000 7fe32d600640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 0x7fe3281a3030 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e9f0 tx=0x7fe31400edf0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 shutdown_connections 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe30c0a3d00 0x7fe30c0a61c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe32810a040 0x7fe3281aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3281096a0 0x7fe3281a3030 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 --2- 172.21.15.57:0/4114995850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328073400 0x7fe3281a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 >> 172.21.15.57:0/4114995850 conn(0x7fe32806eed0 msgr2=0x7fe3280716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 shutdown_connections 2024-04-13T05:22:27.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.270+0000 7fe32d600640 1 -- 172.21.15.57:0/4114995850 wait complete. 2024-04-13T05:22:27.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:27 smithi057 ceph-mon[32331]: pgmap v601: 129 pgs: 129 active+clean; 236 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-13T05:22:27.293 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:22:27.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:22:27.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.555+0000 7fb9cca00640 1 -- 172.21.15.57:0/1778646875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105c10 msgr2=0x7fb9c810c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.555+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1778646875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105c10 0x7fb9c810c4a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009be0 tx=0x7fb9c002bef0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 -- 172.21.15.57:0/1778646875 shutdown_connections 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1778646875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8105250 0x7fb9c81056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1778646875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105c10 0x7fb9c810c4a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1778646875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c8104060 0x7fb9c8104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 -- 172.21.15.57:0/1778646875 >> 172.21.15.57:0/1778646875 conn(0x7fb9c80ff5d0 msgr2=0x7fb9c8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:27.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 -- 172.21.15.57:0/1778646875 shutdown_connections 2024-04-13T05:22:27.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.556+0000 7fb9cca00640 1 -- 172.21.15.57:0/1778646875 wait complete. 2024-04-13T05:22:27.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.557+0000 7fb9cca00640 1 Processor -- start 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 -- start start 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 0x7fb9c819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 0x7fb9c819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c8105c10 0x7fb9c81a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c810fba0 con 0x7fb9c8105250 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c810fa20 con 0x7fb9c8104060 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9cca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c810fd20 con 0x7fb9c8105c10 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9c5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 0x7fb9c819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9c6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 0x7fb9c819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9c5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 0x7fb9c819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54778/0 (socket says 172.21.15.57:54778) 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9c5e00640 1 -- 172.21.15.57:0/1693823485 learned_addr learned my addr 172.21.15.57:0/1693823485 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:27.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.558+0000 7fb9c6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 0x7fb9c819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46438/0 (socket says 172.21.15.57:46438) 2024-04-13T05:22:27.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.559+0000 7fb9c7200640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c8105c10 0x7fb9c81a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.559+0000 7fb9c6800640 1 -- 172.21.15.57:0/1693823485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c8105c10 msgr2=0x7fb9c81a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.559+0000 7fb9c6800640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c8105c10 0x7fb9c81a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.559+0000 7fb9c6800640 1 -- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 msgr2=0x7fb9c819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:27.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.559+0000 7fb9c6800640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 0x7fb9c819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:27.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.559+0000 7fb9c6800640 1 -- 172.21.15.57:0/1693823485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c8195390 con 0x7fb9c8104060 2024-04-13T05:22:27.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.560+0000 7fb9c5e00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 0x7fb9c819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:27.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.560+0000 7fb9c6800640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 0x7fb9c819e670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9c0 tx=0x7fb9b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.561+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ccc0 con 0x7fb9c8104060 2024-04-13T05:22:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.561+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9b80035f0 con 0x7fb9c8104060 2024-04-13T05:22:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.561+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c8195680 con 0x7fb9c8104060 2024-04-13T05:22:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.561+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c8195bd0 con 0x7fb9c8104060 2024-04-13T05:22:27.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.561+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ce60 con 0x7fb9c8104060 2024-04-13T05:22:27.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.563+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80052c0 con 0x7fb9c8104060 2024-04-13T05:22:27.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.566+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9b80037a0 con 0x7fb9c8104060 2024-04-13T05:22:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.568+0000 7fb9bf400640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9ac0a3d00 0x7fb9ac0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.569+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb9b80d0bf0 con 0x7fb9c8104060 2024-04-13T05:22:27.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.569+0000 7fb9c5e00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9ac0a3d00 0x7fb9ac0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:27.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.570+0000 7fb9c5e00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9ac0a3d00 0x7fb9ac0a61c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40096c0 tx=0x7fb9b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:27.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:27.580+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9b809d1d0 con 0x7fb9c8104060 2024-04-13T05:22:27.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:27 smithi189 ceph-mon[35531]: pgmap v601: 129 pgs: 129 active+clean; 236 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-13T05:22:27.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:27 smithi083 ceph-mon[35133]: pgmap v601: 129 pgs: 129 active+clean; 236 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-13T05:22:28.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.004+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a80055b0 con 0x7fb9c8104060 2024-04-13T05:22:28.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.006+0000 7fb9bf400640 1 -- 172.21.15.57:0/1693823485 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11858 (secure 0 0 0) 0x7fb9b809cad0 con 0x7fb9c8104060 2024-04-13T05:22:28.008 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:22:28.009 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:22:16.935949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:22:28.009 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 29 2024-04-13T05:22:28.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9ac0a3d00 msgr2=0x7fb9ac0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9ac0a3d00 0x7fb9ac0a61c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40096c0 tx=0x7fb9b4009260 comp rx=0 tx=0).stop 2024-04-13T05:22:28.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 msgr2=0x7fb9c819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 0x7fb9c819e670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9c0 tx=0x7fb9b800ee80 comp rx=0 tx=0).stop 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 shutdown_connections 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb9ac0a3d00 0x7fb9ac0a61c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c8105c10 0x7fb9c81a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c8105250 0x7fb9c819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 --2- 172.21.15.57:0/1693823485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9c8104060 0x7fb9c819e670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 >> 172.21.15.57:0/1693823485 conn(0x7fb9c80ff5d0 msgr2=0x7fb9c80784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 shutdown_connections 2024-04-13T05:22:28.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.009+0000 7fb9cca00640 1 -- 172.21.15.57:0/1693823485 wait complete. 2024-04-13T05:22:28.034 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:22:28.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.296+0000 7f8813c00640 1 -- 172.21.15.57:0/3625756758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 msgr2=0x7f880c10c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.296+0000 7f8813c00640 1 --2- 172.21.15.57:0/3625756758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c10c680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8804009be0 tx=0x7f880402bef0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 -- 172.21.15.57:0/3625756758 shutdown_connections 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 --2- 172.21.15.57:0/3625756758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c10c680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 --2- 172.21.15.57:0/3625756758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c105430 0x7f880c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 --2- 172.21.15.57:0/3625756758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f880c104240 0x7f880c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 -- 172.21.15.57:0/3625756758 >> 172.21.15.57:0/3625756758 conn(0x7f880c0ff7b0 msgr2=0x7f880c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 -- 172.21.15.57:0/3625756758 shutdown_connections 2024-04-13T05:22:28.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.297+0000 7f8813c00640 1 -- 172.21.15.57:0/3625756758 wait complete. 2024-04-13T05:22:28.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.298+0000 7f8813c00640 1 Processor -- start 2024-04-13T05:22:28.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 -- start start 2024-04-13T05:22:28.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 0x7f880c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f880c105430 0x7f880c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10fdd0 con 0x7f880c104240 2024-04-13T05:22:28.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10fc50 con 0x7f880c105df0 2024-04-13T05:22:28.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.299+0000 7f8813c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10ff50 con 0x7f880c105430 2024-04-13T05:22:28.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 0x7f880c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 0x7f880c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54784/0 (socket says 172.21.15.57:54784) 2024-04-13T05:22:28.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811200640 1 -- 172.21.15.57:0/3214040907 learned_addr learned my addr 172.21.15.57:0/3214040907 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f880be00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f880c105430 0x7f880c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811c00640 1 -- 172.21.15.57:0/3214040907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f880c105430 msgr2=0x7f880c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f880c105430 0x7f880c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.300+0000 7f8811c00640 1 -- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 msgr2=0x7f880c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.301+0000 7f8811c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 0x7f880c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.301+0000 7f8811c00640 1 -- 172.21.15.57:0/3214040907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1a2000 con 0x7f880c105df0 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.301+0000 7f8811200640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 0x7f880c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.301+0000 7f8811c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c1a1990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8804036040 tx=0x7f8804033c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.302+0000 7f8809600640 1 -- 172.21.15.57:0/3214040907 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804043020 con 0x7f880c105df0 2024-04-13T05:22:28.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.302+0000 7f8809600640 1 -- 172.21.15.57:0/3214040907 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8804003e20 con 0x7f880c105df0 2024-04-13T05:22:28.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.302+0000 7f8809600640 1 -- 172.21.15.57:0/3214040907 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804034430 con 0x7f880c105df0 2024-04-13T05:22:28.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.302+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1a2290 con 0x7f880c105df0 2024-04-13T05:22:28.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.302+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a27b0 con 0x7f880c105df0 2024-04-13T05:22:28.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.307+0000 7f8816bb4740 1 Processor -- start 2024-04-13T05:22:28.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.308+0000 7f8809600640 1 -- 172.21.15.57:0/3214040907 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8804046020 con 0x7f880c105df0 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.308+0000 7f8816bb4740 1 -- start start 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df6af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df6da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8816bb4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559d7dec3020 con 0x559d7df6b5c0 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8816bb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d7de69950 con 0x559d7df66450 2024-04-13T05:22:28.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8816bb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559d7debefe0 con 0x559d7df65a60 2024-04-13T05:22:28.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f880be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df6da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f8811200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df6af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f880be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df6da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54796/0 (socket says 172.21.15.57:54796) 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f880be00640 1 -- 172.21.15.57:0/3698069551 learned_addr learned my addr 172.21.15.57:0/3698069551 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.309+0000 7f880be00640 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f880be00640 1 -- 172.21.15.57:0/3698069551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 msgr2=0x559d7df65e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f880be00640 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df65e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f880be00640 1 -- 172.21.15.57:0/3698069551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 msgr2=0x559d7df6af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f880be00640 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df6af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f880be00640 1 -- 172.21.15.57:0/3698069551 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7dbd53f0 con 0x559d7df6b5c0 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f8809600640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f40a3d80 0x7f87f40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f8809600640 1 -- 172.21.15.57:0/3214040907 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f88040f2e00 con 0x7f880c105df0 2024-04-13T05:22:28.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f8811200640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f40a3d80 0x7f87f40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.310+0000 7f880be00640 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df6da50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009fd0 tx=0x7f87f8009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d998f5b32bf55014 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.311+0000 7f87fe000640 1 -- 172.21.15.57:0/3698069551 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800f790 con 0x559d7df6b5c0 2024-04-13T05:22:28.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.311+0000 7f87fe000640 1 -- 172.21.15.57:0/3698069551 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87f800fde0 con 0x559d7df6b5c0 2024-04-13T05:22:28.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.311+0000 7f87fe000640 1 -- 172.21.15.57:0/3698069551 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f801fd60 con 0x559d7df6b5c0 2024-04-13T05:22:28.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8811200640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f40a3d80 0x7f87f40a6240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x559d7df66850 tx=0x7f8800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 -- 172.21.15.57:0/3698069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 msgr2=0x559d7df6da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df6da50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009fd0 tx=0x7f87f8009370 comp rx=0 tx=0).stop 2024-04-13T05:22:28.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 -- 172.21.15.57:0/3698069551 shutdown_connections 2024-04-13T05:22:28.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df6af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df6da50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3698069551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df65e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 -- 172.21.15.57:0/3698069551 >> 172.21.15.57:0/3698069551 conn(0x559d7df60fd0 msgr2=0x559d7df613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 -- 172.21.15.57:0/3698069551 shutdown_connections 2024-04-13T05:22:28.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.312+0000 7f8816bb4740 1 -- 172.21.15.57:0/3698069551 wait complete. 2024-04-13T05:22:28.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.313+0000 7f8816bb4740 1 -- start start 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.313+0000 7f8816bb4740 1 Processor -- start 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df70070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559d7df6e570 con 0x559d7df6b5c0 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d7de7cc80 con 0x559d7df66450 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559d7df6e6f0 con 0x559d7df65a60 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8816bb4740 10 client.0 _ll_register_callbacks cb 0x559d7de7c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:28.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8811c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8811200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df70070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f880be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8811200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df70070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:52816/0 (socket says 172.21.15.57:52816) 2024-04-13T05:22:28.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.314+0000 7f8811200640 1 -- 172.21.15.57:0/2301643379 learned_addr learned my addr 172.21.15.57:0/2301643379 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f8811c00640 1 -- 172.21.15.57:0/2301643379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 msgr2=0x559d7df70070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f8811c00640 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df70070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f8811c00640 1 -- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 msgr2=0x559d7df705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f8811c00640 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f8811c00640 1 -- 172.21.15.57:0/2301643379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7dc9f7d0 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f880be00640 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f8811c00640 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8804034ec0 tx=0x7f88040be8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ec78c76c2db67 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.315+0000 7f87f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f87f2a00640 1 -- 172.21.15.57:0/2301643379 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88040d7020 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f87f2a00640 1 -- 172.21.15.57:0/2301643379 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8804004260 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f800fde0 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f87f2a00640 1 -- 172.21.15.57:0/2301643379 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88040d8220 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f8816bb4740 10 client.16422 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d7df6e570 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.316+0000 7f87f2a00640 1 -- 172.21.15.57:0/2301643379 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87f800fde0 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f87f2a00640 1 -- 172.21.15.57:0/2301643379 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8804050060 con 0x559d7df6b5c0 2024-04-13T05:22:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f8816bb4740 10 client.16422 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f8816bb4740 10 client.16422 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f8816bb4740 20 client.16422 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f8816bb4740 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7df7be50 0x559d7df7e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f8816bb4740 4 client.16422 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.317+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d7de62750 con 0x559d7df7be50 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.318+0000 7f8811200640 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7df7be50 0x559d7df7e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.319+0000 7f8811200640 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7df7be50 0x559d7df7e270 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71aaa1c42c1bcb7e server_cookie=6f9529fb959b8fe6 in_seq=0 out_seq=0 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.319+0000 7f87f2a00640 10 client.16422 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f87f2a00640 1 -- 172.21.15.57:0/2301643379 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d7de62750 con 0x559d7df7be50 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f87f2a00640 10 client.16422 handle_command_reply: tid=0 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 client.16422 shutdown 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 msgr2=0x559d7df775f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8804034ec0 tx=0x7f88040be8a0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 shutdown_connections 2024-04-13T05:22:28.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7df7be50 0x559d7df7e270 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df66450 0x559d7df705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 --2- 172.21.15.57:0/2301643379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df70070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 >> 172.21.15.57:0/2301643379 conn(0x559d7df60fd0 msgr2=0x559d7df57a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.320+0000 7f87f2a00640 0 client.16422 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:22:28.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.321+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 shutdown_connections 2024-04-13T05:22:28.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.321+0000 7f8816bb4740 1 -- 172.21.15.57:0/2301643379 wait complete. 2024-04-13T05:22:28.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.321+0000 7f8816bb4740 20 client.16422 trim_cache size 0 max 16384 2024-04-13T05:22:28.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.343+0000 7f8816bb4740 1 Processor -- start 2024-04-13T05:22:28.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.343+0000 7f8816bb4740 1 -- start start 2024-04-13T05:22:28.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.343+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df65a60 0x559d7df67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df66450 0x559d7df67f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8816bb4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1ab220 con 0x559d7df6b5c0 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8816bb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a8a90 con 0x559d7df65a60 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8816bb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a6900 con 0x559d7df66450 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8811c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8811c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54814/0 (socket says 172.21.15.57:54814) 2024-04-13T05:22:28.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.344+0000 7f8811c00640 1 -- 172.21.15.57:0/3078907525 learned_addr learned my addr 172.21.15.57:0/3078907525 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811200640 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df65a60 0x559d7df67a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811c00640 1 -- 172.21.15.57:0/3078907525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df66450 msgr2=0x559d7df67f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811c00640 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df66450 0x559d7df67f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811c00640 1 -- 172.21.15.57:0/3078907525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df65a60 msgr2=0x559d7df67a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811c00640 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df65a60 0x559d7df67a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811c00640 1 -- 172.21.15.57:0/3078907525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7df7e8f0 con 0x559d7df6b5c0 2024-04-13T05:22:28.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.345+0000 7f8811c00640 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8804006010 tx=0x7f880402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7bff21e5678a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.347+0000 7f87fd600640 1 -- 172.21.15.57:0/3078907525 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88040d7030 con 0x559d7df6b5c0 2024-04-13T05:22:28.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.347+0000 7f87fd600640 1 -- 172.21.15.57:0/3078907525 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88040c3090 con 0x559d7df6b5c0 2024-04-13T05:22:28.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.347+0000 7f87fd600640 1 -- 172.21.15.57:0/3078907525 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004260 con 0x559d7df6b5c0 2024-04-13T05:22:28.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.348+0000 7f8816bb4740 1 -- 172.21.15.57:0/3078907525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 msgr2=0x559d7df775b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.348+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8804006010 tx=0x7f880402bef0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.349+0000 7f8816bb4740 1 -- 172.21.15.57:0/3078907525 shutdown_connections 2024-04-13T05:22:28.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.349+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df6b5c0 0x559d7df775b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.349+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df66450 0x559d7df67f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.349+0000 7f8816bb4740 1 --2- 172.21.15.57:0/3078907525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df65a60 0x559d7df67a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.349+0000 7f8816bb4740 1 -- 172.21.15.57:0/3078907525 >> 172.21.15.57:0/3078907525 conn(0x559d7df60fd0 msgr2=0x559d7df70070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.350+0000 7f8816bb4740 1 -- 172.21.15.57:0/3078907525 shutdown_connections 2024-04-13T05:22:28.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.350+0000 7f8816bb4740 1 -- 172.21.15.57:0/3078907525 wait complete. 2024-04-13T05:22:28.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.350+0000 7f8816bb4740 1 -- start start 2024-04-13T05:22:28.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.350+0000 7f8816bb4740 1 Processor -- start 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.351+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 0x559d7df75070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8816bb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df6b5c0 0x559d7df76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8816bb4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a6f50 con 0x559d7df66450 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8816bb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a83c0 con 0x559d7df6b5c0 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8816bb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a7620 con 0x559d7df65a60 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f880be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 0x559d7df75070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8816bb4740 10 client.0 _ll_register_callbacks cb 0x7f88040beab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8811200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.352+0000 7f8811c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df6b5c0 0x559d7df76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f8811c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df6b5c0 0x559d7df76030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46482/0 (socket says 172.21.15.57:46482) 2024-04-13T05:22:28.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f880be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 0x559d7df75070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54824/0 (socket says 172.21.15.57:54824) 2024-04-13T05:22:28.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f8811c00640 1 -- 172.21.15.57:0/364044346 learned_addr learned my addr 172.21.15.57:0/364044346 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f880be00640 1 -- 172.21.15.57:0/364044346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 msgr2=0x559d7df74b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f880be00640 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df74b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f880be00640 1 -- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df6b5c0 msgr2=0x559d7df76030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f880be00640 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df6b5c0 0x559d7df76030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.353+0000 7f880be00640 1 -- 172.21.15.57:0/364044346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7df76620 con 0x559d7df66450 2024-04-13T05:22:28.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.354+0000 7f880be00640 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 0x559d7df75070 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f87f8010010 tx=0x7f87f8005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9516dc984b80320e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.354+0000 7f87f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:28.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.354+0000 7f87f2a00640 1 -- 172.21.15.57:0/364044346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800f790 con 0x559d7df66450 2024-04-13T05:22:28.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.355+0000 7f87f2a00640 1 -- 172.21.15.57:0/364044346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f87f800f930 con 0x559d7df66450 2024-04-13T05:22:28.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.355+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d7ded0a40 con 0x559d7df66450 2024-04-13T05:22:28.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.355+0000 7f87f2a00640 1 -- 172.21.15.57:0/364044346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87f800fb50 con 0x559d7df66450 2024-04-13T05:22:28.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.355+0000 7f8816bb4740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:28.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.355+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d7df77920 con 0x559d7df66450 2024-04-13T05:22:28.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.356+0000 7f87f2a00640 1 -- 172.21.15.57:0/364044346 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559d7ded0a40 con 0x559d7df66450 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.356+0000 7f87f2a00640 1 -- 172.21.15.57:0/364044346 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87f801d020 con 0x559d7df66450 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.357+0000 7f8816bb4740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.357+0000 7f8816bb4740 10 client.16428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.357+0000 7f8816bb4740 20 client.16428 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.357+0000 7f8816bb4740 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7dfe2880 0x559d7dfe4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.357+0000 7f8816bb4740 4 client.16428 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:22:28.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.357+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d7deca8b0 con 0x559d7dfe2880 2024-04-13T05:22:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.446+0000 7f8811200640 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7dfe2880 0x559d7dfe4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.454+0000 7f8811200640 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7dfe2880 0x559d7dfe4ca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de844c990573e3f6 server_cookie=a6893db7de43758e in_seq=0 out_seq=0 2024-04-13T05:22:28.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.454+0000 7f87f2a00640 10 client.16428 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:22:28.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f87f2a00640 1 -- 172.21.15.57:0/364044346 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d7deca8b0 con 0x559d7dfe2880 2024-04-13T05:22:28.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f87f2a00640 10 client.16428 handle_command_reply: tid=0 2024-04-13T05:22:28.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 client.16428 shutdown 2024-04-13T05:22:28.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 msgr2=0x559d7df75070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 0x559d7df75070 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f87f8010010 tx=0x7f87f8005580 comp rx=0 tx=0).stop 2024-04-13T05:22:28.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 shutdown_connections 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x559d7dfe2880 0x559d7dfe4ca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d7df6b5c0 0x559d7df76030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559d7df66450 0x559d7df75070 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 --2- 172.21.15.57:0/364044346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d7df65a60 0x559d7df74b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.464+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 >> 172.21.15.57:0/364044346 conn(0x559d7df60fd0 msgr2=0x559d7df70070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.465+0000 7f87f2a00640 0 client.16428 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.465+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 shutdown_connections 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.465+0000 7f8816bb4740 1 -- 172.21.15.57:0/364044346 wait complete. 2024-04-13T05:22:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.465+0000 7f8816bb4740 20 client.16428 trim_cache size 0 max 16384 2024-04-13T05:22:28.469 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:22:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f40a3d80 msgr2=0x7f87f40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f40a3d80 0x7f87f40a6240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x559d7df66850 tx=0x7f8800009260 comp rx=0 tx=0).stop 2024-04-13T05:22:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 msgr2=0x7f880c1a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c1a1990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8804036040 tx=0x7f8804033c40 comp rx=0 tx=0).stop 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 shutdown_connections 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87f40a3d80 0x7f87f40a6240 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f880c105df0 0x7f880c1a1990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f880c105430 0x7f880c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 --2- 172.21.15.57:0/3214040907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f880c104240 0x7f880c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 >> 172.21.15.57:0/3214040907 conn(0x7f880c0ff7b0 msgr2=0x7f880c108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 shutdown_connections 2024-04-13T05:22:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.466+0000 7f8813c00640 1 -- 172.21.15.57:0/3214040907 wait complete. 2024-04-13T05:22:28.490 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:22:28.578 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:28 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1693823485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:22:28.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:28 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1693823485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:22:28.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:28 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1693823485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:22:28.751 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.749+0000 7f7014e00640 1 -- 172.21.15.57:0/3579648351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010105410 msgr2=0x7f7010105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.749+0000 7f7014e00640 1 --2- 172.21.15.57:0/3579648351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010105410 0x7f7010105890 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009900 tx=0x7f6ffc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:22:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 -- 172.21.15.57:0/3579648351 shutdown_connections 2024-04-13T05:22:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 --2- 172.21.15.57:0/3579648351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7010105dd0 0x7f701010c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 --2- 172.21.15.57:0/3579648351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010105410 0x7f7010105890 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 --2- 172.21.15.57:0/3579648351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7010104220 0x7f7010104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 -- 172.21.15.57:0/3579648351 >> 172.21.15.57:0/3579648351 conn(0x7f70100ff770 msgr2=0x7f7010101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 -- 172.21.15.57:0/3579648351 shutdown_connections 2024-04-13T05:22:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.750+0000 7f7014e00640 1 -- 172.21.15.57:0/3579648351 wait complete. 2024-04-13T05:22:28.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.752+0000 7f7014e00640 1 Processor -- start 2024-04-13T05:22:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.752+0000 7f7014e00640 1 -- start start 2024-04-13T05:22:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.752+0000 7f7014e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 0x7f7010079d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.752+0000 7f7014e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7010105410 0x7f701007a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f7014e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7010105dd0 0x7f70100768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f700de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 0x7f7010079d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f7014e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f701010fdb0 con 0x7f7010104220 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f7014e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f701010fc30 con 0x7f7010105dd0 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f7014e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f701010ff30 con 0x7f7010105410 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f700de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 0x7f7010079d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54852/0 (socket says 172.21.15.57:54852) 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f700de00640 1 -- 172.21.15.57:0/3344559361 learned_addr learned my addr 172.21.15.57:0/3344559361 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f700d400640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7010105410 0x7f701007a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.753+0000 7f700e800640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7010105dd0 0x7f70100768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f700de00640 1 -- 172.21.15.57:0/3344559361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7010105410 msgr2=0x7f701007a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f700de00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7010105410 0x7f701007a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f700de00640 1 -- 172.21.15.57:0/3344559361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7010105dd0 msgr2=0x7f70100768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f700de00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7010105dd0 0x7f70100768e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f700de00640 1 -- 172.21.15.57:0/3344559361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70100771a0 con 0x7f7010104220 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f700de00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 0x7f7010079d70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f700000d950 tx=0x7f700000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.754+0000 7f7006a00640 1 -- 172.21.15.57:0/3344559361 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000017070 con 0x7f7010104220 2024-04-13T05:22:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.755+0000 7f7006a00640 1 -- 172.21.15.57:0/3344559361 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7000009c30 con 0x7f7010104220 2024-04-13T05:22:28.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.755+0000 7f7006a00640 1 -- 172.21.15.57:0/3344559361 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700000f620 con 0x7f7010104220 2024-04-13T05:22:28.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.755+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70101a66d0 con 0x7f7010104220 2024-04-13T05:22:28.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.755+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70101a6a80 con 0x7f7010104220 2024-04-13T05:22:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.759+0000 7f7017da4740 1 Processor -- start 2024-04-13T05:22:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.759+0000 7f7017da4740 1 -- start start 2024-04-13T05:22:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f7017da4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab04927250 0x55ab04927670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f7017da4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927c40 0x55ab0492c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f7017da4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab0492cdb0 0x55ab0492f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f7017da4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ab0488d020 con 0x55ab0492cdb0 2024-04-13T05:22:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f7017da4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ab04833950 con 0x55ab04927250 2024-04-13T05:22:28.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f7017da4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ab04888fe0 con 0x55ab04927c40 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f700d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab0492cdb0 0x55ab0492f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f700d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab0492cdb0 0x55ab0492f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54866/0 (socket says 172.21.15.57:54866) 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f700d400640 1 -- 172.21.15.57:0/2193574430 learned_addr learned my addr 172.21.15.57:0/2193574430 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.760+0000 7f700e800640 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab04927250 0x55ab04927670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f700e800640 1 -- 172.21.15.57:0/2193574430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927c40 msgr2=0x55ab0492c740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f700e800640 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927c40 0x55ab0492c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f700e800640 1 -- 172.21.15.57:0/2193574430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab0492cdb0 msgr2=0x55ab0492f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f700e800640 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab0492cdb0 0x55ab0492f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f700e800640 1 -- 172.21.15.57:0/2193574430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0459f3f0 con 0x55ab04927250 2024-04-13T05:22:28.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f700e800640 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab04927250 0x55ab04927670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7008009f00 tx=0x7f7008011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33387a58b4efa091 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.761+0000 7f7006a00640 1 -- 172.21.15.57:0/3344559361 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f700001a070 con 0x7f7010104220 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.762+0000 7f6ffb400640 1 -- 172.21.15.57:0/2193574430 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008004d20 con 0x55ab04927250 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.762+0000 7f6ffb400640 1 -- 172.21.15.57:0/2193574430 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008004030 con 0x55ab04927250 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.762+0000 7f6ffb400640 1 -- 172.21.15.57:0/2193574430 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008005260 con 0x55ab04927250 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 -- 172.21.15.57:0/2193574430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab04927250 msgr2=0x55ab04927670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab04927250 0x55ab04927670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7008009f00 tx=0x7f7008011810 comp rx=0 tx=0).stop 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 -- 172.21.15.57:0/2193574430 shutdown_connections 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab0492cdb0 0x55ab0492f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927c40 0x55ab0492c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 --2- 172.21.15.57:0/2193574430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab04927250 0x55ab04927670 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 -- 172.21.15.57:0/2193574430 >> 172.21.15.57:0/2193574430 conn(0x55ab049227c0 msgr2=0x55ab04922bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 -- 172.21.15.57:0/2193574430 shutdown_connections 2024-04-13T05:22:28.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.763+0000 7f7017da4740 1 -- 172.21.15.57:0/2193574430 wait complete. 2024-04-13T05:22:28.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.764+0000 7f7017da4740 1 -- start start 2024-04-13T05:22:28.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.764+0000 7f7017da4740 1 Processor -- start 2024-04-13T05:22:28.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.764+0000 7f7006a00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ff40a3bc0 0x7f6ff40a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 0x55ab04931d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04938d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ab0492fd60 con 0x55ab04927c40 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ab04846c80 con 0x55ab0492cdb0 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ab0492fee0 con 0x55ab04927250 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f7017da4740 10 client.0 _ll_register_callbacks cb 0x55ab04846670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f700d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 0x55ab04931d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f700d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 0x55ab04931d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54868/0 (socket says 172.21.15.57:54868) 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f700d400640 1 -- 172.21.15.57:0/774085928 learned_addr learned my addr 172.21.15.57:0/774085928 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.765+0000 7f700e800640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ff40a3bc0 0x7f6ff40a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f7006a00640 1 -- 172.21.15.57:0/3344559361 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f70000d0180 con 0x7f7010104220 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f700d400640 1 -- 172.21.15.57:0/774085928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 msgr2=0x55ab049317f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f700de00640 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f700d400640 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f700d400640 1 -- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 msgr2=0x55ab04938d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f700d400640 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04938d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.766+0000 7f700d400640 1 -- 172.21.15.57:0/774085928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab046697d0 con 0x55ab04927c40 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.767+0000 7f700e800640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ff40a3bc0 0x7f6ff40a6080 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7008011d10 tx=0x7f7008005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.767+0000 7f700d400640 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 0x55ab04931d30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc036040 tx=0x7f6ffc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7561e3de05e838d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.767+0000 7f6fefe00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.767+0000 7f6fefe00640 1 -- 172.21.15.57:0/774085928 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc044070 con 0x55ab04927c40 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.767+0000 7f6fefe00640 1 -- 172.21.15.57:0/774085928 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6ffc005030 con 0x55ab04927c40 2024-04-13T05:22:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.767+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008004030 con 0x55ab04927c40 2024-04-13T05:22:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.768+0000 7f6fefe00640 1 -- 172.21.15.57:0/774085928 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ffc005230 con 0x55ab04927c40 2024-04-13T05:22:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.768+0000 7f7017da4740 10 client.16440 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.768+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab0493c150 con 0x55ab04927c40 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.768+0000 7f6fefe00640 1 -- 172.21.15.57:0/774085928 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008004030 con 0x55ab04927c40 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.768+0000 7f6fefe00640 1 -- 172.21.15.57:0/774085928 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6ffc0035f0 con 0x55ab04927c40 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f7017da4740 10 client.16440 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f7017da4740 10 client.16440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f7017da4740 20 client.16440 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f7017da4740 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab0493d8c0 0x55ab0493fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f7017da4740 4 client.16440 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0482c750 con 0x55ab0493d8c0 2024-04-13T05:22:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.769+0000 7f700de00640 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab0493d8c0 0x55ab0493fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.770+0000 7f700de00640 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab0493d8c0 0x55ab0493fce0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f59a4580f033d4e server_cookie=ed0ad5d0908f450d in_seq=0 out_seq=0 2024-04-13T05:22:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.770+0000 7f6fefe00640 10 client.16440 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:22:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.771+0000 7f6fefe00640 1 -- 172.21.15.57:0/774085928 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab0482c750 con 0x55ab0493d8c0 2024-04-13T05:22:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.771+0000 7f6fefe00640 10 client.16440 handle_command_reply: tid=0 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.771+0000 7f7017da4740 1 client.16440 shutdown 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.771+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 msgr2=0x55ab04931d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.771+0000 7f7017da4740 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 0x55ab04931d30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc036040 tx=0x7f6ffc004250 comp rx=0 tx=0).stop 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 shutdown_connections 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab0493d8c0 0x55ab0493fce0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04938d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927c40 0x55ab04931d30 secure :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc036040 tx=0x7f6ffc004250 comp rx=0 tx=0).stop 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 --2- 172.21.15.57:0/774085928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 >> 172.21.15.57:0/774085928 conn(0x55ab049227c0 msgr2=0x55ab04921a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f6fefe00640 0 client.16440 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:22:28.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 shutdown_connections 2024-04-13T05:22:28.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 1 -- 172.21.15.57:0/774085928 wait complete. 2024-04-13T05:22:28.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.772+0000 7f7017da4740 20 client.16440 trim_cache size 0 max 16384 2024-04-13T05:22:28.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.792+0000 7f7017da4740 1 Processor -- start 2024-04-13T05:22:28.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 -- start start 2024-04-13T05:22:28.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927250 0x55ab04930e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab0492cdb0 0x55ab04938d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab049309c0 0x55ab049211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f70101b3840 con 0x55ab04927250 2024-04-13T05:22:28.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70101b10b0 con 0x55ab049309c0 2024-04-13T05:22:28.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.793+0000 7f7017da4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70101aef00 con 0x55ab0492cdb0 2024-04-13T05:22:28.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab049309c0 0x55ab049211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927250 0x55ab04930e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab0492cdb0 0x55ab04938d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab049309c0 0x55ab049211f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46522/0 (socket says 172.21.15.57:46522) 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700e800640 1 -- 172.21.15.57:0/595875112 learned_addr learned my addr 172.21.15.57:0/595875112 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700d400640 1 -- 172.21.15.57:0/595875112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab049309c0 msgr2=0x55ab049211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700d400640 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab049309c0 0x55ab049211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.794+0000 7f700d400640 1 -- 172.21.15.57:0/595875112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927250 msgr2=0x55ab04930e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.795+0000 7f700d400640 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927250 0x55ab04930e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.795+0000 7f700d400640 1 -- 172.21.15.57:0/595875112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab04926e90 con 0x55ab0492cdb0 2024-04-13T05:22:28.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.795+0000 7f700de00640 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927250 0x55ab04930e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:28.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.795+0000 7f700d400640 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab0492cdb0 0x55ab04938d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc042000 tx=0x7f6ffc02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7de77367d65a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.796+0000 7f6ffaa00640 1 -- 172.21.15.57:0/595875112 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc044040 con 0x55ab0492cdb0 2024-04-13T05:22:28.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.796+0000 7f6ffaa00640 1 -- 172.21.15.57:0/595875112 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ffc003c80 con 0x55ab0492cdb0 2024-04-13T05:22:28.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.796+0000 7f6ffaa00640 1 -- 172.21.15.57:0/595875112 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc005630 con 0x55ab0492cdb0 2024-04-13T05:22:28.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 -- 172.21.15.57:0/595875112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab0492cdb0 msgr2=0x55ab04938d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab0492cdb0 0x55ab04938d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc042000 tx=0x7f6ffc02bc30 comp rx=0 tx=0).stop 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 -- 172.21.15.57:0/595875112 shutdown_connections 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab049309c0 0x55ab049211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab0492cdb0 0x55ab04938d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 --2- 172.21.15.57:0/595875112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab04927250 0x55ab04930e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 -- 172.21.15.57:0/595875112 >> 172.21.15.57:0/595875112 conn(0x55ab049227c0 msgr2=0x55ab049302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.797+0000 7f7017da4740 1 -- 172.21.15.57:0/595875112 shutdown_connections 2024-04-13T05:22:28.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.798+0000 7f7017da4740 1 -- 172.21.15.57:0/595875112 wait complete. 2024-04-13T05:22:28.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.798+0000 7f7017da4740 1 -- start start 2024-04-13T05:22:28.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.798+0000 7f7017da4740 1 Processor -- start 2024-04-13T05:22:28.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04936d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab049309c0 0x55ab04937ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f70101af570 con 0x55ab049309c0 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70101b09e0 con 0x55ab0492cdb0 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70101afc40 con 0x55ab04927250 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f7017da4740 10 client.0 _ll_register_callbacks cb 0x7f6ffc02ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f700d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04936d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f700d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04936d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46538/0 (socket says 172.21.15.57:46538) 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f700d400640 1 -- 172.21.15.57:0/3574712488 learned_addr learned my addr 172.21.15.57:0/3574712488 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f700de00640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.799+0000 7f700e800640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab049309c0 0x55ab04937ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.800+0000 7f700d400640 1 -- 172.21.15.57:0/3574712488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 msgr2=0x55ab049367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.800+0000 7f700d400640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049367c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.800+0000 7f700d400640 1 -- 172.21.15.57:0/3574712488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab049309c0 msgr2=0x55ab04937ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.800+0000 7f700d400640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab049309c0 0x55ab04937ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.800+0000 7f700d400640 1 -- 172.21.15.57:0/3574712488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab04938290 con 0x55ab0492cdb0 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.801+0000 7f700d400640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04936d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc003a90 tx=0x7f6ffc005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7211d0681701420 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.801+0000 7f6fefe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:22:28.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.801+0000 7f6fefe00640 1 -- 172.21.15.57:0/3574712488 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc05e060 con 0x55ab0492cdb0 2024-04-13T05:22:28.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.801+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ffc003c80 con 0x55ab0492cdb0 2024-04-13T05:22:28.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.801+0000 7f6fefe00640 1 -- 172.21.15.57:0/3574712488 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6ffc004fd0 con 0x55ab0492cdb0 2024-04-13T05:22:28.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.802+0000 7f6fefe00640 1 -- 172.21.15.57:0/3574712488 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ffc065050 con 0x55ab0492cdb0 2024-04-13T05:22:28.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.802+0000 7f7017da4740 10 client.35864 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:28.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.802+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab049a4180 con 0x55ab0492cdb0 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.802+0000 7f6fefe00640 1 -- 172.21.15.57:0/3574712488 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ffc003c80 con 0x55ab0492cdb0 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f6fefe00640 1 -- 172.21.15.57:0/3574712488 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6ffc052420 con 0x55ab0492cdb0 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f7017da4740 10 client.35864 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f7017da4740 10 client.35864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f7017da4740 20 client.35864 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f7017da4740 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab049a4a90 0x55ab049a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f7017da4740 4 client.35864 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:22:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab049315e0 con 0x55ab049a4a90 2024-04-13T05:22:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.803+0000 7f700de00640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab049a4a90 0x55ab049a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.804+0000 7f700de00640 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab049a4a90 0x55ab049a6eb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0478a59f07d828f server_cookie=84f15059e9e15a47 in_seq=0 out_seq=0 2024-04-13T05:22:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.804+0000 7f6fefe00640 10 client.35864 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:22:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.804+0000 7f6fefe00640 1 -- 172.21.15.57:0/3574712488 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab049315e0 con 0x55ab049a4a90 2024-04-13T05:22:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.804+0000 7f6fefe00640 10 client.35864 handle_command_reply: tid=0 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.804+0000 7f7017da4740 1 client.35864 shutdown 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 msgr2=0x55ab04936d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04936d00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc003a90 tx=0x7f6ffc005780 comp rx=0 tx=0).stop 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 shutdown_connections 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ab049a4a90 0x55ab049a6eb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab0492cdb0 0x55ab04936d00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ab049309c0 0x55ab04937ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 --2- 172.21.15.57:0/3574712488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab04927250 0x55ab049367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f6fefe00640 0 client.35864 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 >> 172.21.15.57:0/3574712488 conn(0x55ab049227c0 msgr2=0x55ab04930da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 shutdown_connections 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 1 -- 172.21.15.57:0/3574712488 wait complete. 2024-04-13T05:22:28.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.805+0000 7f7017da4740 20 client.35864 trim_cache size 0 max 16384 2024-04-13T05:22:28.809 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:22:28.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ff40a3bc0 msgr2=0x7f6ff40a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ff40a3bc0 0x7f6ff40a6080 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7008011d10 tx=0x7f7008005fd0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 msgr2=0x7f7010079d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 0x7f7010079d70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f700000d950 tx=0x7f700000de10 comp rx=0 tx=0).stop 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 shutdown_connections 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6ff40a3bc0 0x7f6ff40a6080 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7010105dd0 0x7f70100768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7010105410 0x7f701007a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 --2- 172.21.15.57:0/3344559361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7010104220 0x7f7010079d70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 >> 172.21.15.57:0/3344559361 conn(0x7f70100ff770 msgr2=0x7f7010108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.806+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 shutdown_connections 2024-04-13T05:22:28.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:28.807+0000 7f7014e00640 1 -- 172.21.15.57:0/3344559361 wait complete. 2024-04-13T05:22:28.830 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 -- 172.21.15.57:0/1745553668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc664101b90 msgr2=0x7fc6641000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 --2- 172.21.15.57:0/1745553668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc664101b90 0x7fc6641000c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009ca0 tx=0x7fc65c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 -- 172.21.15.57:0/1745553668 shutdown_connections 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 --2- 172.21.15.57:0/1745553668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc664101b90 0x7fc6641000c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 --2- 172.21.15.57:0/1745553668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641011d0 0x7fc664101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 --2- 172.21.15.57:0/1745553668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641024a0 0x7fc6641028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 -- 172.21.15.57:0/1745553668 >> 172.21.15.57:0/1745553668 conn(0x7fc6640fbc00 msgr2=0x7fc6640fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.087+0000 7fc66ba00640 1 -- 172.21.15.57:0/1745553668 shutdown_connections 2024-04-13T05:22:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.088+0000 7fc66ba00640 1 -- 172.21.15.57:0/1745553668 wait complete. 2024-04-13T05:22:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 Processor -- start 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 -- start start 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6641011d0 0x7fc66419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 0x7fc66419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641024a0 0x7fc6641a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc66410fde0 con 0x7fc6641024a0 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc66410fc60 con 0x7fc6641011d0 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.089+0000 7fc66ba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc66410ff60 con 0x7fc664101b90 2024-04-13T05:22:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.090+0000 7fc669a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641024a0 0x7fc6641a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.090+0000 7fc663e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 0x7fc66419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.090+0000 7fc663e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 0x7fc66419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:52912/0 (socket says 172.21.15.57:52912) 2024-04-13T05:22:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.090+0000 7fc663e00640 1 -- 172.21.15.57:0/395631197 learned_addr learned my addr 172.21.15.57:0/395631197 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.090+0000 7fc669000640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6641011d0 0x7fc66419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc663e00640 1 -- 172.21.15.57:0/395631197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6641011d0 msgr2=0x7fc66419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc663e00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6641011d0 0x7fc66419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc663e00640 1 -- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641024a0 msgr2=0x7fc6641a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc663e00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641024a0 0x7fc6641a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc663e00640 1 -- 172.21.15.57:0/395631197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6641a2010 con 0x7fc664101b90 2024-04-13T05:22:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc669a00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641024a0 0x7fc6641a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.091+0000 7fc663e00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 0x7fc66419a930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ea60 tx=0x7fc65800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.092+0000 7fc661600640 1 -- 172.21.15.57:0/395631197 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65800cde0 con 0x7fc664101b90 2024-04-13T05:22:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.092+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6641a2300 con 0x7fc664101b90 2024-04-13T05:22:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.092+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641a28a0 con 0x7fc664101b90 2024-04-13T05:22:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.092+0000 7fc661600640 1 -- 172.21.15.57:0/395631197 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc658009470 con 0x7fc664101b90 2024-04-13T05:22:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.092+0000 7fc661600640 1 -- 172.21.15.57:0/395631197 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658010430 con 0x7fc664101b90 2024-04-13T05:22:29.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.096+0000 7fc66e837740 1 Processor -- start 2024-04-13T05:22:29.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.096+0000 7fc66e837740 1 -- start start 2024-04-13T05:22:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc66e837740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b432a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc66e837740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b432f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc66e837740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432fdb0 0x5568b4332240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc66e837740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5568b4290020 con 0x5568b432ac40 2024-04-13T05:22:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc66e837740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5568b4236950 con 0x5568b432fdb0 2024-04-13T05:22:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc66e837740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5568b428bfe0 con 0x5568b432a250 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc669000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b432f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc669000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b432f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54920/0 (socket says 172.21.15.57:54920) 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc669000640 1 -- 172.21.15.57:0/301070679 learned_addr learned my addr 172.21.15.57:0/301070679 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.097+0000 7fc669a00640 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b432a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.098+0000 7fc669000640 1 -- 172.21.15.57:0/301070679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 msgr2=0x5568b432a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.098+0000 7fc669000640 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b432a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.098+0000 7fc669000640 1 -- 172.21.15.57:0/301070679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432fdb0 msgr2=0x5568b4332240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.098+0000 7fc669000640 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432fdb0 0x5568b4332240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.098+0000 7fc669000640 1 -- 172.21.15.57:0/301070679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b3fa23f0 con 0x5568b432ac40 2024-04-13T05:22:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.099+0000 7fc669000640 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b432f740 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc654011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9edfdd727d6a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.099+0000 7fc661600640 1 -- 172.21.15.57:0/395631197 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc6580035f0 con 0x7fc664101b90 2024-04-13T05:22:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.099+0000 7fc652000640 1 -- 172.21.15.57:0/301070679 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654004220 con 0x5568b432ac40 2024-04-13T05:22:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.099+0000 7fc652000640 1 -- 172.21.15.57:0/301070679 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc654009e50 con 0x5568b432ac40 2024-04-13T05:22:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.099+0000 7fc652000640 1 -- 172.21.15.57:0/301070679 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654003ba0 con 0x5568b432ac40 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 -- 172.21.15.57:0/301070679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 msgr2=0x5568b432f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b432f740 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc654011810 comp rx=0 tx=0).stop 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 -- 172.21.15.57:0/301070679 shutdown_connections 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432fdb0 0x5568b4332240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b432f740 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 --2- 172.21.15.57:0/301070679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b432a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 -- 172.21.15.57:0/301070679 >> 172.21.15.57:0/301070679 conn(0x5568b43257c0 msgr2=0x5568b4325bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 -- 172.21.15.57:0/301070679 shutdown_connections 2024-04-13T05:22:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.101+0000 7fc66e837740 1 -- 172.21.15.57:0/301070679 wait complete. 2024-04-13T05:22:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.102+0000 7fc66e837740 1 -- start start 2024-04-13T05:22:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.102+0000 7fc66e837740 1 Processor -- start 2024-04-13T05:22:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b43347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432ac40 0x5568b4334d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b433bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5568b4332d60 con 0x5568b432fdb0 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5568b4249c80 con 0x5568b432ac40 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5568b4332ee0 con 0x5568b432a250 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc66e837740 10 client.0 _ll_register_callbacks cb 0x5568b4249670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.103+0000 7fc661600640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc64c0a3b40 0x7fc64c0a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc661600640 1 -- 172.21.15.57:0/395631197 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc6580d08c0 con 0x7fc664101b90 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc669a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b43347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc669a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b43347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:52926/0 (socket says 172.21.15.57:52926) 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc669a00640 1 -- 172.21.15.57:0/1928572988 learned_addr learned my addr 172.21.15.57:0/1928572988 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc669a00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc64c0a3b40 0x7fc64c0a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc663e00640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b433bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.104+0000 7fc669000640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432ac40 0x5568b4334d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.105+0000 7fc663e00640 1 -- 172.21.15.57:0/1928572988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 msgr2=0x5568b43347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.105+0000 7fc663e00640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b43347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.105+0000 7fc663e00640 1 -- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432ac40 msgr2=0x5568b4334d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.105+0000 7fc663e00640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432ac40 0x5568b4334d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.105+0000 7fc663e00640 1 -- 172.21.15.57:0/1928572988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b406c7d0 con 0x5568b432fdb0 2024-04-13T05:22:29.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.106+0000 7fc663e00640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b433bd60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ea30 tx=0x7fc6580039a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77a180c839d2919 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:29.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.106+0000 7fc669a00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc64c0a3b40 0x7fc64c0a6000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc65c032550 tx=0x7fc65c009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:29.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.106+0000 7fc646a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:29.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.106+0000 7fc646a00640 1 -- 172.21.15.57:0/1928572988 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6580a0b50 con 0x5568b432fdb0 2024-04-13T05:22:29.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.107+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc654009e50 con 0x5568b432fdb0 2024-04-13T05:22:29.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.107+0000 7fc646a00640 1 -- 172.21.15.57:0/1928572988 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc6580870c0 con 0x5568b432fdb0 2024-04-13T05:22:29.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.107+0000 7fc646a00640 1 -- 172.21.15.57:0/1928572988 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6580a0b50 con 0x5568b432fdb0 2024-04-13T05:22:29.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.107+0000 7fc66e837740 10 client.16458 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:29.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.107+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568b433f1f0 con 0x5568b432fdb0 2024-04-13T05:22:29.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc646a00640 1 -- 172.21.15.57:0/1928572988 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc654009e50 con 0x5568b432fdb0 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc646a00640 1 -- 172.21.15.57:0/1928572988 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc658098600 con 0x5568b432fdb0 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc66e837740 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc66e837740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc66e837740 20 client.16458 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc66e837740 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b4340810 0x5568b4342c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc66e837740 4 client.16458 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.108+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b422f750 con 0x5568b4340810 2024-04-13T05:22:29.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.109+0000 7fc669000640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b4340810 0x5568b4342c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.110+0000 7fc669000640 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b4340810 0x5568b4342c70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52ee2a3fd8aca28 server_cookie=3f8d784ec375f853 in_seq=0 out_seq=0 2024-04-13T05:22:29.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.110+0000 7fc646a00640 10 client.16458 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:29.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.111+0000 7fc646a00640 1 -- 172.21.15.57:0/1928572988 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568b422f750 con 0x5568b4340810 2024-04-13T05:22:29.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.111+0000 7fc646a00640 10 client.16458 handle_command_reply: tid=0 2024-04-13T05:22:29.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.111+0000 7fc66e837740 1 client.16458 shutdown 2024-04-13T05:22:29.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 msgr2=0x5568b433bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b433bd60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ea30 tx=0x7fc6580039a0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 shutdown_connections 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b4340810 0x5568b4342c70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432ac40 0x5568b4334d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b433bd60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 --2- 172.21.15.57:0/1928572988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432a250 0x5568b43347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 >> 172.21.15.57:0/1928572988 conn(0x5568b43257c0 msgr2=0x5568b4324a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc646a00640 0 client.16458 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 shutdown_connections 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 1 -- 172.21.15.57:0/1928572988 wait complete. 2024-04-13T05:22:29.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.112+0000 7fc66e837740 20 client.16458 trim_cache size 0 max 16384 2024-04-13T05:22:29.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.136+0000 7fc66e837740 1 Processor -- start 2024-04-13T05:22:29.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.136+0000 7fc66e837740 1 -- start start 2024-04-13T05:22:29.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc66e837740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b4329130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc66e837740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432ac40 0x5568b433bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc66e837740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b4347530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc66e837740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc6480124a0 con 0x5568b432fdb0 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc66e837740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800fd10 con 0x5568b432a250 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc66e837740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800db60 con 0x5568b432ac40 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc669000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b4329130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc669000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b4329130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46572/0 (socket says 172.21.15.57:46572) 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.137+0000 7fc669000640 1 -- 172.21.15.57:0/3142127872 learned_addr learned my addr 172.21.15.57:0/3142127872 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc663e00640 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432ac40 0x5568b433bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669a00640 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b4347530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669000640 1 -- 172.21.15.57:0/3142127872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432ac40 msgr2=0x5568b433bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669000640 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432ac40 0x5568b433bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669000640 1 -- 172.21.15.57:0/3142127872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 msgr2=0x5568b4347530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669000640 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b4347530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669000640 1 -- 172.21.15.57:0/3142127872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b4320910 con 0x5568b432a250 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc663e00640 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432ac40 0x5568b433bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.138+0000 7fc669000640 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b4329130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc654005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c7ced4589a42db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.139+0000 7fc651600640 1 -- 172.21.15.57:0/3142127872 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654009e50 con 0x5568b432a250 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.139+0000 7fc651600640 1 -- 172.21.15.57:0/3142127872 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc65401a850 con 0x5568b432a250 2024-04-13T05:22:29.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.139+0000 7fc651600640 1 -- 172.21.15.57:0/3142127872 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654004680 con 0x5568b432a250 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 -- 172.21.15.57:0/3142127872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 msgr2=0x5568b4329130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b4329130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc654005fb0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 -- 172.21.15.57:0/3142127872 shutdown_connections 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432fdb0 0x5568b4347530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432ac40 0x5568b433bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 --2- 172.21.15.57:0/3142127872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b4329130 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 -- 172.21.15.57:0/3142127872 >> 172.21.15.57:0/3142127872 conn(0x5568b43257c0 msgr2=0x5568b4328a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 -- 172.21.15.57:0/3142127872 shutdown_connections 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.140+0000 7fc66e837740 1 -- 172.21.15.57:0/3142127872 wait complete. 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.141+0000 7fc66e837740 1 -- start start 2024-04-13T05:22:29.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.141+0000 7fc66e837740 1 Processor -- start 2024-04-13T05:22:29.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b43390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b4339620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432fdb0 0x5568b433a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800e1d0 con 0x5568b432ac40 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800f640 con 0x5568b432a250 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc64800e8a0 con 0x5568b432fdb0 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc66e837740 10 client.0 _ll_register_callbacks cb 0x7fc65809c660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc663e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b4339620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc663e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b4339620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54930/0 (socket says 172.21.15.57:54930) 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc663e00640 1 -- 172.21.15.57:0/3841624004 learned_addr learned my addr 172.21.15.57:0/3841624004 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.142+0000 7fc669000640 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b43390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc663e00640 1 -- 172.21.15.57:0/3841624004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432fdb0 msgr2=0x5568b433a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc663e00640 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432fdb0 0x5568b433a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc663e00640 1 -- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 msgr2=0x5568b43390e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc663e00640 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b43390e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc663e00640 1 -- 172.21.15.57:0/3841624004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b433ac20 con 0x5568b432ac40 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc663e00640 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b4339620 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc65809c4a0 tx=0x7fc6580098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7e1807b6c1265f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc646a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc646a00640 1 -- 172.21.15.57:0/3841624004 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65809c100 con 0x5568b432ac40 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568b429da40 con 0x5568b432ac40 2024-04-13T05:22:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.143+0000 7fc646a00640 1 -- 172.21.15.57:0/3841624004 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc6580a5180 con 0x5568b432ac40 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc646a00640 1 -- 172.21.15.57:0/3841624004 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6580a5410 con 0x5568b432ac40 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc66e837740 10 client.16464 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568b433bf20 con 0x5568b432ac40 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc646a00640 1 -- 172.21.15.57:0/3841624004 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5568b429da40 con 0x5568b432ac40 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc646a00640 1 -- 172.21.15.57:0/3841624004 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc65809b870 con 0x5568b432ac40 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc66e837740 10 client.16464 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc66e837740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc66e837740 20 client.16464 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.144+0000 7fc66e837740 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b43a7380 0x5568b43a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:29.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.145+0000 7fc66e837740 4 client.16464 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:22:29.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.145+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568b42978b0 con 0x5568b43a7380 2024-04-13T05:22:29.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.145+0000 7fc669000640 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b43a7380 0x5568b43a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:29.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc669000640 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b43a7380 0x5568b43a97a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f86d57cdc46026 server_cookie=90041fcb3d7870f2 in_seq=0 out_seq=0 2024-04-13T05:22:29.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc646a00640 10 client.16464 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc646a00640 1 -- 172.21.15.57:0/3841624004 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568b42978b0 con 0x5568b43a7380 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc646a00640 10 client.16464 handle_command_reply: tid=0 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc66e837740 1 client.16464 shutdown 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 msgr2=0x5568b4339620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.146+0000 7fc66e837740 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b4339620 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc65809c4a0 tx=0x7fc6580098b0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 shutdown_connections 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5568b43a7380 0x5568b43a97a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568b432fdb0 0x5568b433a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568b432ac40 0x5568b4339620 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 --2- 172.21.15.57:0/3841624004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568b432a250 0x5568b43390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc646a00640 0 client.16464 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:29.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 >> 172.21.15.57:0/3841624004 conn(0x5568b43257c0 msgr2=0x5568b43292b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:29.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 shutdown_connections 2024-04-13T05:22:29.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 1 -- 172.21.15.57:0/3841624004 wait complete. 2024-04-13T05:22:29.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.147+0000 7fc66e837740 20 client.16464 trim_cache size 0 max 16384 2024-04-13T05:22:29.151 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc64c0a3b40 msgr2=0x7fc64c0a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc64c0a3b40 0x7fc64c0a6000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc65c032550 tx=0x7fc65c009800 comp rx=0 tx=0).stop 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 msgr2=0x7fc66419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 0x7fc66419a930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ea60 tx=0x7fc65800ef20 comp rx=0 tx=0).stop 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 shutdown_connections 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc64c0a3b40 0x7fc64c0a6000 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6641024a0 0x7fc6641a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664101b90 0x7fc66419a930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 --2- 172.21.15.57:0/395631197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6641011d0 0x7fc66419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.148+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 >> 172.21.15.57:0/395631197 conn(0x7fc6640fbc00 msgr2=0x7fc664103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.149+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 shutdown_connections 2024-04-13T05:22:29.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:29.149+0000 7fc66ba00640 1 -- 172.21.15.57:0/395631197 wait complete. 2024-04-13T05:22:29.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:29 smithi057 ceph-mon[32331]: pgmap v602: 129 pgs: 129 active+clean; 236 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-13T05:22:29.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:29 smithi189 ceph-mon[35531]: pgmap v602: 129 pgs: 129 active+clean; 236 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-13T05:22:29.637 INFO:tasks.workunit.client.0.smithi057.stdout: 41 8090 62 5433 53 2495 91 2024-04-13T05:22:29.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:29 smithi083 ceph-mon[35133]: pgmap v602: 129 pgs: 129 active+clean; 236 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-04-13T05:22:30.174 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:22:30.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:22:30.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:22:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 -- 172.21.15.57:0/3528357278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d8104100 msgr2=0x7fb7d8104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/3528357278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d8104100 0x7fb7d8104520 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009900 tx=0x7fb7c802bc40 comp rx=0 tx=0).stop 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 -- 172.21.15.57:0/3528357278 shutdown_connections 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/3528357278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d81052f0 0x7fb7d8105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/3528357278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7d8105cb0 0x7fb7d810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/3528357278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d8104100 0x7fb7d8104520 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.452+0000 7fb7d7e00640 1 -- 172.21.15.57:0/3528357278 >> 172.21.15.57:0/3528357278 conn(0x7fb7d80ff670 msgr2=0x7fb7d8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.453+0000 7fb7d7e00640 1 -- 172.21.15.57:0/3528357278 shutdown_connections 2024-04-13T05:22:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.453+0000 7fb7d7e00640 1 -- 172.21.15.57:0/3528357278 wait complete. 2024-04-13T05:22:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.454+0000 7fb7d7e00640 1 Processor -- start 2024-04-13T05:22:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.454+0000 7fb7d7e00640 1 -- start start 2024-04-13T05:22:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.454+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7d8104100 0x7fb7d819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.454+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d81052f0 0x7fb7d819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8105cb0 0x7fb7d81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d7e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d810fc90 con 0x7fb7d81052f0 2024-04-13T05:22:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d810fb10 con 0x7fb7d8105cb0 2024-04-13T05:22:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d810fe10 con 0x7fb7d8104100 2024-04-13T05:22:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d81052f0 0x7fb7d819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d81052f0 0x7fb7d819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54950/0 (socket says 172.21.15.57:54950) 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d6000640 1 -- 172.21.15.57:0/1659350453 learned_addr learned my addr 172.21.15.57:0/1659350453 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8105cb0 0x7fb7d81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.455+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7d8104100 0x7fb7d819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7d7400640 1 -- 172.21.15.57:0/1659350453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7d8104100 msgr2=0x7fb7d819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7d7400640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7d8104100 0x7fb7d819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7d7400640 1 -- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d81052f0 msgr2=0x7fb7d819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7d7400640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d81052f0 0x7fb7d819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7d7400640 1 -- 172.21.15.57:0/1659350453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d81a1f90 con 0x7fb7d8105cb0 2024-04-13T05:22:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7d7400640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8105cb0 0x7fb7d81a1920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000e940 tx=0x7fb7d000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.456+0000 7fb7cf400640 1 -- 172.21.15.57:0/1659350453 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0004d20 con 0x7fb7d8105cb0 2024-04-13T05:22:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.457+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d81a2280 con 0x7fb7d8105cb0 2024-04-13T05:22:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.457+0000 7fb7cf400640 1 -- 172.21.15.57:0/1659350453 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7d0005020 con 0x7fb7d8105cb0 2024-04-13T05:22:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.457+0000 7fb7cf400640 1 -- 172.21.15.57:0/1659350453 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00116d0 con 0x7fb7d8105cb0 2024-04-13T05:22:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.457+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d81a27d0 con 0x7fb7d8105cb0 2024-04-13T05:22:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.460+0000 7fb7e077c740 1 Processor -- start 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.462+0000 7fb7cf400640 1 -- 172.21.15.57:0/1659350453 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb7d000cb20 con 0x7fb7d8105cb0 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 -- start start 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0dc30 0x55e430a0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a13120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e43096bbc0 con 0x55e430a13790 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e430912660 con 0x55e430a0dc30 2024-04-13T05:22:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.463+0000 7fb7e077c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e430967b80 con 0x55e430a0e620 2024-04-13T05:22:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0dc30 0x55e430a0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a15c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a15c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54964/0 (socket says 172.21.15.57:54964) 2024-04-13T05:22:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 -- 172.21.15.57:0/1733856241 learned_addr learned my addr 172.21.15.57:0/1733856241 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a13120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 -- 172.21.15.57:0/1733856241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 msgr2=0x55e430a13120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a13120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 -- 172.21.15.57:0/1733856241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0dc30 msgr2=0x55e430a0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0dc30 0x55e430a0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.464+0000 7fb7d6000640 1 -- 172.21.15.57:0/1733856241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e430935220 con 0x55e430a13790 2024-04-13T05:22:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.465+0000 7fb7cf400640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb7bc0a3c30 0x7fb7bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.465+0000 7fb7d6000640 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a15c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4010a00 tx=0x7fb7c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaab56ba82ec4442 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.465+0000 7fb7cf400640 1 -- 172.21.15.57:0/1659350453 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb7d00d0580 con 0x7fb7d8105cb0 2024-04-13T05:22:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.465+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb7bc0a3c30 0x7fb7bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.466+0000 7fb7c3e00640 1 -- 172.21.15.57:0/1733856241 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4016030 con 0x55e430a13790 2024-04-13T05:22:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.466+0000 7fb7c3e00640 1 -- 172.21.15.57:0/1733856241 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7c401e3b0 con 0x55e430a13790 2024-04-13T05:22:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.466+0000 7fb7c3e00640 1 -- 172.21.15.57:0/1733856241 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4019040 con 0x55e430a13790 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.466+0000 7fb7e077c740 1 -- 172.21.15.57:0/1733856241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 msgr2=0x55e430a15c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.466+0000 7fb7e077c740 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a15c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4010a00 tx=0x7fb7c4005f90 comp rx=0 tx=0).stop 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 -- 172.21.15.57:0/1733856241 shutdown_connections 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a15c20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a13120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 --2- 172.21.15.57:0/1733856241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0dc30 0x55e430a0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 -- 172.21.15.57:0/1733856241 >> 172.21.15.57:0/1733856241 conn(0x55e430a00990 msgr2=0x55e430a00d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 -- 172.21.15.57:0/1733856241 shutdown_connections 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb7bc0a3c30 0x7fb7bc0a60f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb7c802c250 tx=0x7fb7c8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.467+0000 7fb7e077c740 1 -- 172.21.15.57:0/1733856241 wait complete. 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.468+0000 7fb7e077c740 1 -- start start 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.468+0000 7fb7e077c740 1 Processor -- start 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.468+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a13790 0x55e430a1f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7e077c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e430a16740 con 0x55e430a0dc30 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7e077c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e430925a30 con 0x55e430a0e620 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7e077c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e430a168c0 con 0x55e430a13790 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7e077c740 10 client.0 _ll_register_callbacks cb 0x55e43067dc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7d6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7d6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54980/0 (socket says 172.21.15.57:54980) 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7d6a00640 1 -- 172.21.15.57:0/2762940325 learned_addr learned my addr 172.21.15.57:0/2762940325 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7d6000640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.469+0000 7fb7d7400640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a13790 0x55e430a1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6a00640 1 -- 172.21.15.57:0/2762940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a13790 msgr2=0x55e430a1f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a13790 0x55e430a1f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6a00640 1 -- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 msgr2=0x55e430a186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6a00640 1 -- 172.21.15.57:0/2762940325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4307467e0 con 0x55e430a0dc30 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6000640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a181b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8000c00 tx=0x7fb7c802bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0039436b618924 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.470+0000 7fb7c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7c0c00640 1 -- 172.21.15.57:0/2762940325 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c804f030 con 0x55e430a0dc30 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7c0c00640 1 -- 172.21.15.57:0/2762940325 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb7c8005390 con 0x55e430a0dc30 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c401e3b0 con 0x55e430a0dc30 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7c0c00640 1 -- 172.21.15.57:0/2762940325 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7c805c050 con 0x55e430a0dc30 2024-04-13T05:22:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7e077c740 10 client.16476 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4307a6f70 con 0x55e430a0dc30 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.471+0000 7fb7c0c00640 1 -- 172.21.15.57:0/2762940325 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7c401e3b0 con 0x55e430a0dc30 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7c0c00640 1 -- 172.21.15.57:0/2762940325 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c80037b0 con 0x55e430a0dc30 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7e077c740 10 client.16476 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7e077c740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7e077c740 20 client.16476 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7e077c740 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a23fd0 0x55e430a26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7e077c740 4 client.16476 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:22:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4309795e0 con 0x55e430a23fd0 2024-04-13T05:22:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.472+0000 7fb7d6000640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a23fd0 0x55e430a26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.474+0000 7fb7d6000640 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a23fd0 0x55e430a26440 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151bd9a45a6c6e5b server_cookie=724cf441eb3e1e91 in_seq=0 out_seq=0 2024-04-13T05:22:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.474+0000 7fb7c0c00640 10 client.16476 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.476+0000 7fb7c0c00640 1 -- 172.21.15.57:0/2762940325 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4309795e0 con 0x55e430a23fd0 2024-04-13T05:22:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.476+0000 7fb7c0c00640 10 client.16476 handle_command_reply: tid=0 2024-04-13T05:22:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.476+0000 7fb7e077c740 1 client.16476 shutdown 2024-04-13T05:22:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.476+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 msgr2=0x55e430a181b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.476+0000 7fb7e077c740 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a181b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8000c00 tx=0x7fb7c802bc40 comp rx=0 tx=0).stop 2024-04-13T05:22:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 shutdown_connections 2024-04-13T05:22:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a23fd0 0x55e430a26440 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a13790 0x55e430a1f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 --2- 172.21.15.57:0/2762940325 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a181b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 >> 172.21.15.57:0/2762940325 conn(0x55e430a00990 msgr2=0x55e4309ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7c0c00640 0 client.16476 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 shutdown_connections 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 1 -- 172.21.15.57:0/2762940325 wait complete. 2024-04-13T05:22:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.477+0000 7fb7e077c740 20 client.16476 trim_cache size 0 max 16384 2024-04-13T05:22:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.505+0000 7fb7e077c740 1 Processor -- start 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 -- start start 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0dc30 0x55e430a18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a1f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81ab180 con 0x55e430a13790 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81a89f0 con 0x55e430a0e620 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7e077c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81a6840 con 0x55e430a0dc30 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a04300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54992/0 (socket says 172.21.15.57:54992) 2024-04-13T05:22:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.506+0000 7fb7d7400640 1 -- 172.21.15.57:0/976790519 learned_addr learned my addr 172.21.15.57:0/976790519 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d6000640 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a1f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0dc30 0x55e430a18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d7400640 1 -- 172.21.15.57:0/976790519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0dc30 msgr2=0x55e430a18650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d7400640 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0dc30 0x55e430a18650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d7400640 1 -- 172.21.15.57:0/976790519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 msgr2=0x55e430a1f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d7400640 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a1f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.507+0000 7fb7d7400640 1 -- 172.21.15.57:0/976790519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43063e170 con 0x55e430a13790 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.508+0000 7fb7d7400640 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a04300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005850 tx=0x7fb7d00980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87f62d3646785aa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.508+0000 7fb7c3400640 1 -- 172.21.15.57:0/976790519 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00b4070 con 0x55e430a13790 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.508+0000 7fb7c3400640 1 -- 172.21.15.57:0/976790519 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7d009c460 con 0x55e430a13790 2024-04-13T05:22:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.508+0000 7fb7c3400640 1 -- 172.21.15.57:0/976790519 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d001f040 con 0x55e430a13790 2024-04-13T05:22:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 -- 172.21.15.57:0/976790519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 msgr2=0x55e430a04300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a04300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005850 tx=0x7fb7d00980d0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 -- 172.21.15.57:0/976790519 shutdown_connections 2024-04-13T05:22:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a0e620 0x55e430a1f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a13790 0x55e430a04300 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 --2- 172.21.15.57:0/976790519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0dc30 0x55e430a18650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 -- 172.21.15.57:0/976790519 >> 172.21.15.57:0/976790519 conn(0x55e430a00990 msgr2=0x55e430a03bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 -- 172.21.15.57:0/976790519 shutdown_connections 2024-04-13T05:22:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.509+0000 7fb7e077c740 1 -- 172.21.15.57:0/976790519 wait complete. 2024-04-13T05:22:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 -- start start 2024-04-13T05:22:30.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 Processor -- start 2024-04-13T05:22:30.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a1b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a1b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a13790 0x55e430a1e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81a6eb0 con 0x55e430a0dc30 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.510+0000 7fb7e077c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81a8320 con 0x55e430a13790 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7e077c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81a7580 con 0x55e430a0e620 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7d6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a1b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7d6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a1b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55004/0 (socket says 172.21.15.57:55004) 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7d6a00640 1 -- 172.21.15.57:0/3348426853 learned_addr learned my addr 172.21.15.57:0/3348426853 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7d7400640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a13790 0x55e430a1e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7e077c740 10 client.0 _ll_register_callbacks cb 0x7fb7c8043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.511+0000 7fb7d6000640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a1b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7d6a00640 1 -- 172.21.15.57:0/3348426853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 msgr2=0x55e430a1b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a1b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7d6a00640 1 -- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a13790 msgr2=0x55e430a1e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a13790 0x55e430a1e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7d6a00640 1 -- 172.21.15.57:0/3348426853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e430926be0 con 0x55e430a0dc30 2024-04-13T05:22:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7d6a00640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a1b3c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8036040 tx=0x7fb7c802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adcce16214a27bf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.512+0000 7fb7c0c00640 1 -- 172.21.15.57:0/3348426853 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c804f040 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.513+0000 7fb7c0c00640 1 -- 172.21.15.57:0/3348426853 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb7c8005390 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.513+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d009c460 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.513+0000 7fb7c0c00640 1 -- 172.21.15.57:0/3348426853 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7c8005590 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.513+0000 7fb7e077c740 10 client.16482 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.513+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e430a09540 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7c0c00640 1 -- 172.21.15.57:0/3348426853 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7d009c460 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7c0c00640 1 -- 172.21.15.57:0/3348426853 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c8056730 con 0x55e430a0dc30 2024-04-13T05:22:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7e077c740 10 client.16482 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7e077c740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7e077c740 20 client.16482 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7e077c740 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a09eb0 0x55e430a0c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7e077c740 4 client.16482 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.514+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e430973450 con 0x55e430a09eb0 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.515+0000 7fb7d6000640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a09eb0 0x55e430a0c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.515+0000 7fb7d6000640 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a09eb0 0x55e430a0c2d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df95c3dfc489b495 server_cookie=7aae4f9f292368b1 in_seq=0 out_seq=0 2024-04-13T05:22:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.515+0000 7fb7c0c00640 10 client.16482 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7c0c00640 1 -- 172.21.15.57:0/3348426853 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e430973450 con 0x55e430a09eb0 2024-04-13T05:22:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7c0c00640 10 client.16482 handle_command_reply: tid=0 2024-04-13T05:22:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 client.16482 shutdown 2024-04-13T05:22:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 msgr2=0x55e430a1b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a1b3c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8036040 tx=0x7fb7c802c660 comp rx=0 tx=0).stop 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 shutdown_connections 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e430a09eb0 0x55e430a0c2d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e430a13790 0x55e430a1e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e430a0e620 0x55e430a1b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 --2- 172.21.15.57:0/3348426853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e430a0dc30 0x55e430a1b3c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 >> 172.21.15.57:0/3348426853 conn(0x55e430a00990 msgr2=0x55e430a04480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.517+0000 7fb7c0c00640 0 client.16482 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.518+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 shutdown_connections 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.518+0000 7fb7e077c740 1 -- 172.21.15.57:0/3348426853 wait complete. 2024-04-13T05:22:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.518+0000 7fb7e077c740 20 client.16482 trim_cache size 0 max 16384 2024-04-13T05:22:30.522 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:22:30.522 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:22:30.522 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "5c06c111-66b3-4fcd-a438-bb4ae949d0a3", 2024-04-13T05:22:30.522 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:22:30.522 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb7bc0a3c30 msgr2=0x7fb7bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb7bc0a3c30 0x7fb7bc0a60f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb7c802c250 tx=0x7fb7c8005a20 comp rx=0 tx=0).stop 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8105cb0 msgr2=0x7fb7d81a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8105cb0 0x7fb7d81a1920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000e940 tx=0x7fb7d000ee00 comp rx=0 tx=0).stop 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 shutdown_connections 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb7bc0a3c30 0x7fb7bc0a60f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d8105cb0 0x7fb7d81a1920 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d81052f0 0x7fb7d819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 --2- 172.21.15.57:0/1659350453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7d8104100 0x7fb7d819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.519+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 >> 172.21.15.57:0/1659350453 conn(0x7fb7d80ff670 msgr2=0x7fb7d8108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.520+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 shutdown_connections 2024-04-13T05:22:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.520+0000 7fb7d7e00640 1 -- 172.21.15.57:0/1659350453 wait complete. 2024-04-13T05:22:30.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:22:30.546 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:22:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 -- 172.21.15.57:0/2052734736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c102950 msgr2=0x7f8f7c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 --2- 172.21.15.57:0/2052734736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c102950 0x7f8f7c104d40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009900 tx=0x7f8f7002bc30 comp rx=0 tx=0).stop 2024-04-13T05:22:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 -- 172.21.15.57:0/2052734736 shutdown_connections 2024-04-13T05:22:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 --2- 172.21.15.57:0/2052734736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c107bb0 0x7f8f7c10a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 --2- 172.21.15.57:0/2052734736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c105280 0x7f8f7c107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 --2- 172.21.15.57:0/2052734736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c102950 0x7f8f7c104d40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 -- 172.21.15.57:0/2052734736 >> 172.21.15.57:0/2052734736 conn(0x7f8f7c0fc360 msgr2=0x7f8f7c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.811+0000 7f8f82200640 1 -- 172.21.15.57:0/2052734736 shutdown_connections 2024-04-13T05:22:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.812+0000 7f8f82200640 1 -- 172.21.15.57:0/2052734736 wait complete. 2024-04-13T05:22:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.813+0000 7f8f82200640 1 Processor -- start 2024-04-13T05:22:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.813+0000 7f8f82200640 1 -- start start 2024-04-13T05:22:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f82200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c102950 0x7f8f7c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f82200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c105280 0x7f8f7c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f82200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c107bb0 0x7f8f7c1a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f82200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10da40 con 0x7f8f7c107bb0 2024-04-13T05:22:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f82200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10d8c0 con 0x7f8f7c102950 2024-04-13T05:22:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f82200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10dbc0 con 0x7f8f7c105280 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c105280 0x7f8f7c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f81800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c107bb0 0x7f8f7c1a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f80e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c102950 0x7f8f7c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f80e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c102950 0x7f8f7c195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46640/0 (socket says 172.21.15.57:46640) 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.814+0000 7f8f80e00640 1 -- 172.21.15.57:0/238469771 learned_addr learned my addr 172.21.15.57:0/238469771 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.815+0000 7f8f81800640 1 -- 172.21.15.57:0/238469771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c105280 msgr2=0x7f8f7c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.815+0000 7f8f81800640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c105280 0x7f8f7c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.815+0000 7f8f81800640 1 -- 172.21.15.57:0/238469771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c102950 msgr2=0x7f8f7c195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.815+0000 7f8f81800640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c102950 0x7f8f7c195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.815+0000 7f8f81800640 1 -- 172.21.15.57:0/238469771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1a6300 con 0x7f8f7c107bb0 2024-04-13T05:22:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.815+0000 7f8f81800640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c107bb0 0x7f8f7c1a5d70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009fd0 tx=0x7f8f7400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.816+0000 7f8f79600640 1 -- 172.21.15.57:0/238469771 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400ce20 con 0x7f8f7c107bb0 2024-04-13T05:22:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.816+0000 7f8f79600640 1 -- 172.21.15.57:0/238469771 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f74005020 con 0x7f8f7c107bb0 2024-04-13T05:22:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.816+0000 7f8f79600640 1 -- 172.21.15.57:0/238469771 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74010690 con 0x7f8f7c107bb0 2024-04-13T05:22:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.816+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1a65f0 con 0x7f8f7c107bb0 2024-04-13T05:22:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.817+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c1a6b40 con 0x7f8f7c107bb0 2024-04-13T05:22:30.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.822+0000 7f8f79600640 1 -- 172.21.15.57:0/238469771 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8f740051d0 con 0x7f8f7c107bb0 2024-04-13T05:22:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.823+0000 7f8f867c2740 1 Processor -- start 2024-04-13T05:22:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 -- start start 2024-04-13T05:22:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcf1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dcf46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55682dc53020 con 0x55682dced0e0 2024-04-13T05:22:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55682dbf96d0 con 0x55682dcec6f0 2024-04-13T05:22:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.824+0000 7f8f867c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55682dc4efe0 con 0x55682dcf2250 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f80e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcf1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dcf46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f79600640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8f640a3cb0 0x7f8f640a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46652/0 (socket says 172.21.15.57:46652) 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f7be00640 1 -- 172.21.15.57:0/585416462 learned_addr learned my addr 172.21.15.57:0/585416462 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.825+0000 7f8f79600640 1 -- 172.21.15.57:0/238469771 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8f740d0ef0 con 0x7f8f7c107bb0 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f7be00640 1 -- 172.21.15.57:0/585416462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 msgr2=0x55682dcf46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f7be00640 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dcf46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f7be00640 1 -- 172.21.15.57:0/585416462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 msgr2=0x55682dcf1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f7be00640 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcf1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f7be00640 1 -- 172.21.15.57:0/585416462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682da2f7d0 con 0x55682dcec6f0 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f80e00640 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcf1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f81800640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8f640a3cb0 0x7f8f640a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.826+0000 7f8f7be00640 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcecb10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f680149f0 tx=0x7f8f68004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c5a7ace7f169e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.827+0000 7f8f6e000640 1 -- 172.21.15.57:0/585416462 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68014270 con 0x55682dcec6f0 2024-04-13T05:22:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.827+0000 7f8f6e000640 1 -- 172.21.15.57:0/585416462 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f68018070 con 0x55682dcec6f0 2024-04-13T05:22:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.827+0000 7f8f6e000640 1 -- 172.21.15.57:0/585416462 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680035f0 con 0x55682dcec6f0 2024-04-13T05:22:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.827+0000 7f8f81800640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8f640a3cb0 0x7f8f640a6170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f740997a0 tx=0x7f8f74005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 -- 172.21.15.57:0/585416462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 msgr2=0x55682dcecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcecb10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f680149f0 tx=0x7f8f68004730 comp rx=0 tx=0).stop 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 -- 172.21.15.57:0/585416462 shutdown_connections 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dcf46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcf1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 --2- 172.21.15.57:0/585416462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcecb10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 -- 172.21.15.57:0/585416462 >> 172.21.15.57:0/585416462 conn(0x55682dce7c60 msgr2=0x55682dce8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 -- 172.21.15.57:0/585416462 shutdown_connections 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.828+0000 7f8f867c2740 1 -- 172.21.15.57:0/585416462 wait complete. 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.829+0000 7f8f867c2740 1 -- start start 2024-04-13T05:22:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.829+0000 7f8f867c2740 1 Processor -- start 2024-04-13T05:22:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.829+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd07a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 0x55682dd07fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dcf2250 0x55682dd0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f867c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55682dd06210 con 0x55682dcf2250 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f867c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55682dc0ca90 con 0x55682dced0e0 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f867c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55682dd06390 con 0x55682dcec6f0 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f867c2740 10 client.0 _ll_register_callbacks cb 0x55682dc0c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dcf2250 0x55682dd0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f80e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 0x55682dd07fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f80e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 0x55682dd07fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46668/0 (socket says 172.21.15.57:46668) 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f80e00640 1 -- 172.21.15.57:0/1882432171 learned_addr learned my addr 172.21.15.57:0/1882432171 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.830+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd07a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f80e00640 1 -- 172.21.15.57:0/1882432171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 msgr2=0x55682dd07a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f80e00640 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd07a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f80e00640 1 -- 172.21.15.57:0/1882432171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dcf2250 msgr2=0x55682dd0efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f80e00640 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dcf2250 0x55682dd0efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f80e00640 1 -- 172.21.15.57:0/1882432171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682d9653f0 con 0x55682dced0e0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f80e00640 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 0x55682dd07fa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f7002c240 tx=0x7f8f70004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eeedb66ab5288c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.831+0000 7f8f62a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.832+0000 7f8f62a00640 1 -- 172.21.15.57:0/1882432171 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70046070 con 0x55682dced0e0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.832+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f68018070 con 0x55682dced0e0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.832+0000 7f8f62a00640 1 -- 172.21.15.57:0/1882432171 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8f70005250 con 0x55682dced0e0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.833+0000 7f8f62a00640 1 -- 172.21.15.57:0/1882432171 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f700044a0 con 0x55682dced0e0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.833+0000 7f8f867c2740 10 client.35909 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.833+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55682d9250f0 con 0x55682dced0e0 2024-04-13T05:22:30.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.834+0000 7f8f62a00640 1 -- 172.21.15.57:0/1882432171 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f68018070 con 0x55682dced0e0 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.834+0000 7f8f62a00640 1 -- 172.21.15.57:0/1882432171 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8f70003680 con 0x55682dced0e0 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f867c2740 10 client.35909 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f867c2740 10 client.35909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f867c2740 20 client.35909 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd138a0 0x55682dd15ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f867c2740 4 client.35909 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:22:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55682dbf2600 con 0x55682dd138a0 2024-04-13T05:22:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.835+0000 7f8f7be00640 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd138a0 0x55682dd15ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.837+0000 7f8f7be00640 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd138a0 0x55682dd15ce0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b674649b4d236fcb server_cookie=2ecf7f2db60fbef7 in_seq=0 out_seq=0 2024-04-13T05:22:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.837+0000 7f8f62a00640 10 client.35909 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.839+0000 7f8f62a00640 1 -- 172.21.15.57:0/1882432171 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55682dbf2600 con 0x55682dd138a0 2024-04-13T05:22:30.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.839+0000 7f8f62a00640 10 client.35909 handle_command_reply: tid=0 2024-04-13T05:22:30.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 client.35909 shutdown 2024-04-13T05:22:30.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 msgr2=0x55682dd07fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 0x55682dd07fa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f7002c240 tx=0x7f8f70004270 comp rx=0 tx=0).stop 2024-04-13T05:22:30.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 shutdown_connections 2024-04-13T05:22:30.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd138a0 0x55682dd15ce0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dced0e0 0x55682dd07fa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dcf2250 0x55682dd0efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1882432171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd07a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.840+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 >> 172.21.15.57:0/1882432171 conn(0x55682dce7c60 msgr2=0x55682dce6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.841+0000 7f8f62a00640 0 client.35909 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.841+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 shutdown_connections 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.841+0000 7f8f867c2740 1 -- 172.21.15.57:0/1882432171 wait complete. 2024-04-13T05:22:30.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.841+0000 7f8f867c2740 20 client.35909 trim_cache size 0 max 16384 2024-04-13T05:22:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 Processor -- start 2024-04-13T05:22:30.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 -- start start 2024-04-13T05:22:30.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dd0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1aaff0 con 0x55682dced0e0 2024-04-13T05:22:30.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1a8860 con 0x55682dcec6f0 2024-04-13T05:22:30.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.882+0000 7f8f867c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c073a40 con 0x55682dcf2250 2024-04-13T05:22:30.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:46672/0 (socket says 172.21.15.57:46672) 2024-04-13T05:22:30.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 -- 172.21.15.57:0/1599168909 learned_addr learned my addr 172.21.15.57:0/1599168909 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f80e00640 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dd0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 -- 172.21.15.57:0/1599168909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 msgr2=0x55682dd0efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dd0efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 -- 172.21.15.57:0/1599168909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 msgr2=0x55682dcee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.884+0000 7f8f7be00640 1 -- 172.21.15.57:0/1599168909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682dd12290 con 0x55682dcec6f0 2024-04-13T05:22:30.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.885+0000 7f8f7be00640 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcee4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005780 tx=0x7f8f68016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4eaaea8435d536 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.886+0000 7f8f6d600640 1 -- 172.21.15.57:0/1599168909 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68011e20 con 0x55682dcec6f0 2024-04-13T05:22:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.886+0000 7f8f6d600640 1 -- 172.21.15.57:0/1599168909 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f68018070 con 0x55682dcec6f0 2024-04-13T05:22:30.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.886+0000 7f8f6d600640 1 -- 172.21.15.57:0/1599168909 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680140d0 con 0x55682dcec6f0 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 -- 172.21.15.57:0/1599168909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 msgr2=0x55682dcee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcee4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005780 tx=0x7f8f68016040 comp rx=0 tx=0).stop 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 -- 172.21.15.57:0/1599168909 shutdown_connections 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcf2250 0x55682dd0efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dcee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 --2- 172.21.15.57:0/1599168909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcec6f0 0x55682dcee4a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.888+0000 7f8f867c2740 1 -- 172.21.15.57:0/1599168909 >> 172.21.15.57:0/1599168909 conn(0x55682dce7c60 msgr2=0x55682dd07080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.889+0000 7f8f867c2740 1 -- 172.21.15.57:0/1599168909 shutdown_connections 2024-04-13T05:22:30.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.890+0000 7f8f867c2740 1 -- 172.21.15.57:0/1599168909 wait complete. 2024-04-13T05:22:30.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.890+0000 7f8f867c2740 1 -- start start 2024-04-13T05:22:30.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.890+0000 7f8f867c2740 1 Processor -- start 2024-04-13T05:22:30.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.892+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd0c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dd0c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f867c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcf2250 0x55682dd77e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f867c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1a6e60 con 0x55682dced0e0 2024-04-13T05:22:30.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f867c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1a8190 con 0x55682dcf2250 2024-04-13T05:22:30.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f867c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1a73f0 con 0x55682dcec6f0 2024-04-13T05:22:30.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f867c2740 10 client.0 _ll_register_callbacks cb 0x7f8f7002cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:22:30.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd0c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd0c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:53044/0 (socket says 172.21.15.57:53044) 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f7be00640 1 -- 172.21.15.57:0/420032431 learned_addr learned my addr 172.21.15.57:0/420032431 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.893+0000 7f8f80e00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dd0c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.894+0000 7f8f7be00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcf2250 0x55682dd77e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.894+0000 7f8f7be00640 1 -- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcf2250 msgr2=0x55682dd77e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.894+0000 7f8f7be00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcf2250 0x55682dd77e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.894+0000 7f8f7be00640 1 -- 172.21.15.57:0/420032431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 msgr2=0x55682dd0c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:22:30.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.894+0000 7f8f7be00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dd0c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.894+0000 7f8f7be00640 1 -- 172.21.15.57:0/420032431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682dd78350 con 0x55682dcec6f0 2024-04-13T05:22:30.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.895+0000 7f8f7be00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd0c020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f68011980 tx=0x7f8f68005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9741e594f18a02c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:22:30.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.895+0000 7f8f62a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:22:30.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.896+0000 7f8f62a00640 1 -- 172.21.15.57:0/420032431 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68011b00 con 0x55682dcec6f0 2024-04-13T05:22:30.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.896+0000 7f8f62a00640 1 -- 172.21.15.57:0/420032431 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8f680140d0 con 0x55682dcec6f0 2024-04-13T05:22:30.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.896+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f68018070 con 0x55682dcec6f0 2024-04-13T05:22:30.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.897+0000 7f8f62a00640 1 -- 172.21.15.57:0/420032431 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f6802fae0 con 0x55682dcec6f0 2024-04-13T05:22:30.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.897+0000 7f8f867c2740 10 client.25837 fetch_fsmap learned FSMap version 29 2024-04-13T05:22:30.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.897+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55682dd79610 con 0x55682dcec6f0 2024-04-13T05:22:30.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.898+0000 7f8f62a00640 1 -- 172.21.15.57:0/420032431 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f68018070 con 0x55682dcec6f0 2024-04-13T05:22:30.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.898+0000 7f8f62a00640 1 -- 172.21.15.57:0/420032431 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8f680053a0 con 0x55682dcec6f0 2024-04-13T05:22:30.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.899+0000 7f8f867c2740 10 client.25837 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:22:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.899+0000 7f8f867c2740 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:22:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.899+0000 7f8f867c2740 20 client.25837 populate_metadata read hostname 'smithi057' 2024-04-13T05:22:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.899+0000 7f8f867c2740 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd79c90 0x55682dd7c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:22:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.899+0000 7f8f867c2740 4 client.25837 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:22:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.899+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55682dc60a40 con 0x55682dd79c90 2024-04-13T05:22:30.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.900+0000 7f8f80e00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd79c90 0x55682dd7c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:22:30.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.901+0000 7f8f80e00640 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd79c90 0x55682dd7c0b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf38b6208bc8202 server_cookie=54f257a910ea2f95 in_seq=0 out_seq=0 2024-04-13T05:22:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.902+0000 7f8f62a00640 10 client.25837 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.902+0000 7f8f62a00640 1 -- 172.21.15.57:0/420032431 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55682dc60a40 con 0x55682dd79c90 2024-04-13T05:22:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.902+0000 7f8f62a00640 10 client.25837 handle_command_reply: tid=0 2024-04-13T05:22:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.902+0000 7f8f867c2740 1 client.25837 shutdown 2024-04-13T05:22:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.902+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 msgr2=0x55682dd0c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.902+0000 7f8f867c2740 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd0c020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f68011980 tx=0x7f8f68005bb0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 shutdown_connections 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55682dd79c90 0x55682dd7c0b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55682dcf2250 0x55682dd77e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55682dced0e0 0x55682dd0c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 --2- 172.21.15.57:0/420032431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55682dcec6f0 0x55682dd0c020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 >> 172.21.15.57:0/420032431 conn(0x55682dce7c60 msgr2=0x55682dceaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f62a00640 0 client.25837 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:22:30.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.903+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 shutdown_connections 2024-04-13T05:22:30.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.904+0000 7f8f867c2740 1 -- 172.21.15.57:0/420032431 wait complete. 2024-04-13T05:22:30.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.904+0000 7f8f867c2740 20 client.25837 trim_cache size 0 max 16384 2024-04-13T05:22:30.907 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:22:30.907 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (789 inodes in the stack)", 2024-04-13T05:22:30.907 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout: "5c06c111-66b3-4fcd-a438-bb4ae949d0a3": { 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "5c06c111-66b3-4fcd-a438-bb4ae949d0a3", 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:22:30.908 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8f640a3cb0 msgr2=0x7f8f640a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8f640a3cb0 0x7f8f640a6170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f740997a0 tx=0x7f8f74005b70 comp rx=0 tx=0).stop 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c107bb0 msgr2=0x7f8f7c1a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c107bb0 0x7f8f7c1a5d70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009fd0 tx=0x7f8f7400ef10 comp rx=0 tx=0).stop 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 shutdown_connections 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8f640a3cb0 0x7f8f640a6170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f7c107bb0 0x7f8f7c1a5d70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f7c105280 0x7f8f7c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 --2- 172.21.15.57:0/238469771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f7c102950 0x7f8f7c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:22:30.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.905+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 >> 172.21.15.57:0/238469771 conn(0x7f8f7c0fc360 msgr2=0x7f8f7c102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:22:30.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.906+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 shutdown_connections 2024-04-13T05:22:30.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:22:30.906+0000 7f8f82200640 1 -- 172.21.15.57:0/238469771 wait complete. 2024-04-13T05:22:30.933 INFO:tasks.cephfs.filesystem:scrub status for tag:5c06c111-66b3-4fcd-a438-bb4ae949d0a3 - {'path': '/', 'tag': '5c06c111-66b3-4fcd-a438-bb4ae949d0a3', 'options': 'recursive,force'} 2024-04-13T05:22:31.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:31 smithi189 ceph-mon[35531]: pgmap v603: 129 pgs: 129 active+clean; 241 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-04-13T05:22:31.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:31 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:22:31.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:31 smithi083 ceph-mon[35133]: pgmap v603: 129 pgs: 129 active+clean; 241 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-04-13T05:22:31.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:31 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:22:31.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:31 smithi057 ceph-mon[32331]: pgmap v603: 129 pgs: 129 active+clean; 241 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-04-13T05:22:31.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:31 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:22:32.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:32 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:22:32.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:32 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:32 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:22:32.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:32 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:22:32.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:32 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:32 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:32 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:22:32.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:32 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:32 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:22:32.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:32 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:22:32.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:32 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:32 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:32 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:22:32.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:32 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:32 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:22:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:32 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:22:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:32 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:32 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:33.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:33 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:33.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:33 smithi189 ceph-mon[35531]: pgmap v604: 129 pgs: 129 active+clean; 241 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-13T05:22:33.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:33 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:33.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:33 smithi083 ceph-mon[35133]: pgmap v604: 129 pgs: 129 active+clean; 241 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-13T05:22:33.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:33 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:22:33.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:33 smithi057 ceph-mon[32331]: pgmap v604: 129 pgs: 129 active+clean; 241 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-13T05:22:35.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:35 smithi189 ceph-mon[35531]: pgmap v605: 129 pgs: 129 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-04-13T05:22:35.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:35 smithi083 ceph-mon[35133]: pgmap v605: 129 pgs: 129 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-04-13T05:22:35.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:35 smithi057 ceph-mon[32331]: pgmap v605: 129 pgs: 129 active+clean; 246 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-04-13T05:22:37.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:37 smithi189 ceph-mon[35531]: pgmap v606: 129 pgs: 129 active+clean; 251 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-13T05:22:37.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:37 smithi083 ceph-mon[35133]: pgmap v606: 129 pgs: 129 active+clean; 251 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-13T05:22:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:37 smithi057 ceph-mon[32331]: pgmap v606: 129 pgs: 129 active+clean; 251 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-13T05:22:39.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:39 smithi189 ceph-mon[35531]: pgmap v607: 129 pgs: 129 active+clean; 251 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-13T05:22:39.637 INFO:tasks.workunit.client.0.smithi057.stdout: 42 8485 44 5135 92 3109 120 2024-04-13T05:22:39.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:39 smithi083 ceph-mon[35133]: pgmap v607: 129 pgs: 129 active+clean; 251 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-13T05:22:39.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:39 smithi057 ceph-mon[32331]: pgmap v607: 129 pgs: 129 active+clean; 251 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-04-13T05:22:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:41 smithi189 ceph-mon[35531]: pgmap v608: 129 pgs: 129 active+clean; 254 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-04-13T05:22:41.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:41 smithi083 ceph-mon[35133]: pgmap v608: 129 pgs: 129 active+clean; 254 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-04-13T05:22:41.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:41 smithi057 ceph-mon[32331]: pgmap v608: 129 pgs: 129 active+clean; 254 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-04-13T05:22:43.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:43 smithi189 ceph-mon[35531]: pgmap v609: 129 pgs: 129 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-04-13T05:22:43.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:43 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:22:43.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:43 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:22:43.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:43 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:22:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:43 smithi083 ceph-mon[35133]: pgmap v609: 129 pgs: 129 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-04-13T05:22:43.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:43 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:22:43.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:43 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:22:43.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:43 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:22:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:43 smithi057 ceph-mon[32331]: pgmap v609: 129 pgs: 129 active+clean; 259 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-04-13T05:22:43.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:43 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:22:43.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:43 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:22:43.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:43 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:22:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:45 smithi083 ceph-mon[35133]: pgmap v610: 129 pgs: 129 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-04-13T05:22:45.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:45 smithi057 ceph-mon[32331]: pgmap v610: 129 pgs: 129 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-04-13T05:22:45.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:45 smithi189 ceph-mon[35531]: pgmap v610: 129 pgs: 129 active+clean; 263 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-04-13T05:22:47.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:47 smithi083 ceph-mon[35133]: pgmap v611: 129 pgs: 129 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-04-13T05:22:47.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:47 smithi057 ceph-mon[32331]: pgmap v611: 129 pgs: 129 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-04-13T05:22:47.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:47 smithi189 ceph-mon[35531]: pgmap v611: 129 pgs: 129 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-04-13T05:22:49.637 INFO:tasks.workunit.client.0.smithi057.stdout: 44 7944 70 5341 75 1912 84 2024-04-13T05:22:49.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:49 smithi083 ceph-mon[35133]: pgmap v612: 129 pgs: 129 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-04-13T05:22:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:49 smithi057 ceph-mon[32331]: pgmap v612: 129 pgs: 129 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-04-13T05:22:49.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:49 smithi189 ceph-mon[35531]: pgmap v612: 129 pgs: 129 active+clean; 267 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-04-13T05:22:51.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:51 smithi083 ceph-mon[35133]: pgmap v613: 129 pgs: 129 active+clean; 272 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-04-13T05:22:51.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:51 smithi057 ceph-mon[32331]: pgmap v613: 129 pgs: 129 active+clean; 272 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-04-13T05:22:51.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:51 smithi189 ceph-mon[35531]: pgmap v613: 129 pgs: 129 active+clean; 272 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-04-13T05:22:53.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:53 smithi083 ceph-mon[35133]: pgmap v614: 129 pgs: 129 active+clean; 274 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:22:53.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:53 smithi057 ceph-mon[32331]: pgmap v614: 129 pgs: 129 active+clean; 274 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:22:53.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:53 smithi189 ceph-mon[35531]: pgmap v614: 129 pgs: 129 active+clean; 274 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:22:55.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:55 smithi083 ceph-mon[35133]: pgmap v615: 129 pgs: 129 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-04-13T05:22:55.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:55 smithi057 ceph-mon[32331]: pgmap v615: 129 pgs: 129 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-04-13T05:22:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:55 smithi189 ceph-mon[35531]: pgmap v615: 129 pgs: 129 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-04-13T05:22:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:57 smithi083 ceph-mon[35133]: pgmap v616: 129 pgs: 129 active+clean; 282 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-04-13T05:22:57.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:57 smithi057 ceph-mon[32331]: pgmap v616: 129 pgs: 129 active+clean; 282 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-04-13T05:22:57.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:57 smithi189 ceph-mon[35531]: pgmap v616: 129 pgs: 129 active+clean; 282 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-04-13T05:22:59.638 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:59 smithi189 ceph-mon[35531]: pgmap v617: 129 pgs: 129 active+clean; 282 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-04-13T05:22:59.639 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:22:59 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:22:59.639 INFO:tasks.workunit.client.0.smithi057.stdout: 45 8057 41 5611 60 3247 114 2024-04-13T05:22:59.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:59 smithi083 ceph-mon[35133]: pgmap v617: 129 pgs: 129 active+clean; 282 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-04-13T05:22:59.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:22:59 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:22:59.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:59 smithi057 ceph-mon[32331]: pgmap v617: 129 pgs: 129 active+clean; 282 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-04-13T05:22:59.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:22:59 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:23:00.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:23:00.934 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:23:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 -- 172.21.15.57:0/1480258202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88105430 msgr2=0x7fcd881058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 --2- 172.21.15.57:0/1480258202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88105430 0x7fcd881058b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009900 tx=0x7fcd7402bc30 comp rx=0 tx=0).stop 2024-04-13T05:23:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 -- 172.21.15.57:0/1480258202 shutdown_connections 2024-04-13T05:23:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 --2- 172.21.15.57:0/1480258202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105df0 0x7fcd8810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 --2- 172.21.15.57:0/1480258202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88105430 0x7fcd881058b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 --2- 172.21.15.57:0/1480258202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd88104240 0x7fcd88104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.182+0000 7fcd8f800640 1 -- 172.21.15.57:0/1480258202 >> 172.21.15.57:0/1480258202 conn(0x7fcd880ff7b0 msgr2=0x7fcd88101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.183+0000 7fcd8f800640 1 -- 172.21.15.57:0/1480258202 shutdown_connections 2024-04-13T05:23:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.183+0000 7fcd8f800640 1 -- 172.21.15.57:0/1480258202 wait complete. 2024-04-13T05:23:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 Processor -- start 2024-04-13T05:23:01.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 -- start start 2024-04-13T05:23:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88104240 0x7fcd8819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105430 0x7fcd8819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd88105df0 0x7fcd881a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8810fdd0 con 0x7fcd88104240 2024-04-13T05:23:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8810fc50 con 0x7fcd88105430 2024-04-13T05:23:01.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.184+0000 7fcd8f800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8810ff50 con 0x7fcd88105df0 2024-04-13T05:23:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.185+0000 7fcd87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105430 0x7fcd8819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.185+0000 7fcd87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105430 0x7fcd8819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:39934/0 (socket says 172.21.15.57:39934) 2024-04-13T05:23:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.185+0000 7fcd87e00640 1 -- 172.21.15.57:0/2309782018 learned_addr learned my addr 172.21.15.57:0/2309782018 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.185+0000 7fcd8ce00640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88104240 0x7fcd8819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.185+0000 7fcd8d800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd88105df0 0x7fcd881a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.186+0000 7fcd8ce00640 1 -- 172.21.15.57:0/2309782018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd88105df0 msgr2=0x7fcd881a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.186+0000 7fcd8ce00640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd88105df0 0x7fcd881a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.186+0000 7fcd8ce00640 1 -- 172.21.15.57:0/2309782018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105430 msgr2=0x7fcd8819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.186+0000 7fcd8ce00640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105430 0x7fcd8819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.186+0000 7fcd8ce00640 1 -- 172.21.15.57:0/2309782018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd881a2080 con 0x7fcd88104240 2024-04-13T05:23:01.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.186+0000 7fcd8ce00640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88104240 0x7fcd8819a460 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00e9c0 tx=0x7fcd7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.187+0000 7fcd85600640 1 -- 172.21.15.57:0/2309782018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00ccc0 con 0x7fcd88104240 2024-04-13T05:23:01.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.187+0000 7fcd85600640 1 -- 172.21.15.57:0/2309782018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd7c0035f0 con 0x7fcd88104240 2024-04-13T05:23:01.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.187+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd88076460 con 0x7fcd88104240 2024-04-13T05:23:01.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.187+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd880769b0 con 0x7fcd88104240 2024-04-13T05:23:01.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.187+0000 7fcd85600640 1 -- 172.21.15.57:0/2309782018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00ce60 con 0x7fcd88104240 2024-04-13T05:23:01.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 Processor -- start 2024-04-13T05:23:01.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 -- start start 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2d3030 0x5607ac2d54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5607ac234020 con 0x5607ac2cd4d0 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5607ac1da6d0 con 0x5607ac2cdec0 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.191+0000 7fcd92793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607ac22ffe0 con 0x5607ac2d3030 2024-04-13T05:23:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd8d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd8d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42434/0 (socket says 172.21.15.57:42434) 2024-04-13T05:23:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd8d800640 1 -- 172.21.15.57:0/1320235351 learned_addr learned my addr 172.21.15.57:0/1320235351 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd85600640 1 -- 172.21.15.57:0/2309782018 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcd7c010650 con 0x7fcd88104240 2024-04-13T05:23:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd87e00640 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2d3030 0x5607ac2d54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd8d800640 1 -- 172.21.15.57:0/1320235351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2d3030 msgr2=0x5607ac2d54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd8d800640 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2d3030 0x5607ac2d54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.192+0000 7fcd8d800640 1 -- 172.21.15.57:0/1320235351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 msgr2=0x5607ac2d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.193+0000 7fcd8d800640 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.193+0000 7fcd8d800640 1 -- 172.21.15.57:0/1320235351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ac0107d0 con 0x5607ac2cd4d0 2024-04-13T05:23:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.193+0000 7fcd8d800640 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2cd8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009fc0 tx=0x7fcd80010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f561a007e6e9c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.193+0000 7fcd7a000640 1 -- 172.21.15.57:0/1320235351 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80003b70 con 0x5607ac2cd4d0 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.194+0000 7fcd7a000640 1 -- 172.21.15.57:0/1320235351 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd80003d10 con 0x5607ac2cd4d0 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.194+0000 7fcd7a000640 1 -- 172.21.15.57:0/1320235351 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80011a00 con 0x5607ac2cd4d0 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.194+0000 7fcd85600640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd700a3c30 0x7fcd700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.194+0000 7fcd92793740 1 -- 172.21.15.57:0/1320235351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 msgr2=0x5607ac2cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.194+0000 7fcd92793740 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2cd8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009fc0 tx=0x7fcd80010860 comp rx=0 tx=0).stop 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.194+0000 7fcd85600640 1 -- 172.21.15.57:0/2309782018 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcd7c0d0ac0 con 0x7fcd88104240 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 -- 172.21.15.57:0/1320235351 shutdown_connections 2024-04-13T05:23:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2d3030 0x5607ac2d54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 --2- 172.21.15.57:0/1320235351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2cd8f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 -- 172.21.15.57:0/1320235351 >> 172.21.15.57:0/1320235351 conn(0x5607ac2c8a40 msgr2=0x5607ac2c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd8ce00640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd700a3c30 0x7fcd700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 -- 172.21.15.57:0/1320235351 shutdown_connections 2024-04-13T05:23:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.195+0000 7fcd92793740 1 -- 172.21.15.57:0/1320235351 wait complete. 2024-04-13T05:23:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.196+0000 7fcd92793740 1 -- start start 2024-04-13T05:23:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.196+0000 7fcd8ce00640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd700a3c30 0x7fcd700a60f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0a5140 tx=0x7fcd7c085f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.196+0000 7fcd92793740 1 Processor -- start 2024-04-13T05:23:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.196+0000 7fcd92793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd92793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cdec0 0x5607ac2d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd92793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2d3030 0x5607ac2ded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd92793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5607ac2d5fe0 con 0x5607ac2cd4d0 2024-04-13T05:23:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd92793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5607ac1eda90 con 0x5607ac2d3030 2024-04-13T05:23:01.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd92793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607ac2d6160 con 0x5607ac2cdec0 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd8d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd92793740 10 client.0 _ll_register_callbacks cb 0x5607ac1ed480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd8d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42450/0 (socket says 172.21.15.57:42450) 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd8d800640 1 -- 172.21.15.57:0/3170524529 learned_addr learned my addr 172.21.15.57:0/3170524529 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd8d800640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2d3030 0x5607ac2ded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.197+0000 7fcd87e00640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cdec0 0x5607ac2d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.198+0000 7fcd8d800640 1 -- 172.21.15.57:0/3170524529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cdec0 msgr2=0x5607ac2d7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.198+0000 7fcd8d800640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cdec0 0x5607ac2d7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.198+0000 7fcd8d800640 1 -- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2d3030 msgr2=0x5607ac2ded50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.198+0000 7fcd8d800640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2d3030 0x5607ac2ded50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.198+0000 7fcd8d800640 1 -- 172.21.15.57:0/3170524529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607abf463f0 con 0x5607ac2cd4d0 2024-04-13T05:23:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.199+0000 7fcd8d800640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d77d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcd80004980 tx=0x7fcd800048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4df209876303c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.199+0000 7fcd6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.199+0000 7fcd6ea00640 1 -- 172.21.15.57:0/3170524529 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80011d50 con 0x5607ac2cd4d0 2024-04-13T05:23:01.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.199+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd80003d10 con 0x5607ac2cd4d0 2024-04-13T05:23:01.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.199+0000 7fcd6ea00640 1 -- 172.21.15.57:0/3170524529 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcd80020030 con 0x5607ac2cd4d0 2024-04-13T05:23:01.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.200+0000 7fcd6ea00640 1 -- 172.21.15.57:0/3170524529 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd80011d50 con 0x5607ac2cd4d0 2024-04-13T05:23:01.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.200+0000 7fcd92793740 10 client.16512 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:01.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.200+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607abf060f0 con 0x5607ac2cd4d0 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.200+0000 7fcd6ea00640 1 -- 172.21.15.57:0/3170524529 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd80003d10 con 0x5607ac2cd4d0 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.200+0000 7fcd6ea00640 1 -- 172.21.15.57:0/3170524529 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcd80029040 con 0x5607ac2cd4d0 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.201+0000 7fcd92793740 10 client.16512 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.201+0000 7fcd92793740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.201+0000 7fcd92793740 20 client.16512 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.201+0000 7fcd92793740 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac2e32f0 0x5607ac2e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.201+0000 7fcd92793740 4 client.16512 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.201+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607ac1d3600 con 0x5607ac2e32f0 2024-04-13T05:23:01.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.202+0000 7fcd87e00640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac2e32f0 0x5607ac2e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.203+0000 7fcd87e00640 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac2e32f0 0x5607ac2e56e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14446b2ade9e51df server_cookie=9fbfdaddda774d94 in_seq=0 out_seq=0 2024-04-13T05:23:01.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.203+0000 7fcd6ea00640 10 client.16512 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:01.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.205+0000 7fcd6ea00640 1 -- 172.21.15.57:0/3170524529 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607ac1d3600 con 0x5607ac2e32f0 2024-04-13T05:23:01.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.205+0000 7fcd6ea00640 10 client.16512 handle_command_reply: tid=0 2024-04-13T05:23:01.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.205+0000 7fcd92793740 1 client.16512 shutdown 2024-04-13T05:23:01.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 msgr2=0x5607ac2d77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d77d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcd80004980 tx=0x7fcd800048d0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 shutdown_connections 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac2e32f0 0x5607ac2e56e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2d3030 0x5607ac2ded50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cdec0 0x5607ac2d7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 --2- 172.21.15.57:0/3170524529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d77d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 >> 172.21.15.57:0/3170524529 conn(0x5607ac2c8a40 msgr2=0x5607ac2c7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.206+0000 7fcd6ea00640 0 client.16512 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.207+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 shutdown_connections 2024-04-13T05:23:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.207+0000 7fcd92793740 1 -- 172.21.15.57:0/3170524529 wait complete. 2024-04-13T05:23:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.207+0000 7fcd92793740 20 client.16512 trim_cache size 0 max 16384 2024-04-13T05:23:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.238+0000 7fcd92793740 1 Processor -- start 2024-04-13T05:23:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 -- start start 2024-04-13T05:23:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcd680125a0 con 0x5607ac2d3030 2024-04-13T05:23:01.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6800fe10 con 0x5607ac2cdec0 2024-04-13T05:23:01.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd92793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6800dc60 con 0x5607ac2cd4d0 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.239+0000 7fcd87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2ded10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:39964/0 (socket says 172.21.15.57:39964) 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 -- 172.21.15.57:0/1468506150 learned_addr learned my addr 172.21.15.57:0/1468506150 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd8d800640 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd8d800640 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 -- 172.21.15.57:0/1468506150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 msgr2=0x5607ac2d7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 -- 172.21.15.57:0/1468506150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 msgr2=0x5607ac2c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd87e00640 1 -- 172.21.15.57:0/1468506150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ac2c9490 con 0x5607ac2cdec0 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.240+0000 7fcd8d800640 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.241+0000 7fcd87e00640 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2ded10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcd740096c0 tx=0x7fcd7402c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622bd243cd44fef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.241+0000 7fcd79600640 1 -- 172.21.15.57:0/1468506150 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74048070 con 0x5607ac2cdec0 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.242+0000 7fcd79600640 1 -- 172.21.15.57:0/1468506150 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd7402cb80 con 0x5607ac2cdec0 2024-04-13T05:23:01.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.242+0000 7fcd79600640 1 -- 172.21.15.57:0/1468506150 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7404a570 con 0x5607ac2cdec0 2024-04-13T05:23:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.242+0000 7fcd92793740 1 -- 172.21.15.57:0/1468506150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 msgr2=0x5607ac2ded10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.242+0000 7fcd92793740 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2ded10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcd740096c0 tx=0x7fcd7402c7e0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 -- 172.21.15.57:0/1468506150 shutdown_connections 2024-04-13T05:23:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2ded10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 --2- 172.21.15.57:0/1468506150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 -- 172.21.15.57:0/1468506150 >> 172.21.15.57:0/1468506150 conn(0x5607ac2c8a40 msgr2=0x5607ac2d73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 -- 172.21.15.57:0/1468506150 shutdown_connections 2024-04-13T05:23:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 -- 172.21.15.57:0/1468506150 wait complete. 2024-04-13T05:23:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.243+0000 7fcd92793740 1 -- start start 2024-04-13T05:23:01.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 Processor -- start 2024-04-13T05:23:01.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6800e2d0 con 0x5607ac2d3030 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6800f740 con 0x5607ac2cdec0 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.244+0000 7fcd92793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6800e9a0 con 0x5607ac2cd4d0 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.245+0000 7fcd92793740 10 client.0 _ll_register_callbacks cb 0x7fcd800117d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.245+0000 7fcd87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.245+0000 7fcd87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2da470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:39976/0 (socket says 172.21.15.57:39976) 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.245+0000 7fcd87e00640 1 -- 172.21.15.57:0/80881298 learned_addr learned my addr 172.21.15.57:0/80881298 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:01.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.245+0000 7fcd8d800640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd8d800640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd87e00640 1 -- 172.21.15.57:0/80881298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 msgr2=0x5607ac2d9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd87e00640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd87e00640 1 -- 172.21.15.57:0/80881298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 msgr2=0x5607ac2dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd87e00640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2dd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd87e00640 1 -- 172.21.15.57:0/80881298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ac2dda80 con 0x5607ac2cdec0 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd8d800640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.246+0000 7fcd87e00640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2da470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402c6a0 tx=0x7fcd7404ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12232c4f3e0e2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.247+0000 7fcd6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.247+0000 7fcd6ea00640 1 -- 172.21.15.57:0/80881298 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7404a310 con 0x5607ac2cdec0 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.247+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd7402cb80 con 0x5607ac2cdec0 2024-04-13T05:23:01.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.247+0000 7fcd6ea00640 1 -- 172.21.15.57:0/80881298 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcd7404a4b0 con 0x5607ac2cdec0 2024-04-13T05:23:01.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.248+0000 7fcd6ea00640 1 -- 172.21.15.57:0/80881298 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd7404a6b0 con 0x5607ac2cdec0 2024-04-13T05:23:01.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.248+0000 7fcd92793740 10 client.35930 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:01.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.248+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607ac2dedf0 con 0x5607ac2cdec0 2024-04-13T05:23:01.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.248+0000 7fcd6ea00640 1 -- 172.21.15.57:0/80881298 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd7402cb80 con 0x5607ac2cdec0 2024-04-13T05:23:01.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd6ea00640 1 -- 172.21.15.57:0/80881298 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcd74034440 con 0x5607ac2cdec0 2024-04-13T05:23:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd92793740 10 client.35930 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd92793740 10 client.35930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd92793740 20 client.35930 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd92793740 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac36bd50 0x5607ac36e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd92793740 4 client.35930 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:23:01.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.249+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607ac241a40 con 0x5607ac36bd50 2024-04-13T05:23:01.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.251+0000 7fcd8d800640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac36bd50 0x5607ac36e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.252+0000 7fcd8d800640 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac36bd50 0x5607ac36e170 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707075578c8f6d5f server_cookie=1097ab5268831005 in_seq=0 out_seq=0 2024-04-13T05:23:01.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.252+0000 7fcd6ea00640 10 client.35930 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:01.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.252+0000 7fcd6ea00640 1 -- 172.21.15.57:0/80881298 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607ac241a40 con 0x5607ac36bd50 2024-04-13T05:23:01.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.252+0000 7fcd6ea00640 10 client.35930 handle_command_reply: tid=0 2024-04-13T05:23:01.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.252+0000 7fcd92793740 1 client.35930 shutdown 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 msgr2=0x5607ac2da470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2da470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402c6a0 tx=0x7fcd7404ac50 comp rx=0 tx=0).stop 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 shutdown_connections 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5607ac36bd50 0x5607ac36e170 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd6ea00640 0 client.35930 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5607ac2cdec0 0x5607ac2da470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5607ac2d3030 0x5607ac2dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 --2- 172.21.15.57:0/80881298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607ac2cd4d0 0x5607ac2d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.253+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 >> 172.21.15.57:0/80881298 conn(0x5607ac2c8a40 msgr2=0x5607ac2cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.254+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 shutdown_connections 2024-04-13T05:23:01.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.254+0000 7fcd92793740 1 -- 172.21.15.57:0/80881298 wait complete. 2024-04-13T05:23:01.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.254+0000 7fcd92793740 20 client.35930 trim_cache size 0 max 16384 2024-04-13T05:23:01.257 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:23:01.257 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:23:01.257 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:23:01.258 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:23:01.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd700a3c30 msgr2=0x7fcd700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd700a3c30 0x7fcd700a60f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0a5140 tx=0x7fcd7c085f90 comp rx=0 tx=0).stop 2024-04-13T05:23:01.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88104240 msgr2=0x7fcd8819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88104240 0x7fcd8819a460 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00e9c0 tx=0x7fcd7c00ee80 comp rx=0 tx=0).stop 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 shutdown_connections 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcd700a3c30 0x7fcd700a60f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88105430 0x7fcd8819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd88105df0 0x7fcd881a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 --2- 172.21.15.57:0/2309782018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd88104240 0x7fcd8819a460 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 >> 172.21.15.57:0/2309782018 conn(0x7fcd880ff7b0 msgr2=0x7fcd881087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 shutdown_connections 2024-04-13T05:23:01.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.255+0000 7fcd8f800640 1 -- 172.21.15.57:0/2309782018 wait complete. 2024-04-13T05:23:01.281 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:23:01.282 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: pgmap v618: 129 pgs: 129 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:23:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:23:01.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 -- 172.21.15.57:0/3935379715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801051f0 msgr2=0x7f57801075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 --2- 172.21.15.57:0/3935379715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801051f0 0x7f57801075e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f576c009900 tx=0x7f576c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 -- 172.21.15.57:0/3935379715 shutdown_connections 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 --2- 172.21.15.57:0/3935379715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5780107b20 0x7f5780109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 --2- 172.21.15.57:0/3935379715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801051f0 0x7f57801075e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 --2- 172.21.15.57:0/3935379715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57801028c0 0x7f5780104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 -- 172.21.15.57:0/3935379715 >> 172.21.15.57:0/3935379715 conn(0x7f57800fc290 msgr2=0x7f57800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 -- 172.21.15.57:0/3935379715 shutdown_connections 2024-04-13T05:23:01.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.549+0000 7f577fe00640 1 -- 172.21.15.57:0/3935379715 wait complete. 2024-04-13T05:23:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.550+0000 7f577fe00640 1 Processor -- start 2024-04-13T05:23:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.550+0000 7f577fe00640 1 -- start start 2024-04-13T05:23:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801028c0 0x7f578019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57801051f0 0x7f578019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5780107b20 0x7f57801a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f578010fbd0 con 0x7f57801028c0 2024-04-13T05:23:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f578010fa50 con 0x7f57801051f0 2024-04-13T05:23:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f578010fd50 con 0x7f5780107b20 2024-04-13T05:23:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801028c0 0x7f578019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57801051f0 0x7f578019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57801051f0 0x7f578019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40002/0 (socket says 172.21.15.57:40002) 2024-04-13T05:23:01.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.551+0000 7f577e000640 1 -- 172.21.15.57:0/907715321 learned_addr learned my addr 172.21.15.57:0/907715321 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577f400640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5780107b20 0x7f57801a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577ea00640 1 -- 172.21.15.57:0/907715321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5780107b20 msgr2=0x7f57801a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577ea00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5780107b20 0x7f57801a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577ea00640 1 -- 172.21.15.57:0/907715321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57801051f0 msgr2=0x7f578019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577ea00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57801051f0 0x7f578019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:01.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577ea00640 1 -- 172.21.15.57:0/907715321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57801a1ed0 con 0x7f57801028c0 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.552+0000 7f577f400640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5780107b20 0x7f57801a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.553+0000 7f577ea00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801028c0 0x7f578019a2b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f577000e9f0 tx=0x7f577000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.553+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577000ccc0 con 0x7f57801028c0 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.553+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5770005020 con 0x7f57801028c0 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.553+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770004a00 con 0x7f57801028c0 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.553+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57801a21c0 con 0x7f57801028c0 2024-04-13T05:23:01.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.553+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57801a26c0 con 0x7f57801028c0 2024-04-13T05:23:01.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.555+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57800fdd80 con 0x7f57801028c0 2024-04-13T05:23:01.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.558+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5770004ba0 con 0x7f57801028c0 2024-04-13T05:23:01.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.560+0000 7f5777400640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57640a3c30 0x7f57640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:01.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.560+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f57700d0780 con 0x7f57801028c0 2024-04-13T05:23:01.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.561+0000 7f577e000640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57640a3c30 0x7f57640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:01.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.562+0000 7f577e000640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57640a3c30 0x7f57640a60f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f578019b7c0 tx=0x7f576c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:01.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.566+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f577009cd60 con 0x7f57801028c0 2024-04-13T05:23:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: pgmap v618: 129 pgs: 129 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:23:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:23:01.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: pgmap v618: 129 pgs: 129 active+clean; 286 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:23:01.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:23:02.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:01.999+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578002d5e0 con 0x7f57801028c0 2024-04-13T05:23:02.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.000+0000 7f5777400640 1 -- 172.21.15.57:0/907715321 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11858 (secure 0 0 0) 0x7f57700051d0 con 0x7f57801028c0 2024-04-13T05:23:02.003 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:23:02.003 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:22:16.935949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:23:02.003 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 29 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57640a3c30 msgr2=0x7f57640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57640a3c30 0x7f57640a60f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f578019b7c0 tx=0x7f576c02c7e0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801028c0 msgr2=0x7f578019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801028c0 0x7f578019a2b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f577000e9f0 tx=0x7f577000eeb0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 shutdown_connections 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57640a3c30 0x7f57640a60f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57801051f0 0x7f578019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5780107b20 0x7f57801a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 --2- 172.21.15.57:0/907715321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57801028c0 0x7f578019a2b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 >> 172.21.15.57:0/907715321 conn(0x7f57800fc290 msgr2=0x7f5780103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.003+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 shutdown_connections 2024-04-13T05:23:02.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.004+0000 7f577fe00640 1 -- 172.21.15.57:0/907715321 wait complete. 2024-04-13T05:23:02.029 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:23:02.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.311+0000 7f4a76000640 1 -- 172.21.15.57:0/2554698285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70102980 msgr2=0x7f4a70109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.311+0000 7f4a76000640 1 --2- 172.21.15.57:0/2554698285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70102980 0x7f4a70109210 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009ca0 tx=0x7f4a6802c120 comp rx=0 tx=0).stop 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 -- 172.21.15.57:0/2554698285 shutdown_connections 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 --2- 172.21.15.57:0/2554698285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70102980 0x7f4a70109210 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 --2- 172.21.15.57:0/2554698285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a70101fc0 0x7f4a70102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 --2- 172.21.15.57:0/2554698285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70100dd0 0x7f4a701011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 -- 172.21.15.57:0/2554698285 >> 172.21.15.57:0/2554698285 conn(0x7f4a700fc360 msgr2=0x7f4a700fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 -- 172.21.15.57:0/2554698285 shutdown_connections 2024-04-13T05:23:02.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.312+0000 7f4a76000640 1 -- 172.21.15.57:0/2554698285 wait complete. 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.313+0000 7f4a76000640 1 Processor -- start 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.313+0000 7f4a76000640 1 -- start start 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a76000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70100dd0 0x7f4a7019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a76000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a70101fc0 0x7f4a7019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a76000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 0x7f4a701a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a76000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010b800 con 0x7f4a70100dd0 2024-04-13T05:23:02.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a76000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010b680 con 0x7f4a70102980 2024-04-13T05:23:02.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a76000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010b980 con 0x7f4a70101fc0 2024-04-13T05:23:02.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a75600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 0x7f4a701a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a75600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 0x7f4a701a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40022/0 (socket says 172.21.15.57:40022) 2024-04-13T05:23:02.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.314+0000 7f4a75600640 1 -- 172.21.15.57:0/413327477 learned_addr learned my addr 172.21.15.57:0/413327477 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a70101fc0 0x7f4a7019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a74c00640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70100dd0 0x7f4a7019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a75600640 1 -- 172.21.15.57:0/413327477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a70101fc0 msgr2=0x7f4a7019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a75600640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a70101fc0 0x7f4a7019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a75600640 1 -- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70100dd0 msgr2=0x7f4a7019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a75600640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70100dd0 0x7f4a7019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.315+0000 7f4a75600640 1 -- 172.21.15.57:0/413327477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a701a1e70 con 0x7f4a70102980 2024-04-13T05:23:02.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.316+0000 7f4a75600640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 0x7f4a701a1800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4a68032530 tx=0x7f4a680347b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.316+0000 7f4a6d600640 1 -- 172.21.15.57:0/413327477 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68039040 con 0x7f4a70102980 2024-04-13T05:23:02.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.316+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a701a2100 con 0x7f4a70102980 2024-04-13T05:23:02.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.316+0000 7f4a6d600640 1 -- 172.21.15.57:0/413327477 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a68044070 con 0x7f4a70102980 2024-04-13T05:23:02.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.316+0000 7f4a6d600640 1 -- 172.21.15.57:0/413327477 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a680042a0 con 0x7f4a70102980 2024-04-13T05:23:02.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.317+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701a2640 con 0x7f4a70102980 2024-04-13T05:23:02.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.320+0000 7f4a6d600640 1 -- 172.21.15.57:0/413327477 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4a68003970 con 0x7f4a70102980 2024-04-13T05:23:02.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.321+0000 7f4a7a594740 1 Processor -- start 2024-04-13T05:23:02.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.321+0000 7f4a7a594740 1 -- start start 2024-04-13T05:23:02.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a7a594740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55afdbc82020 con 0x55afdbd1ce30 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a7a594740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55afdbc28950 con 0x55afdbd21fa0 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a7a594740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55afdbc7dfe0 con 0x55afdbd1c440 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60262/0 (socket says 172.21.15.57:60262) 2024-04-13T05:23:02.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.322+0000 7f4a6fe00640 1 -- 172.21.15.57:0/731975630 learned_addr learned my addr 172.21.15.57:0/731975630 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a6d600640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a580a3c10 0x7f4a580a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a74c00640 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a6d600640 1 -- 172.21.15.57:0/413327477 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4a680f2ab0 con 0x7f4a70102980 2024-04-13T05:23:02.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a6fe00640 1 -- 172.21.15.57:0/731975630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 msgr2=0x55afdbd24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd24430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.323+0000 7f4a6fe00640 1 -- 172.21.15.57:0/731975630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 msgr2=0x55afdbd21930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.324+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd21930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.324+0000 7f4a6fe00640 1 -- 172.21.15.57:0/731975630 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdb9943f0 con 0x55afdbd1c440 2024-04-13T05:23:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.324+0000 7f4a74c00640 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd21930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:02.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.324+0000 7f4a75600640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a580a3c10 0x7f4a580a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.324+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1c860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0136e0 tx=0x7f4a5c0096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79280bb08390770a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.324+0000 7f4a75600640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a580a3c10 0x7f4a580a60d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4a68003fd0 tx=0x7f4a680095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.325+0000 7f4a61c00640 1 -- 172.21.15.57:0/731975630 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c013080 con 0x55afdbd1c440 2024-04-13T05:23:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.325+0000 7f4a61c00640 1 -- 172.21.15.57:0/731975630 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c013220 con 0x55afdbd1c440 2024-04-13T05:23:02.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.325+0000 7f4a61c00640 1 -- 172.21.15.57:0/731975630 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00fc20 con 0x55afdbd1c440 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 -- 172.21.15.57:0/731975630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 msgr2=0x55afdbd1c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1c860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0136e0 tx=0x7f4a5c0096b0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 -- 172.21.15.57:0/731975630 shutdown_connections 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd21930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 --2- 172.21.15.57:0/731975630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1c860 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 -- 172.21.15.57:0/731975630 >> 172.21.15.57:0/731975630 conn(0x55afdbd179b0 msgr2=0x55afdbd17db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 -- 172.21.15.57:0/731975630 shutdown_connections 2024-04-13T05:23:02.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.326+0000 7f4a7a594740 1 -- 172.21.15.57:0/731975630 wait complete. 2024-04-13T05:23:02.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.327+0000 7f4a7a594740 1 -- start start 2024-04-13T05:23:02.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.327+0000 7f4a7a594740 1 Processor -- start 2024-04-13T05:23:02.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.328+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1c440 0x55afdbd26980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd26ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd21fa0 0x55afdbd2df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a7a594740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55afdbd24ef0 con 0x55afdbd1ce30 2024-04-13T05:23:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a7a594740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55afdbc3bc80 con 0x55afdbd1c440 2024-04-13T05:23:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a7a594740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55afdbd25070 con 0x55afdbd21fa0 2024-04-13T05:23:02.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a7a594740 10 client.0 _ll_register_callbacks cb 0x55afdbc3b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1c440 0x55afdbd26980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1c440 0x55afdbd26980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40046/0 (socket says 172.21.15.57:40046) 2024-04-13T05:23:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.329+0000 7f4a6fe00640 1 -- 172.21.15.57:0/2553377844 learned_addr learned my addr 172.21.15.57:0/2553377844 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.330+0000 7f4a74c00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd26ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.330+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd21fa0 0x55afdbd2df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a74c00640 1 -- 172.21.15.57:0/2553377844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd21fa0 msgr2=0x55afdbd2df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a74c00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd21fa0 0x55afdbd2df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a74c00640 1 -- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1c440 msgr2=0x55afdbd26980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a74c00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1c440 0x55afdbd26980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a74c00640 1 -- 172.21.15.57:0/2553377844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdba5e7d0 con 0x55afdbd1ce30 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a74c00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd26ec0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55afdbd1d230 tx=0x7f4a64011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9adfc1c548c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a56000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:02.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.331+0000 7f4a56000640 1 -- 172.21.15.57:0/2553377844 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64004030 con 0x55afdbd1ce30 2024-04-13T05:23:02.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.332+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5c013220 con 0x55afdbd1ce30 2024-04-13T05:23:02.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.332+0000 7f4a56000640 1 -- 172.21.15.57:0/2553377844 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4a640041d0 con 0x55afdbd1ce30 2024-04-13T05:23:02.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.332+0000 7f4a56000640 1 -- 172.21.15.57:0/2553377844 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a64025050 con 0x55afdbd1ce30 2024-04-13T05:23:02.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.332+0000 7f4a7a594740 10 client.16536 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:02.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.332+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55afdbd31330 con 0x55afdbd1ce30 2024-04-13T05:23:02.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.332+0000 7f4a56000640 1 -- 172.21.15.57:0/2553377844 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c013220 con 0x55afdbd1ce30 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a56000640 1 -- 172.21.15.57:0/2553377844 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4a64003ad0 con 0x55afdbd1ce30 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a7a594740 10 client.16536 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a7a594740 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a7a594740 20 client.16536 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a7a594740 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd329e0 0x55afdbd34e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a7a594740 4 client.16536 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:02.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afdbc21750 con 0x55afdbd329e0 2024-04-13T05:23:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.333+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd329e0 0x55afdbd34e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.334+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd329e0 0x55afdbd34e60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5f8d155ed0f534d server_cookie=eadb555a2017bec in_seq=0 out_seq=0 2024-04-13T05:23:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.334+0000 7f4a56000640 10 client.16536 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.335+0000 7f4a56000640 1 -- 172.21.15.57:0/2553377844 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afdbc21750 con 0x55afdbd329e0 2024-04-13T05:23:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.335+0000 7f4a56000640 10 client.16536 handle_command_reply: tid=0 2024-04-13T05:23:02.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.335+0000 7f4a7a594740 1 client.16536 shutdown 2024-04-13T05:23:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.335+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 msgr2=0x55afdbd26ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.335+0000 7f4a7a594740 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd26ec0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55afdbd1d230 tx=0x7f4a64011b50 comp rx=0 tx=0).stop 2024-04-13T05:23:02.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 shutdown_connections 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd329e0 0x55afdbd34e60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd21fa0 0x55afdbd2df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a56000640 0 client.16536 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd26ec0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 --2- 172.21.15.57:0/2553377844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1c440 0x55afdbd26980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 >> 172.21.15.57:0/2553377844 conn(0x55afdbd179b0 msgr2=0x55afdbd16cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 shutdown_connections 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 1 -- 172.21.15.57:0/2553377844 wait complete. 2024-04-13T05:23:02.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.336+0000 7f4a7a594740 20 client.16536 trim_cache size 0 max 16384 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.360+0000 7f4a7a594740 1 Processor -- start 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.360+0000 7f4a7a594740 1 -- start start 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a7a594740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701af400 con 0x55afdbd1ce30 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a7a594740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701acc70 con 0x55afdbd21fa0 2024-04-13T05:23:02.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a7a594740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701aaac0 con 0x55afdbd1c440 2024-04-13T05:23:02.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a74c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd39720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40050/0 (socket says 172.21.15.57:40050) 2024-04-13T05:23:02.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.361+0000 7f4a6fe00640 1 -- 172.21.15.57:0/359058112 learned_addr learned my addr 172.21.15.57:0/359058112 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.362+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.362+0000 7f4a6fe00640 1 -- 172.21.15.57:0/359058112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 msgr2=0x55afdbd1b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.362+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.362+0000 7f4a6fe00640 1 -- 172.21.15.57:0/359058112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 msgr2=0x55afdbd2dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.362+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd2dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.362+0000 7f4a6fe00640 1 -- 172.21.15.57:0/359058112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdbd18920 con 0x55afdbd21fa0 2024-04-13T05:23:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.363+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd39720 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55afdbd17830 tx=0x7f4a5c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bc4e725896c2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.363+0000 7f4a61200640 1 -- 172.21.15.57:0/359058112 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c013220 con 0x55afdbd21fa0 2024-04-13T05:23:02.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.363+0000 7f4a61200640 1 -- 172.21.15.57:0/359058112 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c00f730 con 0x55afdbd21fa0 2024-04-13T05:23:02.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.363+0000 7f4a61200640 1 -- 172.21.15.57:0/359058112 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c004680 con 0x55afdbd21fa0 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 -- 172.21.15.57:0/359058112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 msgr2=0x55afdbd39720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd39720 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55afdbd17830 tx=0x7f4a5c009b90 comp rx=0 tx=0).stop 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 -- 172.21.15.57:0/359058112 shutdown_connections 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd21fa0 0x55afdbd39720 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd1ce30 0x55afdbd2dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 --2- 172.21.15.57:0/359058112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd1b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.364+0000 7f4a7a594740 1 -- 172.21.15.57:0/359058112 >> 172.21.15.57:0/359058112 conn(0x55afdbd179b0 msgr2=0x55afdbd1ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.365+0000 7f4a7a594740 1 -- 172.21.15.57:0/359058112 shutdown_connections 2024-04-13T05:23:02.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.365+0000 7f4a7a594740 1 -- 172.21.15.57:0/359058112 wait complete. 2024-04-13T05:23:02.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.365+0000 7f4a7a594740 1 -- start start 2024-04-13T05:23:02.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.365+0000 7f4a7a594740 1 Processor -- start 2024-04-13T05:23:02.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd2b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 0x55afdbd2bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd21fa0 0x55afdbd2cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701ab130 con 0x55afdbd21fa0 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701ac5a0 con 0x55afdbd1ce30 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701ab800 con 0x55afdbd1c440 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.366+0000 7f4a7a594740 10 client.0 _ll_register_callbacks cb 0x7f4a6400bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 0x55afdbd2bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 0x55afdbd2bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40060/0 (socket says 172.21.15.57:40060) 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 -- 172.21.15.57:0/3921360586 learned_addr learned my addr 172.21.15.57:0/3921360586 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a6fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd21fa0 0x55afdbd2cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd2b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 -- 172.21.15.57:0/3921360586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 msgr2=0x55afdbd2b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd2b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 -- 172.21.15.57:0/3921360586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd21fa0 msgr2=0x55afdbd2cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd21fa0 0x55afdbd2cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a74c00640 1 -- 172.21.15.57:0/3921360586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdbd2d300 con 0x55afdbd1ce30 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.367+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd21fa0 0x55afdbd2cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.368+0000 7f4a74c00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 0x55afdbd2bd00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400be90 tx=0x7f4a64005330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6838b1ea236fdeca server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.368+0000 7f4a56000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.368+0000 7f4a56000640 1 -- 172.21.15.57:0/3921360586 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a640035f0 con 0x55afdbd1ce30 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.368+0000 7f4a56000640 1 -- 172.21.15.57:0/3921360586 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4a64003790 con 0x55afdbd1ce30 2024-04-13T05:23:02.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.368+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5c00f730 con 0x55afdbd1ce30 2024-04-13T05:23:02.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.369+0000 7f4a56000640 1 -- 172.21.15.57:0/3921360586 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a64003990 con 0x55afdbd1ce30 2024-04-13T05:23:02.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.369+0000 7f4a7a594740 10 client.35954 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.369+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55afdbd99190 con 0x55afdbd1ce30 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.369+0000 7f4a56000640 1 -- 172.21.15.57:0/3921360586 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c00f730 con 0x55afdbd1ce30 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.369+0000 7f4a56000640 1 -- 172.21.15.57:0/3921360586 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4a6401a800 con 0x55afdbd1ce30 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a7a594740 10 client.35954 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a7a594740 10 client.35954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a7a594740 20 client.35954 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a7a594740 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd99b10 0x55afdbd9bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a7a594740 4 client.35954 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afdbc8fa40 con 0x55afdbd99b10 2024-04-13T05:23:02.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.370+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd99b10 0x55afdbd9bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.371+0000 7f4a6fe00640 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd99b10 0x55afdbd9bf30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d09899bc62e0120 server_cookie=92a5d60d407b2a5f in_seq=0 out_seq=0 2024-04-13T05:23:02.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.371+0000 7f4a56000640 10 client.35954 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:02.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.371+0000 7f4a56000640 1 -- 172.21.15.57:0/3921360586 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afdbc8fa40 con 0x55afdbd99b10 2024-04-13T05:23:02.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.371+0000 7f4a56000640 10 client.35954 handle_command_reply: tid=0 2024-04-13T05:23:02.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.371+0000 7f4a7a594740 1 client.35954 shutdown 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 msgr2=0x55afdbd2bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 0x55afdbd2bd00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400be90 tx=0x7f4a64005330 comp rx=0 tx=0).stop 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 shutdown_connections 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55afdbd99b10 0x55afdbd9bf30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdbd1ce30 0x55afdbd2bd00 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400be90 tx=0x7f4a64005330 comp rx=0 tx=0).stop 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a56000640 0 client.35954 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55afdbd21fa0 0x55afdbd2cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 --2- 172.21.15.57:0/3921360586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdbd1c440 0x55afdbd2b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 >> 172.21.15.57:0/3921360586 conn(0x55afdbd179b0 msgr2=0x55afdbd1b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 shutdown_connections 2024-04-13T05:23:02.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.372+0000 7f4a7a594740 1 -- 172.21.15.57:0/3921360586 wait complete. 2024-04-13T05:23:02.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.373+0000 7f4a7a594740 20 client.35954 trim_cache size 0 max 16384 2024-04-13T05:23:02.376 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:23:02.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a580a3c10 msgr2=0x7f4a580a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a580a3c10 0x7f4a580a60d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4a68003fd0 tx=0x7f4a680095c0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 msgr2=0x7f4a701a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 0x7f4a701a1800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4a68032530 tx=0x7f4a680347b0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 shutdown_connections 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4a580a3c10 0x7f4a580a60d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a70102980 0x7f4a701a1800 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a70101fc0 0x7f4a7019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 --2- 172.21.15.57:0/413327477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a70100dd0 0x7f4a7019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.374+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 >> 172.21.15.57:0/413327477 conn(0x7f4a700fc360 msgr2=0x7f4a70190ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.375+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 shutdown_connections 2024-04-13T05:23:02.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.375+0000 7f4a76000640 1 -- 172.21.15.57:0/413327477 wait complete. 2024-04-13T05:23:02.400 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:23:02.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 -- 172.21.15.57:0/2351561331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6528104100 msgr2=0x7f6528104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 --2- 172.21.15.57:0/2351561331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6528104100 0x7f6528104520 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6518009900 tx=0x7f651802bc30 comp rx=0 tx=0).stop 2024-04-13T05:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 -- 172.21.15.57:0/2351561331 shutdown_connections 2024-04-13T05:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 --2- 172.21.15.57:0/2351561331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65281052f0 0x7f6528105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 --2- 172.21.15.57:0/2351561331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528105cb0 0x7f652810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 --2- 172.21.15.57:0/2351561331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6528104100 0x7f6528104520 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.662+0000 7f6527200640 1 -- 172.21.15.57:0/2351561331 >> 172.21.15.57:0/2351561331 conn(0x7f65280ff670 msgr2=0x7f6528101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.663+0000 7f6527200640 1 -- 172.21.15.57:0/2351561331 shutdown_connections 2024-04-13T05:23:02.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.663+0000 7f6527200640 1 -- 172.21.15.57:0/2351561331 wait complete. 2024-04-13T05:23:02.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.664+0000 7f6527200640 1 Processor -- start 2024-04-13T05:23:02.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.664+0000 7f6527200640 1 -- start start 2024-04-13T05:23:02.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6527200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 0x7f652819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6527200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65281052f0 0x7f652819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6527200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6528105cb0 0x7f65281a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6527200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f652810fc90 con 0x7f65281052f0 2024-04-13T05:23:02.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6527200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f652810fb10 con 0x7f6528105cb0 2024-04-13T05:23:02.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6527200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f652810fe10 con 0x7f6528104100 2024-04-13T05:23:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6525e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 0x7f652819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6526800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6528105cb0 0x7f65281a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6525e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 0x7f652819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60330/0 (socket says 172.21.15.57:60330) 2024-04-13T05:23:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.665+0000 7f6525e00640 1 -- 172.21.15.57:0/3469437992 learned_addr learned my addr 172.21.15.57:0/3469437992 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525400640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65281052f0 0x7f652819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525e00640 1 -- 172.21.15.57:0/3469437992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6528105cb0 msgr2=0x7f65281a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525e00640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6528105cb0 0x7f65281a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525e00640 1 -- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65281052f0 msgr2=0x7f652819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525e00640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65281052f0 0x7f652819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525e00640 1 -- 172.21.15.57:0/3469437992 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65281a1f70 con 0x7f6528104100 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.666+0000 7f6525400640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65281052f0 0x7f652819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:23:02.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.667+0000 7f6525e00640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 0x7f652819a350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f651802c840 tx=0x7f6518004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.667+0000 7f651ea00640 1 -- 172.21.15.57:0/3469437992 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518043020 con 0x7f6528104100 2024-04-13T05:23:02.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.667+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65281a2200 con 0x7f6528104100 2024-04-13T05:23:02.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.667+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65281a27a0 con 0x7f6528104100 2024-04-13T05:23:02.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.667+0000 7f651ea00640 1 -- 172.21.15.57:0/3469437992 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65180052b0 con 0x7f6528104100 2024-04-13T05:23:02.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.667+0000 7f651ea00640 1 -- 172.21.15.57:0/3469437992 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518003d90 con 0x7f6528104100 2024-04-13T05:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.671+0000 7f652fc09740 1 Processor -- start 2024-04-13T05:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.673+0000 7f651ea00640 1 -- 172.21.15.57:0/3469437992 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6518046020 con 0x7f6528104100 2024-04-13T05:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.673+0000 7f652fc09740 1 -- start start 2024-04-13T05:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.674+0000 7f652fc09740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.674+0000 7f652fc09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.674+0000 7f652fc09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.674+0000 7f652fc09740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56429a818020 con 0x56429a8bac50 2024-04-13T05:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.674+0000 7f652fc09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56429a7be950 con 0x56429a8c07b0 2024-04-13T05:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.674+0000 7f652fc09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56429a813fe0 con 0x56429a8bb640 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.675+0000 7f6525400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.675+0000 7f6525400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8c2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40074/0 (socket says 172.21.15.57:40074) 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.675+0000 7f6525400640 1 -- 172.21.15.57:0/1318020125 learned_addr learned my addr 172.21.15.57:0/1318020125 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.675+0000 7f6525e00640 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.675+0000 7f6526800640 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f6525e00640 1 -- 172.21.15.57:0/1318020125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 msgr2=0x56429a8c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f6525e00640 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f6525e00640 1 -- 172.21.15.57:0/1318020125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 msgr2=0x56429a8bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f6525e00640 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8bb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f6525e00640 1 -- 172.21.15.57:0/1318020125 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429a52a3f0 con 0x56429a8bb640 2024-04-13T05:23:02.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f651ea00640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f650c0b4c90 0x7f650c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.676+0000 7f6525e00640 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8c0140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f65180f0270 tx=0x7f65180f6c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c77f2417ba835f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.677+0000 7f651ea00640 1 -- 172.21.15.57:0/3469437992 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f65180f2bd0 con 0x7f6528104100 2024-04-13T05:23:02.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.677+0000 7f6513000640 1 -- 172.21.15.57:0/1318020125 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65180be880 con 0x56429a8bb640 2024-04-13T05:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.677+0000 7f6513000640 1 -- 172.21.15.57:0/1318020125 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6518104070 con 0x56429a8bb640 2024-04-13T05:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.677+0000 7f6513000640 1 -- 172.21.15.57:0/1318020125 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65180be5f0 con 0x56429a8bb640 2024-04-13T05:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.678+0000 7f6526800640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f650c0b4c90 0x7f650c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.678+0000 7f652fc09740 1 -- 172.21.15.57:0/1318020125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 msgr2=0x56429a8c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.678+0000 7f652fc09740 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8c0140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f65180f0270 tx=0x7f65180f6c00 comp rx=0 tx=0).stop 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f6526800640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f650c0b4c90 0x7f650c0b7150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56429a8bbea0 tx=0x7f6520009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 -- 172.21.15.57:0/1318020125 shutdown_connections 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8c0140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 --2- 172.21.15.57:0/1318020125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 -- 172.21.15.57:0/1318020125 >> 172.21.15.57:0/1318020125 conn(0x56429a8b61c0 msgr2=0x56429a8b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 -- 172.21.15.57:0/1318020125 shutdown_connections 2024-04-13T05:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 -- 172.21.15.57:0/1318020125 wait complete. 2024-04-13T05:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.679+0000 7f652fc09740 1 -- start start 2024-04-13T05:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 Processor -- start 2024-04-13T05:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bac50 0x56429a8aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 0x56429a8af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56429a8ad5e0 con 0x56429a8bb640 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56429a7d1c80 con 0x56429a8c07b0 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56429a8ad760 con 0x56429a8bac50 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.680+0000 7f652fc09740 10 client.0 _ll_register_callbacks cb 0x56429a7d1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 0x56429a8af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 0x56429a8af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42554/0 (socket says 172.21.15.57:42554) 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 -- 172.21.15.57:0/3600322994 learned_addr learned my addr 172.21.15.57:0/3600322994 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6526800640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525e00640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bac50 0x56429a8aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 -- 172.21.15.57:0/3600322994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bac50 msgr2=0x56429a8aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bac50 0x56429a8aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 -- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 msgr2=0x56429a8b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525400640 1 -- 172.21.15.57:0/3600322994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429a5f47d0 con 0x56429a8bb640 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.681+0000 7f6525e00640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bac50 0x56429a8aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.682+0000 7f6525400640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 0x56429a8af530 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f651400bb30 tx=0x7f65140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fffbefc1ec68c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.682+0000 7f6507400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.682+0000 7f6507400640 1 -- 172.21.15.57:0/3600322994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651401b640 con 0x56429a8bb640 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.683+0000 7f6507400640 1 -- 172.21.15.57:0/3600322994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f651401b7e0 con 0x56429a8bb640 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.683+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6518104070 con 0x56429a8bb640 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.683+0000 7f6507400640 1 -- 172.21.15.57:0/3600322994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6514024050 con 0x56429a8bb640 2024-04-13T05:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.683+0000 7f652fc09740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.683+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56429a8d8050 con 0x56429a8bb640 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.683+0000 7f6507400640 1 -- 172.21.15.57:0/3600322994 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6518104070 con 0x56429a8bb640 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f6507400640 1 -- 172.21.15.57:0/3600322994 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6514019020 con 0x56429a8bb640 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f652fc09740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f652fc09740 10 client.16554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f652fc09740 20 client.16554 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f652fc09740 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a8d97c0 0x56429a8dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f652fc09740 4 client.16554 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:02.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.684+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56429a7b7750 con 0x56429a8d97c0 2024-04-13T05:23:02.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.685+0000 7f6525e00640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a8d97c0 0x56429a8dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.686+0000 7f6525e00640 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a8d97c0 0x56429a8dbc40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49523c90c6c4c513 server_cookie=897a5b9d5a426c97 in_seq=0 out_seq=0 2024-04-13T05:23:02.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.686+0000 7f6507400640 10 client.16554 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:02.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.688+0000 7f6507400640 1 -- 172.21.15.57:0/3600322994 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56429a7b7750 con 0x56429a8d97c0 2024-04-13T05:23:02.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.688+0000 7f6507400640 10 client.16554 handle_command_reply: tid=0 2024-04-13T05:23:02.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.688+0000 7f652fc09740 1 client.16554 shutdown 2024-04-13T05:23:02.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.688+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 msgr2=0x56429a8af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.688+0000 7f652fc09740 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 0x56429a8af530 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f651400bb30 tx=0x7f65140045e0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 shutdown_connections 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a8d97c0 0x56429a8dbc40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8c07b0 0x56429a8b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bb640 0x56429a8af530 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 --2- 172.21.15.57:0/3600322994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bac50 0x56429a8aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f6507400640 0 client.16554 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 >> 172.21.15.57:0/3600322994 conn(0x56429a8b61c0 msgr2=0x56429a8accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 shutdown_connections 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 1 -- 172.21.15.57:0/3600322994 wait complete. 2024-04-13T05:23:02.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.689+0000 7f652fc09740 20 client.16554 trim_cache size 0 max 16384 2024-04-13T05:23:02.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 Processor -- start 2024-04-13T05:23:02.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 -- start start 2024-04-13T05:23:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bac50 0x56429a8adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8c07b0 0x56429a8bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f65281a2970 con 0x56429a8c07b0 2024-04-13T05:23:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6528071930 con 0x56429a8bac50 2024-04-13T05:23:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.712+0000 7f652fc09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f652806f780 con 0x56429a8bb640 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6525e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bac50 0x56429a8adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6525e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bac50 0x56429a8adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40088/0 (socket says 172.21.15.57:40088) 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6525e00640 1 -- 172.21.15.57:0/3909049066 learned_addr learned my addr 172.21.15.57:0/3909049066 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6526800640 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8c07b0 0x56429a8bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6525400640 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6526800640 1 -- 172.21.15.57:0/3909049066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 msgr2=0x56429a8ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6526800640 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6526800640 1 -- 172.21.15.57:0/3909049066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bac50 msgr2=0x56429a8adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6526800640 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bac50 0x56429a8adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.713+0000 7f6526800640 1 -- 172.21.15.57:0/3909049066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56429a8ad870 con 0x56429a8c07b0 2024-04-13T05:23:02.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.714+0000 7f6526800640 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8c07b0 0x56429a8bcb40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6520005cc0 tx=0x7f6520015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ff64496c25e825 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.715+0000 7f6512600640 1 -- 172.21.15.57:0/3909049066 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520020310 con 0x56429a8c07b0 2024-04-13T05:23:02.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.715+0000 7f6512600640 1 -- 172.21.15.57:0/3909049066 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6520010520 con 0x56429a8c07b0 2024-04-13T05:23:02.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.715+0000 7f6512600640 1 -- 172.21.15.57:0/3909049066 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520005260 con 0x56429a8c07b0 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 -- 172.21.15.57:0/3909049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8c07b0 msgr2=0x56429a8bcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8c07b0 0x56429a8bcb40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6520005cc0 tx=0x7f6520015000 comp rx=0 tx=0).stop 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 -- 172.21.15.57:0/3909049066 shutdown_connections 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8c07b0 0x56429a8bcb40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8bb640 0x56429a8ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 --2- 172.21.15.57:0/3909049066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bac50 0x56429a8adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 -- 172.21.15.57:0/3909049066 >> 172.21.15.57:0/3909049066 conn(0x56429a8b61c0 msgr2=0x56429a8b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 -- 172.21.15.57:0/3909049066 shutdown_connections 2024-04-13T05:23:02.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 -- 172.21.15.57:0/3909049066 wait complete. 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.716+0000 7f652fc09740 1 -- start start 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 Processor -- start 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8ead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bb640 0x56429a8eb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8c07b0 0x56429a8ec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f652806fdf0 con 0x56429a8bac50 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6528071260 con 0x56429a8bb640 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65280704c0 con 0x56429a8c07b0 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.717+0000 7f652fc09740 10 client.0 _ll_register_callbacks cb 0x7f65140044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8ead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42570/0 (socket says 172.21.15.57:42570) 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 -- 172.21.15.57:0/1410990975 learned_addr learned my addr 172.21.15.57:0/1410990975 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:02.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525400640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bb640 0x56429a8eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 -- 172.21.15.57:0/1410990975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8c07b0 msgr2=0x56429a8ec280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6526800640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8c07b0 0x56429a8ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8c07b0 0x56429a8ec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 -- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bb640 msgr2=0x56429a8eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bb640 0x56429a8eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525e00640 1 -- 172.21.15.57:0/1410990975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429a8ec870 con 0x56429a8bac50 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.718+0000 7f6525400640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bb640 0x56429a8eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.719+0000 7f6525e00640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8ead30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f65180f60b0 tx=0x7f65180f67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38005fb5e116895a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.719+0000 7f6507400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.719+0000 7f6507400640 1 -- 172.21.15.57:0/1410990975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65180bcbf0 con 0x56429a8bac50 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.719+0000 7f6507400640 1 -- 172.21.15.57:0/1410990975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f65180f03e0 con 0x56429a8bac50 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.719+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6520010520 con 0x56429a8bac50 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.720+0000 7f6507400640 1 -- 172.21.15.57:0/1410990975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f65180be410 con 0x56429a8bac50 2024-04-13T05:23:02.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.720+0000 7f652fc09740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:02.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.720+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56429a93fe10 con 0x56429a8bac50 2024-04-13T05:23:02.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.721+0000 7f6507400640 1 -- 172.21.15.57:0/1410990975 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6520010520 con 0x56429a8bac50 2024-04-13T05:23:02.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.721+0000 7f6507400640 1 -- 172.21.15.57:0/1410990975 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f65180bb090 con 0x56429a8bac50 2024-04-13T05:23:02.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f652fc09740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:02.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f652fc09740 10 client.16560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:23:02.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f652fc09740 20 client.16560 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f652fc09740 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a940550 0x56429a942970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f652fc09740 4 client.16560 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56429a825a40 con 0x56429a940550 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.722+0000 7f6525400640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a940550 0x56429a942970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.723+0000 7f6507400640 10 client.16560 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.723+0000 7f6525400640 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a940550 0x56429a942970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54eede14c8b24eca server_cookie=863fe2c27f8368f9 in_seq=0 out_seq=0 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.723+0000 7f6507400640 1 -- 172.21.15.57:0/1410990975 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56429a825a40 con 0x56429a940550 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.723+0000 7f6507400640 10 client.16560 handle_command_reply: tid=0 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.723+0000 7f652fc09740 1 client.16560 shutdown 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 msgr2=0x56429a8ead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8ead30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f65180f60b0 tx=0x7f65180f67b0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 shutdown_connections 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56429a940550 0x56429a942970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56429a8bb640 0x56429a8eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56429a8c07b0 0x56429a8ec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 --2- 172.21.15.57:0/1410990975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56429a8bac50 0x56429a8ead30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f6507400640 0 client.16560 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.724+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 >> 172.21.15.57:0/1410990975 conn(0x56429a8b61c0 msgr2=0x56429a8bcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.725+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 shutdown_connections 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.725+0000 7f652fc09740 1 -- 172.21.15.57:0/1410990975 wait complete. 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.725+0000 7f652fc09740 20 client.16560 trim_cache size 0 max 16384 2024-04-13T05:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f650c0b4c90 msgr2=0x7f650c0b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f650c0b4c90 0x7f650c0b7150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56429a8bbea0 tx=0x7f6520009600 comp rx=0 tx=0).stop 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 msgr2=0x7f652819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 0x7f652819a350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f651802c840 tx=0x7f6518004270 comp rx=0 tx=0).stop 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 shutdown_connections 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f650c0b4c90 0x7f650c0b7150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6528105cb0 0x7f65281a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65281052f0 0x7f652819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 --2- 172.21.15.57:0/3469437992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6528104100 0x7f652819a350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:02.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 >> 172.21.15.57:0/3469437992 conn(0x7f65280ff670 msgr2=0x7f6528108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:02.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 shutdown_connections 2024-04-13T05:23:02.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:02.726+0000 7f6527200640 1 -- 172.21.15.57:0/3469437992 wait complete. 2024-04-13T05:23:02.748 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:02 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-04-13T05:23:02.748 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:02 smithi057 ceph-mon[32331]: Unable to set osd_memory_target on smithi057 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-13T05:23:02.748 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/907715321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:23:02.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:02 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-04-13T05:23:02.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:02 smithi083 ceph-mon[35133]: Unable to set osd_memory_target on smithi057 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-13T05:23:02.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:02 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/907715321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:23:02.753 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:23:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:02 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-04-13T05:23:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:02 smithi189 ceph-mon[35531]: Unable to set osd_memory_target on smithi057 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-13T05:23:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:02 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/907715321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 -- 172.21.15.57:0/1114077073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 msgr2=0x7f9d8c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 --2- 172.21.15.57:0/1114077073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 0x7f9d8c104d40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009ca0 tx=0x7f9d8402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 -- 172.21.15.57:0/1114077073 shutdown_connections 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 --2- 172.21.15.57:0/1114077073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c10a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 --2- 172.21.15.57:0/1114077073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d8c105280 0x7f9d8c107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 --2- 172.21.15.57:0/1114077073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 0x7f9d8c104d40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 -- 172.21.15.57:0/1114077073 >> 172.21.15.57:0/1114077073 conn(0x7f9d8c0fc360 msgr2=0x7f9d8c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 -- 172.21.15.57:0/1114077073 shutdown_connections 2024-04-13T05:23:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.016+0000 7f9d91400640 1 -- 172.21.15.57:0/1114077073 wait complete. 2024-04-13T05:23:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.018+0000 7f9d91400640 1 Processor -- start 2024-04-13T05:23:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.018+0000 7f9d91400640 1 -- start start 2024-04-13T05:23:03.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d91400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 0x7f9d8c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d91400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d8c105280 0x7f9d8c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d91400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10da10 con 0x7f9d8c102950 2024-04-13T05:23:03.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d91400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10d890 con 0x7f9d8c107bb0 2024-04-13T05:23:03.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d91400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10db90 con 0x7f9d8c105280 2024-04-13T05:23:03.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d90a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d90a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40128/0 (socket says 172.21.15.57:40128) 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d90a00640 1 -- 172.21.15.57:0/344955926 learned_addr learned my addr 172.21.15.57:0/344955926 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d8be00640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 0x7f9d8c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.019+0000 7f9d8b400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d8c105280 0x7f9d8c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.020+0000 7f9d90a00640 1 -- 172.21.15.57:0/344955926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d8c105280 msgr2=0x7f9d8c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.020+0000 7f9d90a00640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d8c105280 0x7f9d8c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.020+0000 7f9d90a00640 1 -- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 msgr2=0x7f9d8c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.020+0000 7f9d90a00640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 0x7f9d8c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.020+0000 7f9d90a00640 1 -- 172.21.15.57:0/344955926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c19fd30 con 0x7f9d8c107bb0 2024-04-13T05:23:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.020+0000 7f9d90a00640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c19f6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009f70 tx=0x7f9d7c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:03.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.021+0000 7f9d88c00640 1 -- 172.21.15.57:0/344955926 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00cd20 con 0x7f9d8c107bb0 2024-04-13T05:23:03.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.021+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c1007a0 con 0x7f9d8c107bb0 2024-04-13T05:23:03.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.021+0000 7f9d88c00640 1 -- 172.21.15.57:0/344955926 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c0054d0 con 0x7f9d8c107bb0 2024-04-13T05:23:03.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.021+0000 7f9d88c00640 1 -- 172.21.15.57:0/344955926 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0047a0 con 0x7f9d8c107bb0 2024-04-13T05:23:03.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.021+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c100cf0 con 0x7f9d8c107bb0 2024-04-13T05:23:03.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.025+0000 7f9d95881740 1 Processor -- start 2024-04-13T05:23:03.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.026+0000 7f9d88c00640 1 -- 172.21.15.57:0/344955926 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9d7c004940 con 0x7f9d8c107bb0 2024-04-13T05:23:03.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d88c00640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d700a3c30 0x7f9d700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d88c00640 1 -- 172.21.15.57:0/344955926 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9d7c0d0670 con 0x7f9d8c107bb0 2024-04-13T05:23:03.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d95881740 1 -- start start 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d95881740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a8c50 0x5601024a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d95881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a9640 0x5601024ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d8b400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d700a3c30 0x7f9d700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.029+0000 7f9d95881740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d95881740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560102406020 con 0x5601024ae7b0 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d95881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5601023ac950 con 0x5601024a9640 2024-04-13T05:23:03.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d95881740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560102401fe0 con 0x5601024a8c50 2024-04-13T05:23:03.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d90a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d90a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42604/0 (socket says 172.21.15.57:42604) 2024-04-13T05:23:03.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d90a00640 1 -- 172.21.15.57:0/169991530 learned_addr learned my addr 172.21.15.57:0/169991530 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.030+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a8c50 0x5601024a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d90a00640 1 -- 172.21.15.57:0/169991530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a8c50 msgr2=0x5601024a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d90a00640 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a8c50 0x5601024a9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d90a00640 1 -- 172.21.15.57:0/169991530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a9640 msgr2=0x5601024ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d90a00640 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a9640 0x5601024ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d90a00640 1 -- 172.21.15.57:0/169991530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601021183f0 con 0x5601024ae7b0 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d8b400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d700a3c30 0x7f9d700a60f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c199620 tx=0x7f9d740041d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:03.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d90a00640 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024b0c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c098ac0 tx=0x7f9d7c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40741b7b0ba12b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:03.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.031+0000 7f9d81200640 1 -- 172.21.15.57:0/169991530 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c09bfc0 con 0x5601024ae7b0 2024-04-13T05:23:03.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.032+0000 7f9d81200640 1 -- 172.21.15.57:0/169991530 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c09c660 con 0x5601024ae7b0 2024-04-13T05:23:03.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.032+0000 7f9d81200640 1 -- 172.21.15.57:0/169991530 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c09aa30 con 0x5601024ae7b0 2024-04-13T05:23:03.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.032+0000 7f9d95881740 1 -- 172.21.15.57:0/169991530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 msgr2=0x5601024b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.032+0000 7f9d95881740 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024b0c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c098ac0 tx=0x7f9d7c0097d0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 -- 172.21.15.57:0/169991530 shutdown_connections 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a9640 0x5601024ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024b0c40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 --2- 172.21.15.57:0/169991530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a8c50 0x5601024a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 -- 172.21.15.57:0/169991530 >> 172.21.15.57:0/169991530 conn(0x5601024a41c0 msgr2=0x5601024a45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 -- 172.21.15.57:0/169991530 shutdown_connections 2024-04-13T05:23:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.033+0000 7f9d95881740 1 -- 172.21.15.57:0/169991530 wait complete. 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 -- start start 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 Processor -- start 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5601024b1700 con 0x5601024ae7b0 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5601023bfc80 con 0x5601024a8c50 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5601024b1880 con 0x5601024a9640 2024-04-13T05:23:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.034+0000 7f9d95881740 10 client.0 _ll_register_callbacks cb 0x5601023bf670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:03.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.035+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.035+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40152/0 (socket says 172.21.15.57:40152) 2024-04-13T05:23:03.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.035+0000 7f9d8be00640 1 -- 172.21.15.57:0/4237430773 learned_addr learned my addr 172.21.15.57:0/4237430773 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.035+0000 7f9d90a00640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.035+0000 7f9d8b400640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d8be00640 1 -- 172.21.15.57:0/4237430773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 msgr2=0x5601024b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d8be00640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d8be00640 1 -- 172.21.15.57:0/4237430773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 msgr2=0x5601024ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d8be00640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024ba710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d8be00640 1 -- 172.21.15.57:0/4237430773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601021e27d0 con 0x5601024a8c50 2024-04-13T05:23:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d90a00640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.036+0000 7f9d8be00640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b3190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d84032420 tx=0x7f9d84009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65e57f28e46a2bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.037+0000 7f9d79600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.037+0000 7f9d79600640 1 -- 172.21.15.57:0/4237430773 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84044070 con 0x5601024a8c50 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.037+0000 7f9d79600640 1 -- 172.21.15.57:0/4237430773 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9d840325f0 con 0x5601024a8c50 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.037+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c09c660 con 0x5601024a8c50 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.038+0000 7f9d79600640 1 -- 172.21.15.57:0/4237430773 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d84050050 con 0x5601024a8c50 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.038+0000 7f9d95881740 10 client.35981 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.038+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601024bdaf0 con 0x5601024a8c50 2024-04-13T05:23:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.038+0000 7f9d79600640 1 -- 172.21.15.57:0/4237430773 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c09c660 con 0x5601024a8c50 2024-04-13T05:23:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d79600640 1 -- 172.21.15.57:0/4237430773 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9d84004330 con 0x5601024a8c50 2024-04-13T05:23:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d95881740 10 client.35981 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d95881740 10 client.35981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d95881740 20 client.35981 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d95881740 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5601024bf120 0x5601024c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d95881740 4 client.35981 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.039+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601023a5750 con 0x5601024bf120 2024-04-13T05:23:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.040+0000 7f9d8b400640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5601024bf120 0x5601024c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.040+0000 7f9d8b400640 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5601024bf120 0x5601024c1590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30db494f55e794e5 server_cookie=9c8a8d1326402e41 in_seq=0 out_seq=0 2024-04-13T05:23:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.040+0000 7f9d79600640 10 client.35981 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.041+0000 7f9d79600640 1 -- 172.21.15.57:0/4237430773 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601023a5750 con 0x5601024bf120 2024-04-13T05:23:03.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.041+0000 7f9d79600640 10 client.35981 handle_command_reply: tid=0 2024-04-13T05:23:03.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.041+0000 7f9d95881740 1 client.35981 shutdown 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 msgr2=0x5601024b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b3190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d84032420 tx=0x7f9d84009800 comp rx=0 tx=0).stop 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 shutdown_connections 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5601024bf120 0x5601024c1590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 --2- 172.21.15.57:0/4237430773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b3190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d79600640 0 client.35981 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 >> 172.21.15.57:0/4237430773 conn(0x5601024a41c0 msgr2=0x56010249acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 shutdown_connections 2024-04-13T05:23:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 1 -- 172.21.15.57:0/4237430773 wait complete. 2024-04-13T05:23:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.042+0000 7f9d95881740 20 client.35981 trim_cache size 0 max 16384 2024-04-13T05:23:03.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.065+0000 7f9d95881740 1 Processor -- start 2024-04-13T05:23:03.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.065+0000 7f9d95881740 1 -- start start 2024-04-13T05:23:03.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d95881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x560102496b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d95881740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d95881740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d95881740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1ab0a0 con 0x5601024ae7b0 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d95881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1a8910 con 0x5601024a8c50 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d95881740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1a6760 con 0x5601024a9640 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x560102496b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.066+0000 7f9d8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60420/0 (socket says 172.21.15.57:60420) 2024-04-13T05:23:03.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 -- 172.21.15.57:0/2673612290 learned_addr learned my addr 172.21.15.57:0/2673612290 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:03.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d90a00640 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 -- 172.21.15.57:0/2673612290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 msgr2=0x560102496b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x560102496b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 -- 172.21.15.57:0/2673612290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 msgr2=0x5601024a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d8b400640 1 -- 172.21.15.57:0/2673612290 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601024970a0 con 0x5601024a9640 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.067+0000 7f9d90a00640 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.068+0000 7f9d8b400640 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024ba6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005fd0 tx=0x7f9d74015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785569b37dc92c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.068+0000 7f9d7be00640 1 -- 172.21.15.57:0/2673612290 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7401f8f0 con 0x5601024a9640 2024-04-13T05:23:03.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.069+0000 7f9d7be00640 1 -- 172.21.15.57:0/2673612290 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7402d070 con 0x5601024a9640 2024-04-13T05:23:03.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.069+0000 7f9d7be00640 1 -- 172.21.15.57:0/2673612290 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7401e6a0 con 0x5601024a9640 2024-04-13T05:23:03.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 -- 172.21.15.57:0/2673612290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 msgr2=0x5601024ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024ba6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005fd0 tx=0x7f9d74015000 comp rx=0 tx=0).stop 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 -- 172.21.15.57:0/2673612290 shutdown_connections 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024ae7b0 0x5601024a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024a9640 0x5601024ba6d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 --2- 172.21.15.57:0/2673612290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x560102496b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 -- 172.21.15.57:0/2673612290 >> 172.21.15.57:0/2673612290 conn(0x5601024a41c0 msgr2=0x56010249acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 -- 172.21.15.57:0/2673612290 shutdown_connections 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 -- 172.21.15.57:0/2673612290 wait complete. 2024-04-13T05:23:03.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.070+0000 7f9d95881740 1 -- start start 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.071+0000 7f9d95881740 1 Processor -- start 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.071+0000 7f9d95881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.071+0000 7f9d95881740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 0x5601024b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.071+0000 7f9d95881740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024ae7b0 0x5601024b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d95881740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1a6dd0 con 0x5601024a9640 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d95881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1a8240 con 0x5601024a8c50 2024-04-13T05:23:03.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d95881740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1a74a0 con 0x5601024ae7b0 2024-04-13T05:23:03.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d8b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 0x5601024b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d8b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 0x5601024b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42624/0 (socket says 172.21.15.57:42624) 2024-04-13T05:23:03.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d8b400640 1 -- 172.21.15.57:0/3265921823 learned_addr learned my addr 172.21.15.57:0/3265921823 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:03.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d95881740 10 client.0 _ll_register_callbacks cb 0x7f9d84005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:03.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d8be00640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.072+0000 7f9d90a00640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024ae7b0 0x5601024b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.073+0000 7f9d8b400640 1 -- 172.21.15.57:0/3265921823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024ae7b0 msgr2=0x5601024b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.073+0000 7f9d8b400640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024ae7b0 0x5601024b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.073+0000 7f9d8b400640 1 -- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 msgr2=0x5601024b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:03.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.073+0000 7f9d8b400640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.073+0000 7f9d8b400640 1 -- 172.21.15.57:0/3265921823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601024b9480 con 0x5601024a9640 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.074+0000 7f9d8b400640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 0x5601024b7e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9d74000c00 tx=0x7f9d74003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af4fdcdc8b23543 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.074+0000 7f9d79600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.074+0000 7f9d79600640 1 -- 172.21.15.57:0/3265921823 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74035070 con 0x5601024a9640 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.074+0000 7f9d79600640 1 -- 172.21.15.57:0/3265921823 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9d7401f8f0 con 0x5601024a9640 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.074+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7402d070 con 0x5601024a9640 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d79600640 1 -- 172.21.15.57:0/3265921823 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d7401faf0 con 0x5601024a9640 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d95881740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:03.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601024ba790 con 0x5601024a9640 2024-04-13T05:23:03.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d79600640 1 -- 172.21.15.57:0/3265921823 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7402d070 con 0x5601024a9640 2024-04-13T05:23:03.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d79600640 1 -- 172.21.15.57:0/3265921823 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9d74042020 con 0x5601024a9640 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d95881740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d95881740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d95881740 20 client.16578 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.075+0000 7f9d95881740 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x560102525ca0 0x5601025280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.076+0000 7f9d95881740 4 client.16578 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.076+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560102413a40 con 0x560102525ca0 2024-04-13T05:23:03.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.076+0000 7f9d8be00640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x560102525ca0 0x5601025280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d8be00640 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x560102525ca0 0x5601025280c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538ae11f37598de6 server_cookie=fa047f1292570570 in_seq=0 out_seq=0 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d79600640 10 client.16578 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d79600640 1 -- 172.21.15.57:0/3265921823 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560102413a40 con 0x560102525ca0 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d79600640 10 client.16578 handle_command_reply: tid=0 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d95881740 1 client.16578 shutdown 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 msgr2=0x5601024b7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.077+0000 7f9d95881740 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 0x5601024b7e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9d74000c00 tx=0x7f9d74003dd0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 shutdown_connections 2024-04-13T05:23:03.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x560102525ca0 0x5601025280c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601024ae7b0 0x5601024b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5601024a9640 0x5601024b7e80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 --2- 172.21.15.57:0/3265921823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601024a8c50 0x5601024b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d79600640 0 client.16578 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:03.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 >> 172.21.15.57:0/3265921823 conn(0x5601024a41c0 msgr2=0x56010249acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:03.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 shutdown_connections 2024-04-13T05:23:03.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 1 -- 172.21.15.57:0/3265921823 wait complete. 2024-04-13T05:23:03.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.078+0000 7f9d95881740 20 client.16578 trim_cache size 0 max 16384 2024-04-13T05:23:03.082 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d700a3c30 msgr2=0x7f9d700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d700a3c30 0x7f9d700a60f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c199620 tx=0x7f9d740041d0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 msgr2=0x7f9d8c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c19f6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009f70 tx=0x7f9d7c00edd0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 shutdown_connections 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9d700a3c30 0x7f9d700a60f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8c107bb0 0x7f9d8c19f6c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d8c105280 0x7f9d8c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 --2- 172.21.15.57:0/344955926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9d8c102950 0x7f9d8c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.079+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 >> 172.21.15.57:0/344955926 conn(0x7f9d8c0fc360 msgr2=0x7f9d8c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:03.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.080+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 shutdown_connections 2024-04-13T05:23:03.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:03.080+0000 7f9d91400640 1 -- 172.21.15.57:0/344955926 wait complete. 2024-04-13T05:23:03.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:03 smithi083 ceph-mon[35133]: pgmap v619: 129 pgs: 129 active+clean; 289 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-04-13T05:23:03.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:03 smithi057 ceph-mon[32331]: pgmap v619: 129 pgs: 129 active+clean; 289 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-04-13T05:23:03.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:03 smithi189 ceph-mon[35531]: pgmap v619: 129 pgs: 129 active+clean; 289 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-04-13T05:23:04.103 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:23:04.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:23:04.104 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:23:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 -- 172.21.15.57:0/3865751738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f792410a0b0 msgr2=0x7f7924110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 --2- 172.21.15.57:0/3865751738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f792410a0b0 0x7f7924110940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f791c009ca0 tx=0x7f791c02c120 comp rx=0 tx=0).stop 2024-04-13T05:23:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 -- 172.21.15.57:0/3865751738 shutdown_connections 2024-04-13T05:23:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 --2- 172.21.15.57:0/3865751738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79241096f0 0x7f7924109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 --2- 172.21.15.57:0/3865751738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f792410a0b0 0x7f7924110940 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 --2- 172.21.15.57:0/3865751738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7924108500 0x7f7924108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 -- 172.21.15.57:0/3865751738 >> 172.21.15.57:0/3865751738 conn(0x7f7924077490 msgr2=0x7f79240798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 -- 172.21.15.57:0/3865751738 shutdown_connections 2024-04-13T05:23:04.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.363+0000 7f7929800640 1 -- 172.21.15.57:0/3865751738 wait complete. 2024-04-13T05:23:04.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 Processor -- start 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 -- start start 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 0x7f792419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79241096f0 0x7f792419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f792410a0b0 0x7f79241a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7924114040 con 0x7f7924108500 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7924113ec0 con 0x7f792410a0b0 2024-04-13T05:23:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.365+0000 7f7929800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79241141c0 con 0x7f79241096f0 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 0x7f792419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79241096f0 0x7f792419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 0x7f792419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42648/0 (socket says 172.21.15.57:42648) 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 -- 172.21.15.57:0/2589524109 learned_addr learned my addr 172.21.15.57:0/2589524109 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7928e00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f792410a0b0 0x7f79241a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 -- 172.21.15.57:0/2589524109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79241096f0 msgr2=0x7f792419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79241096f0 0x7f792419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 -- 172.21.15.57:0/2589524109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f792410a0b0 msgr2=0x7f79241a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f792410a0b0 0x7f79241a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.366+0000 7f7923e00640 1 -- 172.21.15.57:0/2589524109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241a6340 con 0x7f7924108500 2024-04-13T05:23:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.367+0000 7f7923e00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 0x7f792419e770 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f791400d950 tx=0x7f791400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.367+0000 7f7920c00640 1 -- 172.21.15.57:0/2589524109 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914017070 con 0x7f7924108500 2024-04-13T05:23:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.368+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79241a6630 con 0x7f7924108500 2024-04-13T05:23:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.368+0000 7f7920c00640 1 -- 172.21.15.57:0/2589524109 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7914009c30 con 0x7f7924108500 2024-04-13T05:23:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.368+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241a6b80 con 0x7f7924108500 2024-04-13T05:23:04.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.368+0000 7f7920c00640 1 -- 172.21.15.57:0/2589524109 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914021060 con 0x7f7924108500 2024-04-13T05:23:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.372+0000 7f792dd70740 1 Processor -- start 2024-04-13T05:23:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.372+0000 7f792dd70740 1 -- start start 2024-04-13T05:23:04.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.373+0000 7f792dd70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1e430 0x55da22b1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.373+0000 7f792dd70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b23920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.373+0000 7f792dd70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.373+0000 7f792dd70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55da22a73bc0 con 0x55da22b1e430 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.373+0000 7f792dd70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55da22a1a660 con 0x55da22b23f90 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.373+0000 7f792dd70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da22a6fb80 con 0x55da22b1ee20 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7920c00640 1 -- 172.21.15.57:0/2589524109 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f791401a070 con 0x7f7924108500 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b26420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40208/0 (socket says 172.21.15.57:40208) 2024-04-13T05:23:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 -- 172.21.15.57:0/438955584 learned_addr learned my addr 172.21.15.57:0/438955584 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 -- 172.21.15.57:0/438955584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 msgr2=0x55da22b23920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7928e00640 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1e430 0x55da22b1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b23920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 -- 172.21.15.57:0/438955584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1e430 msgr2=0x55da22b1e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1e430 0x55da22b1e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.374+0000 7f7923400640 1 -- 172.21.15.57:0/438955584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da22a3d220 con 0x55da22b23f90 2024-04-13T05:23:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.375+0000 7f7928e00640 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1e430 0x55da22b1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.375+0000 7f7923400640 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b26420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f791000bc90 tx=0x7f79100107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf7896c14e30eea server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.376+0000 7f7920c00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79080a3c00 0x7f79080a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.376+0000 7f7920c00640 1 -- 172.21.15.57:0/2589524109 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f79140cf4f0 con 0x7f7924108500 2024-04-13T05:23:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.376+0000 7f7919600640 1 -- 172.21.15.57:0/438955584 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910005220 con 0x55da22b23f90 2024-04-13T05:23:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.376+0000 7f7919600640 1 -- 172.21.15.57:0/438955584 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f791000be40 con 0x55da22b23f90 2024-04-13T05:23:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.376+0000 7f7919600640 1 -- 172.21.15.57:0/438955584 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100117b0 con 0x55da22b23f90 2024-04-13T05:23:04.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.377+0000 7f7928e00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79080a3c00 0x7f79080a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.377+0000 7f792dd70740 1 -- 172.21.15.57:0/438955584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 msgr2=0x55da22b26420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.377+0000 7f792dd70740 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b26420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f791000bc90 tx=0x7f79100107a0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f7928e00640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79080a3c00 0x7f79080a60c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55da22b1f680 tx=0x7f791c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 -- 172.21.15.57:0/438955584 shutdown_connections 2024-04-13T05:23:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b26420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b23920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 --2- 172.21.15.57:0/438955584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1e430 0x55da22b1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 -- 172.21.15.57:0/438955584 >> 172.21.15.57:0/438955584 conn(0x55da22b08990 msgr2=0x55da22b08d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 -- 172.21.15.57:0/438955584 shutdown_connections 2024-04-13T05:23:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 -- 172.21.15.57:0/438955584 wait complete. 2024-04-13T05:23:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.378+0000 7f792dd70740 1 -- start start 2024-04-13T05:23:04.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 Processor -- start 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b28f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b2ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55da22b26f40 con 0x55da22b1ee20 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55da22a2da30 con 0x55da22b23f90 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da22b270c0 con 0x55da22b1e430 2024-04-13T05:23:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.379+0000 7f792dd70740 10 client.0 _ll_register_callbacks cb 0x55da22785c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.380+0000 7f7928e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b2ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.380+0000 7f7923400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b28f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.380+0000 7f7923e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.380+0000 7f7923e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b289d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60460/0 (socket says 172.21.15.57:60460) 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.380+0000 7f7923e00640 1 -- 172.21.15.57:0/1550533987 learned_addr learned my addr 172.21.15.57:0/1550533987 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.381+0000 7f7923e00640 1 -- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 msgr2=0x55da22b2ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.381+0000 7f7923e00640 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b2ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.381+0000 7f7923e00640 1 -- 172.21.15.57:0/1550533987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 msgr2=0x55da22b28f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.381+0000 7f7923e00640 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b28f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.381+0000 7f7923e00640 1 -- 172.21.15.57:0/1550533987 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2284e7e0 con 0x55da22b1e430 2024-04-13T05:23:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.381+0000 7f7923400640 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b28f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.382+0000 7f790e000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:23:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.382+0000 7f7923e00640 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b289d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79140ccde0 tx=0x7f791409b1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7de93088bcf0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.382+0000 7f790e000640 1 -- 172.21.15.57:0/1550533987 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791409f970 con 0x55da22b1e430 2024-04-13T05:23:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.382+0000 7f790e000640 1 -- 172.21.15.57:0/1550533987 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7914097c30 con 0x55da22b1e430 2024-04-13T05:23:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.383+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da22a815e0 con 0x55da22b1e430 2024-04-13T05:23:04.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.383+0000 7f790e000640 1 -- 172.21.15.57:0/1550533987 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79140a4230 con 0x55da22b1e430 2024-04-13T05:23:04.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.383+0000 7f792dd70740 10 client.25912 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:04.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.384+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da228aef70 con 0x55da22b1e430 2024-04-13T05:23:04.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.384+0000 7f790e000640 1 -- 172.21.15.57:0/1550533987 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55da22a815e0 con 0x55da22b1e430 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f790e000640 1 -- 172.21.15.57:0/1550533987 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7914097310 con 0x55da22b1e430 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f792dd70740 10 client.25912 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f792dd70740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f792dd70740 20 client.25912 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f792dd70740 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b347d0 0x55da22b36c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f792dd70740 4 client.25912 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:04.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da22a7b450 con 0x55da22b347d0 2024-04-13T05:23:04.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.385+0000 7f7923400640 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b347d0 0x55da22b36c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.386+0000 7f7923400640 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b347d0 0x55da22b36c50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a87f40d1d3730b server_cookie=a45feb6c9935071 in_seq=0 out_seq=0 2024-04-13T05:23:04.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.386+0000 7f790e000640 10 client.25912 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f790e000640 1 -- 172.21.15.57:0/1550533987 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da22a7b450 con 0x55da22b347d0 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f790e000640 10 client.25912 handle_command_reply: tid=0 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 client.25912 shutdown 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 msgr2=0x55da22b289d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b289d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79140ccde0 tx=0x7f791409b1c0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 shutdown_connections 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b347d0 0x55da22b36c50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f790e000640 0 client.25912 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b2ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b28f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 --2- 172.21.15.57:0/1550533987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b289d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 >> 172.21.15.57:0/1550533987 conn(0x55da22b08990 msgr2=0x55da22b07b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.388+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 shutdown_connections 2024-04-13T05:23:04.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.389+0000 7f792dd70740 1 -- 172.21.15.57:0/1550533987 wait complete. 2024-04-13T05:23:04.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.389+0000 7f792dd70740 20 client.25912 trim_cache size 0 max 16384 2024-04-13T05:23:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.422+0000 7f792dd70740 1 Processor -- start 2024-04-13T05:23:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.422+0000 7f792dd70740 1 -- start start 2024-04-13T05:23:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.423+0000 7f792dd70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b28cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.423+0000 7f792dd70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b2ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.423+0000 7f792dd70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b07300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.423+0000 7f792dd70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7904012570 con 0x55da22b1ee20 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.423+0000 7f792dd70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f790400fde0 con 0x55da22b23f90 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.423+0000 7f792dd70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f790400dc30 con 0x55da22b1e430 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.424+0000 7f7923400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b2ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.424+0000 7f7928e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b07300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.424+0000 7f7923400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b2ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42688/0 (socket says 172.21.15.57:42688) 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.424+0000 7f7928e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b07300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40224/0 (socket says 172.21.15.57:40224) 2024-04-13T05:23:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.424+0000 7f7928e00640 1 -- 172.21.15.57:0/618158607 learned_addr learned my addr 172.21.15.57:0/618158607 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.424+0000 7f7923e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b28cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.425+0000 7f7923400640 1 -- 172.21.15.57:0/618158607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 msgr2=0x55da22b28cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.425+0000 7f7923400640 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b28cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.425+0000 7f7923400640 1 -- 172.21.15.57:0/618158607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 msgr2=0x55da22b07300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.425+0000 7f7923400640 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b07300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.425+0000 7f7923400640 1 -- 172.21.15.57:0/618158607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da22746170 con 0x55da22b1ee20 2024-04-13T05:23:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.425+0000 7f7923400640 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b2ff10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f791000bc60 tx=0x7f7910013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d91048c21415139 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.426+0000 7f7918c00640 1 -- 172.21.15.57:0/618158607 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910004020 con 0x55da22b1ee20 2024-04-13T05:23:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.426+0000 7f7918c00640 1 -- 172.21.15.57:0/618158607 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79100041c0 con 0x55da22b1ee20 2024-04-13T05:23:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.426+0000 7f7918c00640 1 -- 172.21.15.57:0/618158607 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910003850 con 0x55da22b1ee20 2024-04-13T05:23:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.427+0000 7f792dd70740 1 -- 172.21.15.57:0/618158607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 msgr2=0x55da22b2ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.427+0000 7f792dd70740 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b2ff10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f791000bc60 tx=0x7f7910013040 comp rx=0 tx=0).stop 2024-04-13T05:23:04.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.427+0000 7f792dd70740 1 -- 172.21.15.57:0/618158607 shutdown_connections 2024-04-13T05:23:04.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.427+0000 7f792dd70740 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b23f90 0x55da22b07300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.427+0000 7f792dd70740 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b1ee20 0x55da22b2ff10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.428+0000 7f792dd70740 1 --2- 172.21.15.57:0/618158607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1e430 0x55da22b28cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.428+0000 7f792dd70740 1 -- 172.21.15.57:0/618158607 >> 172.21.15.57:0/618158607 conn(0x55da22b08990 msgr2=0x55da22b285e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.428+0000 7f792dd70740 1 -- 172.21.15.57:0/618158607 shutdown_connections 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.428+0000 7f792dd70740 1 -- 172.21.15.57:0/618158607 wait complete. 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.428+0000 7f792dd70740 1 -- start start 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.428+0000 7f792dd70740 1 Processor -- start 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b1e430 0x55da22b2d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b2d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b23f90 0x55da22b2e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f790400e2a0 con 0x55da22b23f90 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f790400f710 con 0x55da22b1e430 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f790400e970 con 0x55da22b1ee20 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.429+0000 7f792dd70740 10 client.0 _ll_register_callbacks cb 0x7f7914005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7923e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b1e430 0x55da22b2d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7928e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b23f90 0x55da22b2e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7923e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b1e430 0x55da22b2d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40238/0 (socket says 172.21.15.57:40238) 2024-04-13T05:23:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7923e00640 1 -- 172.21.15.57:0/1833553116 learned_addr learned my addr 172.21.15.57:0/1833553116 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7923400640 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b2d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7928e00640 1 -- 172.21.15.57:0/1833553116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 msgr2=0x55da22b2d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7928e00640 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b2d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7928e00640 1 -- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b1e430 msgr2=0x55da22b2d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7928e00640 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b1e430 0x55da22b2d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.430+0000 7f7928e00640 1 -- 172.21.15.57:0/1833553116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da22a2ebe0 con 0x55da22b23f90 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.431+0000 7f7928e00640 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b23f90 0x55da22b2e890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f791c02c730 tx=0x7f791c02c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63b73ecb984e3b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.431+0000 7f790e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.431+0000 7f790e000640 1 -- 172.21.15.57:0/1833553116 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c050030 con 0x55da22b23f90 2024-04-13T05:23:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.431+0000 7f790e000640 1 -- 172.21.15.57:0/1833553116 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f791c03fbd0 con 0x55da22b23f90 2024-04-13T05:23:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.431+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79100041c0 con 0x55da22b23f90 2024-04-13T05:23:04.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f790e000640 1 -- 172.21.15.57:0/1833553116 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f791c05d050 con 0x55da22b23f90 2024-04-13T05:23:04.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f792dd70740 10 client.16593 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:04.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da22b30060 con 0x55da22b23f90 2024-04-13T05:23:04.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f790e000640 1 -- 172.21.15.57:0/1833553116 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79100041c0 con 0x55da22b23f90 2024-04-13T05:23:04.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f790e000640 1 -- 172.21.15.57:0/1833553116 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f791c049050 con 0x55da22b23f90 2024-04-13T05:23:04.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f792dd70740 10 client.16593 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f792dd70740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f792dd70740 20 client.16593 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.432+0000 7f792dd70740 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b9b480 0x55da22b9d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.433+0000 7f792dd70740 4 client.16593 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.433+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da229f7a70 con 0x55da22b9b480 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.433+0000 7f7923e00640 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b9b480 0x55da22b9d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.434+0000 7f7923e00640 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b9b480 0x55da22b9d8a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d8275faca23c48 server_cookie=b90c1a142edad26e in_seq=0 out_seq=0 2024-04-13T05:23:04.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.434+0000 7f790e000640 10 client.16593 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.436+0000 7f790e000640 1 -- 172.21.15.57:0/1833553116 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da229f7a70 con 0x55da22b9b480 2024-04-13T05:23:04.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.436+0000 7f790e000640 10 client.16593 handle_command_reply: tid=0 2024-04-13T05:23:04.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.437+0000 7f792dd70740 1 client.16593 shutdown 2024-04-13T05:23:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b23f90 msgr2=0x55da22b2e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b23f90 0x55da22b2e890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f791c02c730 tx=0x7f791c02c120 comp rx=0 tx=0).stop 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 shutdown_connections 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55da22b9b480 0x55da22b9d8a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da22b23f90 0x55da22b2e890 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da22b1ee20 0x55da22b2d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 --2- 172.21.15.57:0/1833553116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da22b1e430 0x55da22b2d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 >> 172.21.15.57:0/1833553116 conn(0x55da22b08990 msgr2=0x55da22b28de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f790e000640 0 client.16593 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.438+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 shutdown_connections 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.439+0000 7f792dd70740 1 -- 172.21.15.57:0/1833553116 wait complete. 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.439+0000 7f792dd70740 20 client.16593 trim_cache size 0 max 16384 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "71c393fb-e4f3-4fec-994d-2589d99efdd9", 2024-04-13T05:23:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:23:04.443 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:23:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79080a3c00 msgr2=0x7f79080a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79080a3c00 0x7f79080a60c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55da22b1f680 tx=0x7f791c02e040 comp rx=0 tx=0).stop 2024-04-13T05:23:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 msgr2=0x7f792419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 0x7f792419e770 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f791400d950 tx=0x7f791400de10 comp rx=0 tx=0).stop 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 shutdown_connections 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79080a3c00 0x7f79080a60c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f792410a0b0 0x7f79241a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79241096f0 0x7f792419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 --2- 172.21.15.57:0/2589524109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7924108500 0x7f792419e770 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 >> 172.21.15.57:0/2589524109 conn(0x7f7924077490 msgr2=0x7f792410ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 shutdown_connections 2024-04-13T05:23:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.441+0000 7f7929800640 1 -- 172.21.15.57:0/2589524109 wait complete. 2024-04-13T05:23:04.464 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:04 smithi057 ceph-mon[32331]: pgmap v620: 129 pgs: 129 active+clean; 290 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-04-13T05:23:04.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:23:04.467 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:23:04.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:04 smithi083 ceph-mon[35133]: pgmap v620: 129 pgs: 129 active+clean; 290 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-04-13T05:23:04.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.751+0000 7fac0bc00640 1 -- 172.21.15.57:0/2593849203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04105410 msgr2=0x7fac04105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.751+0000 7fac0bc00640 1 --2- 172.21.15.57:0/2593849203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04105410 0x7fac04105890 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009ca0 tx=0x7fabfc02c100 comp rx=0 tx=0).stop 2024-04-13T05:23:04.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 -- 172.21.15.57:0/2593849203 shutdown_connections 2024-04-13T05:23:04.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 --2- 172.21.15.57:0/2593849203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac04105dd0 0x7fac0410c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 --2- 172.21.15.57:0/2593849203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04105410 0x7fac04105890 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 --2- 172.21.15.57:0/2593849203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04104220 0x7fac04104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 -- 172.21.15.57:0/2593849203 >> 172.21.15.57:0/2593849203 conn(0x7fac040ff7b0 msgr2=0x7fac04101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 -- 172.21.15.57:0/2593849203 shutdown_connections 2024-04-13T05:23:04.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.752+0000 7fac0bc00640 1 -- 172.21.15.57:0/2593849203 wait complete. 2024-04-13T05:23:04.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.753+0000 7fac0bc00640 1 Processor -- start 2024-04-13T05:23:04.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 -- start start 2024-04-13T05:23:04.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 0x7fac0419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04105dd0 0x7fac0419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0419fc00 0x7fac041a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410fd60 con 0x7fac04104220 2024-04-13T05:23:04.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410fbe0 con 0x7fac0419fc00 2024-04-13T05:23:04.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac0bc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410fee0 con 0x7fac04105dd0 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac09200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 0x7fac0419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac09200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 0x7fac0419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42712/0 (socket says 172.21.15.57:42712) 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.754+0000 7fac09200640 1 -- 172.21.15.57:0/1055121382 learned_addr learned my addr 172.21.15.57:0/1055121382 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac03e00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04105dd0 0x7fac0419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac09c00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0419fc00 0x7fac041a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac03e00640 1 -- 172.21.15.57:0/1055121382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0419fc00 msgr2=0x7fac041a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac03e00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0419fc00 0x7fac041a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac03e00640 1 -- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 msgr2=0x7fac0419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac03e00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 0x7fac0419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac03e00640 1 -- 172.21.15.57:0/1055121382 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac041a2660 con 0x7fac04105dd0 2024-04-13T05:23:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.755+0000 7fac09200640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 0x7fac0419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.756+0000 7fac03e00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04105dd0 0x7fac0419aba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabfc008000 tx=0x7fabfc035d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.757+0000 7fac01600640 1 -- 172.21.15.57:0/1055121382 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc039040 con 0x7fac04105dd0 2024-04-13T05:23:04.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.757+0000 7fac01600640 1 -- 172.21.15.57:0/1055121382 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabfc034700 con 0x7fac04105dd0 2024-04-13T05:23:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.757+0000 7fac01600640 1 -- 172.21.15.57:0/1055121382 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0042a0 con 0x7fac04105dd0 2024-04-13T05:23:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.757+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac041a28f0 con 0x7fac04105dd0 2024-04-13T05:23:04.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.757+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac041a2d60 con 0x7fac04105dd0 2024-04-13T05:23:04.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.763+0000 7fac0ebe1740 1 Processor -- start 2024-04-13T05:23:04.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.763+0000 7fac0ebe1740 1 -- start start 2024-04-13T05:23:04.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.764+0000 7fac01600640 1 -- 172.21.15.57:0/1055121382 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fabfc046030 con 0x7fac04105dd0 2024-04-13T05:23:04.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.764+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a91310b020 con 0x55a9131ad6d0 2024-04-13T05:23:04.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9130b16d0 con 0x55a9131acce0 2024-04-13T05:23:04.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a913106fe0 con 0x55a9131b2840 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac09c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac09c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40268/0 (socket says 172.21.15.57:40268) 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.765+0000 7fac09c00640 1 -- 172.21.15.57:0/3618847840 learned_addr learned my addr 172.21.15.57:0/3618847840 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09200640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac03e00640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09c00640 1 -- 172.21.15.57:0/3618847840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 msgr2=0x55a9131b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09c00640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09c00640 1 -- 172.21.15.57:0/3618847840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 msgr2=0x55a9131b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09c00640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09c00640 1 -- 172.21.15.57:0/3618847840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a912ee77d0 con 0x55a9131acce0 2024-04-13T05:23:04.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac09200640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:04.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.766+0000 7fac03e00640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:04.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.767+0000 7fac09c00640 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131ad100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009c10 tx=0x7fabf00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e32271e64844af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.768+0000 7fabfa000640 1 -- 172.21.15.57:0/3618847840 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0012450 con 0x55a9131acce0 2024-04-13T05:23:04.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.768+0000 7fac01600640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabe80a3d00 0x7fabe80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.768+0000 7fabfa000640 1 -- 172.21.15.57:0/3618847840 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabf0012aa0 con 0x55a9131acce0 2024-04-13T05:23:04.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.769+0000 7fabfa000640 1 -- 172.21.15.57:0/3618847840 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000fac0 con 0x55a9131acce0 2024-04-13T05:23:04.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.769+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3618847840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 msgr2=0x55a9131ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.769+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131ad100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009c10 tx=0x7fabf00117b0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac01600640 1 -- 172.21.15.57:0/1055121382 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fabfc0f2000 con 0x7fac04105dd0 2024-04-13T05:23:04.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac09200640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabe80a3d00 0x7fabe80a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3618847840 shutdown_connections 2024-04-13T05:23:04.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3618847840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131ad100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.770+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3618847840 >> 172.21.15.57:0/3618847840 conn(0x55a91319fa40 msgr2=0x55a91319fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.771+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3618847840 shutdown_connections 2024-04-13T05:23:04.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.771+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3618847840 wait complete. 2024-04-13T05:23:04.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.771+0000 7fac09200640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabe80a3d00 0x7fabe80a61c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a9131a09b0 tx=0x7fabf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.772+0000 7fac0ebe1740 1 -- start start 2024-04-13T05:23:04.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.772+0000 7fac0ebe1740 1 Processor -- start 2024-04-13T05:23:04.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131acce0 0x55a9131b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131ad6d0 0x55a9131b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131be4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a9131b5840 con 0x55a9131acce0 2024-04-13T05:23:04.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9130c4a90 con 0x55a9131ad6d0 2024-04-13T05:23:04.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a9131b59c0 con 0x55a9131b2840 2024-04-13T05:23:04.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac0ebe1740 10 client.0 _ll_register_callbacks cb 0x55a9130c4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:04.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.773+0000 7fac09200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131acce0 0x55a9131b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.774+0000 7fac09200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131acce0 0x55a9131b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42732/0 (socket says 172.21.15.57:42732) 2024-04-13T05:23:04.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.774+0000 7fac09200640 1 -- 172.21.15.57:0/3879088866 learned_addr learned my addr 172.21.15.57:0/3879088866 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.774+0000 7fac03e00640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131ad6d0 0x55a9131b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.774+0000 7fac09c00640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131be4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.775+0000 7fac09c00640 1 -- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131ad6d0 msgr2=0x55a9131b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.775+0000 7fac09c00640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131ad6d0 0x55a9131b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.775+0000 7fac09c00640 1 -- 172.21.15.57:0/3879088866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131acce0 msgr2=0x55a9131b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.775+0000 7fac09c00640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131acce0 0x55a9131b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.775+0000 7fac09c00640 1 -- 172.21.15.57:0/3879088866 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a912e1d3f0 con 0x55a9131b2840 2024-04-13T05:23:04.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.775+0000 7fac09c00640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131be4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fabf000f480 tx=0x7fabf0012840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365da21f21077506 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.776+0000 7fabeea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.776+0000 7fabeea00640 1 -- 172.21.15.57:0/3879088866 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0029030 con 0x55a9131b2840 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.776+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf0012aa0 con 0x55a9131b2840 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.776+0000 7fabeea00640 1 -- 172.21.15.57:0/3879088866 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fabf0020040 con 0x55a9131b2840 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.777+0000 7fabeea00640 1 -- 172.21.15.57:0/3879088866 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabf0033050 con 0x55a9131b2840 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.777+0000 7fac0ebe1740 10 client.25927 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:04.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.777+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a912ddd0f0 con 0x55a9131b2840 2024-04-13T05:23:04.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.777+0000 7fabeea00640 1 -- 172.21.15.57:0/3879088866 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabf0012aa0 con 0x55a9131b2840 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fabeea00640 1 -- 172.21.15.57:0/3879088866 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fabf00050b0 con 0x55a9131b2840 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fac0ebe1740 10 client.25927 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fac0ebe1740 10 client.25927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fac0ebe1740 20 client.25927 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a9131c2ab0 0x55a9131c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fac0ebe1740 4 client.25927 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.778+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9130aa600 con 0x55a9131c2ab0 2024-04-13T05:23:04.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.779+0000 7fac09200640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a9131c2ab0 0x55a9131c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.780+0000 7fac09200640 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a9131c2ab0 0x55a9131c4ea0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0fd8852467297 server_cookie=a020b182044b8705 in_seq=0 out_seq=0 2024-04-13T05:23:04.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.780+0000 7fabeea00640 10 client.25927 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.783+0000 7fabeea00640 1 -- 172.21.15.57:0/3879088866 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9130aa600 con 0x55a9131c2ab0 2024-04-13T05:23:04.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.783+0000 7fabeea00640 10 client.25927 handle_command_reply: tid=0 2024-04-13T05:23:04.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.783+0000 7fac0ebe1740 1 client.25927 shutdown 2024-04-13T05:23:04.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.784+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 msgr2=0x55a9131be4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.784+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131be4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fabf000f480 tx=0x7fabf0012840 comp rx=0 tx=0).stop 2024-04-13T05:23:04.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.784+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 shutdown_connections 2024-04-13T05:23:04.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.784+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a9131c2ab0 0x55a9131c4ea0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.784+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131ad6d0 0x55a9131b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131b2840 0x55a9131be4a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/3879088866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131acce0 0x55a9131b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 >> 172.21.15.57:0/3879088866 conn(0x55a91319fa40 msgr2=0x55a91319eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fabeea00640 0 client.25927 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 shutdown_connections 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fac0ebe1740 1 -- 172.21.15.57:0/3879088866 wait complete. 2024-04-13T05:23:04.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.785+0000 7fac0ebe1740 20 client.25927 trim_cache size 0 max 16384 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 Processor -- start 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 -- start start 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131acce0 0x55a91319e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131b2840 0x55a91319e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fac041ab6d0 con 0x55a9131ad6d0 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac041a8f40 con 0x55a9131b2840 2024-04-13T05:23:04.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.817+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac041a6db0 con 0x55a9131acce0 2024-04-13T05:23:04.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.818+0000 7fac03e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.818+0000 7fac03e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131be3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42736/0 (socket says 172.21.15.57:42736) 2024-04-13T05:23:04.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.818+0000 7fac03e00640 1 -- 172.21.15.57:0/178276018 learned_addr learned my addr 172.21.15.57:0/178276018 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.818+0000 7fac09200640 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131acce0 0x55a91319e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.818+0000 7fac03e00640 1 -- 172.21.15.57:0/178276018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131acce0 msgr2=0x55a91319e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.818+0000 7fac03e00640 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131acce0 0x55a91319e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.819+0000 7fac03e00640 1 -- 172.21.15.57:0/178276018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131b2840 msgr2=0x55a91319e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.819+0000 7fac03e00640 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131b2840 0x55a91319e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.819+0000 7fac03e00640 1 -- 172.21.15.57:0/178276018 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9131ad470 con 0x55a9131ad6d0 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.819+0000 7fac03e00640 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131be3c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009430 tx=0x7fabfc02c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e2f22add86c01b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.820+0000 7fabf9600640 1 -- 172.21.15.57:0/178276018 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0d7070 con 0x55a9131ad6d0 2024-04-13T05:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.820+0000 7fabf9600640 1 -- 172.21.15.57:0/178276018 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabfc0098b0 con 0x55a9131ad6d0 2024-04-13T05:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.820+0000 7fabf9600640 1 -- 172.21.15.57:0/178276018 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc003a00 con 0x55a9131ad6d0 2024-04-13T05:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 -- 172.21.15.57:0/178276018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 msgr2=0x55a9131be3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131be3c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009430 tx=0x7fabfc02c100 comp rx=0 tx=0).stop 2024-04-13T05:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 -- 172.21.15.57:0/178276018 shutdown_connections 2024-04-13T05:23:04.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131b2840 0x55a91319e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131ad6d0 0x55a9131be3c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/178276018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131acce0 0x55a91319e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.821+0000 7fac0ebe1740 1 -- 172.21.15.57:0/178276018 >> 172.21.15.57:0/178276018 conn(0x55a91319fa40 msgr2=0x55a9131b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.822+0000 7fac0ebe1740 1 -- 172.21.15.57:0/178276018 shutdown_connections 2024-04-13T05:23:04.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.822+0000 7fac0ebe1740 1 -- 172.21.15.57:0/178276018 wait complete. 2024-04-13T05:23:04.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.822+0000 7fac0ebe1740 1 -- start start 2024-04-13T05:23:04.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.822+0000 7fac0ebe1740 1 Processor -- start 2024-04-13T05:23:04.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131bbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131ad6d0 0x55a9131bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131b2840 0x55a9131bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fac041a7400 con 0x55a9131b2840 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac041a8870 con 0x55a9131acce0 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac041a7ad0 con 0x55a9131ad6d0 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.823+0000 7fac0ebe1740 10 client.0 _ll_register_callbacks cb 0x7fabf0012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131bbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131bbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:40300/0 (socket says 172.21.15.57:40300) 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 -- 172.21.15.57:0/1093771200 learned_addr learned my addr 172.21.15.57:0/1093771200 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac03e00640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131ad6d0 0x55a9131bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09c00640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131b2840 0x55a9131bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 -- 172.21.15.57:0/1093771200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131ad6d0 msgr2=0x55a9131bc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131ad6d0 0x55a9131bc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 -- 172.21.15.57:0/1093771200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131b2840 msgr2=0x55a9131bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131b2840 0x55a9131bd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.824+0000 7fac09200640 1 -- 172.21.15.57:0/1093771200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9131bdaa0 con 0x55a9131acce0 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.825+0000 7fac09200640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131bbf60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55a9131c56e0 tx=0x7fabf40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e388a258b1b216 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.825+0000 7fabeea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.825+0000 7fac03e00640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131ad6d0 0x55a9131bc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:23:04.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.825+0000 7fac09c00640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131b2840 0x55a9131bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:23:04.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.828+0000 7fabeea00640 1 -- 172.21.15.57:0/1093771200 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4031060 con 0x55a9131acce0 2024-04-13T05:23:04.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.828+0000 7fabeea00640 1 -- 172.21.15.57:0/1093771200 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fabf402c430 con 0x55a9131acce0 2024-04-13T05:23:04.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.828+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabfc0098b0 con 0x55a9131acce0 2024-04-13T05:23:04.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:04 smithi189 ceph-mon[35531]: pgmap v620: 129 pgs: 129 active+clean; 290 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-04-13T05:23:04.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.830+0000 7fabeea00640 1 -- 172.21.15.57:0/1093771200 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabf402c630 con 0x55a9131acce0 2024-04-13T05:23:04.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.831+0000 7fac0ebe1740 10 client.36017 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:04.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.831+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a913229500 con 0x55a9131acce0 2024-04-13T05:23:04.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.831+0000 7fabeea00640 1 -- 172.21.15.57:0/1093771200 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabfc0098b0 con 0x55a9131acce0 2024-04-13T05:23:04.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fabeea00640 1 -- 172.21.15.57:0/1093771200 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fabf402a4d0 con 0x55a9131acce0 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fac0ebe1740 10 client.36017 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fac0ebe1740 10 client.36017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fac0ebe1740 20 client.36017 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a913229e40 0x55a91322c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fac0ebe1740 4 client.36017 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.832+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9131b6d10 con 0x55a913229e40 2024-04-13T05:23:04.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.833+0000 7fac03e00640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a913229e40 0x55a91322c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:04.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.834+0000 7fabeea00640 10 client.36017 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.834+0000 7fac03e00640 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a913229e40 0x55a91322c260 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95461d8a07648c8d server_cookie=12c4ceb422aa30ef in_seq=0 out_seq=0 2024-04-13T05:23:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.836+0000 7fabeea00640 1 -- 172.21.15.57:0/1093771200 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55a9131b6d10 con 0x55a913229e40 2024-04-13T05:23:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.836+0000 7fabeea00640 10 client.36017 handle_command_reply: tid=0 2024-04-13T05:23:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.836+0000 7fac0ebe1740 1 client.36017 shutdown 2024-04-13T05:23:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 msgr2=0x55a9131bbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131bbf60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55a9131c56e0 tx=0x7fabf40046c0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 shutdown_connections 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a913229e40 0x55a91322c260 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a9131b2840 0x55a9131bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a9131ad6d0 0x55a9131bc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fabeea00640 0 client.36017 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 --2- 172.21.15.57:0/1093771200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9131acce0 0x55a9131bbf60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 >> 172.21.15.57:0/1093771200 conn(0x55a91319fa40 msgr2=0x55a9131a2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 shutdown_connections 2024-04-13T05:23:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 1 -- 172.21.15.57:0/1093771200 wait complete. 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.837+0000 7fac0ebe1740 20 client.36017 trim_cache size 0 max 16384 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1049 inodes in the stack)", 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "71c393fb-e4f3-4fec-994d-2589d99efdd9": { 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "71c393fb-e4f3-4fec-994d-2589d99efdd9", 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:23:04.841 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:23:04.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabe80a3d00 msgr2=0x7fabe80a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabe80a3d00 0x7fabe80a61c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a9131a09b0 tx=0x7fabf4005f90 comp rx=0 tx=0).stop 2024-04-13T05:23:04.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04105dd0 msgr2=0x7fac0419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:04.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04105dd0 0x7fac0419aba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabfc008000 tx=0x7fabfc035d60 comp rx=0 tx=0).stop 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 shutdown_connections 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fabe80a3d00 0x7fabe80a61c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0419fc00 0x7fac041a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac04105dd0 0x7fac0419aba0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabfc008000 tx=0x7fabfc035d60 comp rx=0 tx=0).stop 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 --2- 172.21.15.57:0/1055121382 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac04104220 0x7fac0419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 >> 172.21.15.57:0/1055121382 conn(0x7fac040ff7b0 msgr2=0x7fac04108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 shutdown_connections 2024-04-13T05:23:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:04.839+0000 7fac0bc00640 1 -- 172.21.15.57:0/1055121382 wait complete. 2024-04-13T05:23:04.872 INFO:tasks.cephfs.filesystem:scrub status for tag:71c393fb-e4f3-4fec-994d-2589d99efdd9 - {'path': '/', 'tag': '71c393fb-e4f3-4fec-994d-2589d99efdd9', 'options': 'recursive,force'} 2024-04-13T05:23:05.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:05.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:05 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:05 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:05.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:05 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:06 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:06 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:06 smithi083 ceph-mon[35133]: pgmap v621: 129 pgs: 129 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-04-13T05:23:06.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:06 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:06 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:06 smithi057 ceph-mon[32331]: pgmap v621: 129 pgs: 129 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-04-13T05:23:06.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:06 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:06 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:06.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:06 smithi189 ceph-mon[35531]: pgmap v621: 129 pgs: 129 active+clean; 296 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 136 op/s 2024-04-13T05:23:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:08 smithi083 ceph-mon[35133]: pgmap v622: 129 pgs: 129 active+clean; 297 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-04-13T05:23:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:08 smithi057 ceph-mon[32331]: pgmap v622: 129 pgs: 129 active+clean; 297 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-04-13T05:23:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:08 smithi189 ceph-mon[35531]: pgmap v622: 129 pgs: 129 active+clean; 297 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-04-13T05:23:09.637 INFO:tasks.workunit.client.0.smithi057.stdout: 47 7702 133 5152 17 3013 85 2024-04-13T05:23:11.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:10 smithi083 ceph-mon[35133]: pgmap v623: 129 pgs: 129 active+clean; 302 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:23:11.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:10 smithi057 ceph-mon[32331]: pgmap v623: 129 pgs: 129 active+clean; 302 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:23:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:10 smithi189 ceph-mon[35531]: pgmap v623: 129 pgs: 129 active+clean; 302 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:23:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:12 smithi083 ceph-mon[35133]: pgmap v624: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:23:13.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:12 smithi057 ceph-mon[32331]: pgmap v624: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:23:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:12 smithi189 ceph-mon[35531]: pgmap v624: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:23:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:14 smithi083 ceph-mon[35133]: pgmap v625: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-04-13T05:23:15.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:14 smithi057 ceph-mon[32331]: pgmap v625: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-04-13T05:23:15.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:14 smithi189 ceph-mon[35531]: pgmap v625: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-04-13T05:23:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:16 smithi083 ceph-mon[35133]: pgmap v626: 129 pgs: 129 active+clean; 310 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-04-13T05:23:17.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:16 smithi057 ceph-mon[32331]: pgmap v626: 129 pgs: 129 active+clean; 310 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-04-13T05:23:17.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:16 smithi189 ceph-mon[35531]: pgmap v626: 129 pgs: 129 active+clean; 310 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-04-13T05:23:18.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:17 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:23:18.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:17 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:23:18.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:17 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:23:18.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:17 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:23:18.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:17 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:23:18.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:17 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:23:18.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:17 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:23:18.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:17 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:23:18.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:17 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:23:19.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:18 smithi083 ceph-mon[35133]: pgmap v627: 129 pgs: 129 active+clean; 310 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-04-13T05:23:19.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:18 smithi057 ceph-mon[32331]: pgmap v627: 129 pgs: 129 active+clean; 310 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-04-13T05:23:19.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:18 smithi189 ceph-mon[35531]: pgmap v627: 129 pgs: 129 active+clean; 310 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-04-13T05:23:19.637 INFO:tasks.workunit.client.0.smithi057.stdout: 49 8751 145 5352 37 2565 91 2024-04-13T05:23:21.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:20 smithi083 ceph-mon[35133]: pgmap v628: 129 pgs: 129 active+clean; 318 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-04-13T05:23:21.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:20 smithi057 ceph-mon[32331]: pgmap v628: 129 pgs: 129 active+clean; 318 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-04-13T05:23:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:20 smithi189 ceph-mon[35531]: pgmap v628: 129 pgs: 129 active+clean; 318 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-04-13T05:23:23.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:22 smithi083 ceph-mon[35133]: pgmap v629: 129 pgs: 129 active+clean; 319 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-04-13T05:23:23.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:22 smithi057 ceph-mon[32331]: pgmap v629: 129 pgs: 129 active+clean; 319 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-04-13T05:23:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:22 smithi189 ceph-mon[35531]: pgmap v629: 129 pgs: 129 active+clean; 319 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-04-13T05:23:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:24 smithi083 ceph-mon[35133]: pgmap v630: 129 pgs: 129 active+clean; 323 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2024-04-13T05:23:25.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:24 smithi057 ceph-mon[32331]: pgmap v630: 129 pgs: 129 active+clean; 323 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2024-04-13T05:23:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:24 smithi189 ceph-mon[35531]: pgmap v630: 129 pgs: 129 active+clean; 323 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2024-04-13T05:23:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:26 smithi083 ceph-mon[35133]: pgmap v631: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-13T05:23:27.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:26 smithi057 ceph-mon[32331]: pgmap v631: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-13T05:23:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:26 smithi189 ceph-mon[35531]: pgmap v631: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-13T05:23:29.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:28 smithi083 ceph-mon[35133]: pgmap v632: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-04-13T05:23:29.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:28 smithi057 ceph-mon[32331]: pgmap v632: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-04-13T05:23:29.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:28 smithi189 ceph-mon[35531]: pgmap v632: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-04-13T05:23:29.638 INFO:tasks.workunit.client.0.smithi057.stdout: 49 8043 79 5049 61 2753 94 2024-04-13T05:23:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:30 smithi083 ceph-mon[35133]: pgmap v633: 129 pgs: 129 active+clean; 333 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-04-13T05:23:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:30 smithi057 ceph-mon[32331]: pgmap v633: 129 pgs: 129 active+clean; 333 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-04-13T05:23:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:30 smithi189 ceph-mon[35531]: pgmap v633: 129 pgs: 129 active+clean; 333 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-04-13T05:23:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:32 smithi083 ceph-mon[35133]: pgmap v634: 129 pgs: 129 active+clean; 334 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-13T05:23:33.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:32 smithi057 ceph-mon[32331]: pgmap v634: 129 pgs: 129 active+clean; 334 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-13T05:23:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:32 smithi189 ceph-mon[35531]: pgmap v634: 129 pgs: 129 active+clean; 334 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-13T05:23:34.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:23:34.875 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.135+0000 7fdbe5e00640 1 -- 172.21.15.57:0/3608889803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 msgr2=0x7fdbe0101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.135+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/3608889803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe0101f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009be0 tx=0x7fdbd802bef0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 -- 172.21.15.57:0/3608889803 shutdown_connections 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/3608889803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe0101f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/3608889803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe0101100 0x7fdbe0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/3608889803 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe01007a0 0x7fdbe0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 -- 172.21.15.57:0/3608889803 >> 172.21.15.57:0/3608889803 conn(0x7fdbe00fc360 msgr2=0x7fdbe00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 -- 172.21.15.57:0/3608889803 shutdown_connections 2024-04-13T05:23:35.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.136+0000 7fdbe5e00640 1 -- 172.21.15.57:0/3608889803 wait complete. 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.137+0000 7fdbe5e00640 1 Processor -- start 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.137+0000 7fdbe5e00640 1 -- start start 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.137+0000 7fdbe5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe01007a0 0x7fdbe019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0101100 0x7fdbe019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fc20 con 0x7fdbe0101100 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010faa0 con 0x7fdbe0101aa0 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fda0 con 0x7fdbe01007a0 2024-04-13T05:23:35.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0101100 0x7fdbe019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe01a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45498/0 (socket says 172.21.15.57:45498) 2024-04-13T05:23:35.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe5400640 1 -- 172.21.15.57:0/729637005 learned_addr learned my addr 172.21.15.57:0/729637005 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:35.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.138+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe01007a0 0x7fdbe019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.139+0000 7fdbe5400640 1 -- 172.21.15.57:0/729637005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe01007a0 msgr2=0x7fdbe019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.139+0000 7fdbe5400640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe01007a0 0x7fdbe019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.139+0000 7fdbe5400640 1 -- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0101100 msgr2=0x7fdbe019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.139+0000 7fdbe5400640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0101100 0x7fdbe019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.139+0000 7fdbe5400640 1 -- 172.21.15.57:0/729637005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe01a1eb0 con 0x7fdbe0101aa0 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.139+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0101100 0x7fdbe019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.140+0000 7fdbe5400640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe01a1840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8034fd0 tx=0x7fdbd8033c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.141+0000 7fdbdd600640 1 -- 172.21.15.57:0/729637005 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8043020 con 0x7fdbe0101aa0 2024-04-13T05:23:35.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.141+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe01a2140 con 0x7fdbe0101aa0 2024-04-13T05:23:35.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.141+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe01a26e0 con 0x7fdbe0101aa0 2024-04-13T05:23:35.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.141+0000 7fdbdd600640 1 -- 172.21.15.57:0/729637005 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbd8003e20 con 0x7fdbe0101aa0 2024-04-13T05:23:35.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.141+0000 7fdbdd600640 1 -- 172.21.15.57:0/729637005 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8034430 con 0x7fdbe0101aa0 2024-04-13T05:23:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.144+0000 7fdbea37a740 1 Processor -- start 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.146+0000 7fdbdd600640 1 -- 172.21.15.57:0/729637005 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdbd8046020 con 0x7fdbe0101aa0 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.148+0000 7fdbdd600640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc80a3c00 0x7fdbc80a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.149+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc80a3c00 0x7fdbc80a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.149+0000 7fdbdd600640 1 -- 172.21.15.57:0/729637005 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdbd80bc910 con 0x7fdbe0101aa0 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.149+0000 7fdbea37a740 1 -- start start 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.149+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d144495320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.149+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.149+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d14449aa60 0x55d14449cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbea37a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d1443f3020 con 0x55d1444958f0 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbea37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1443996d0 con 0x55d144494f00 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbea37a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d1443eefe0 con 0x55d14449aa60 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc80a3c00 0x7fdbc80a60c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdbe019b7a0 tx=0x7fdbcc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbe4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d144495320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbe4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d144495320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45508/0 (socket says 172.21.15.57:45508) 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbe4a00640 1 -- 172.21.15.57:0/895374738 learned_addr learned my addr 172.21.15.57:0/895374738 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.150+0000 7fdbe5400640 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d14449aa60 0x55d14449cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbe4a00640 1 -- 172.21.15.57:0/895374738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d14449aa60 msgr2=0x55d14449cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d14449aa60 0x55d14449cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbe4a00640 1 -- 172.21.15.57:0/895374738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 msgr2=0x55d14449a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbe4a00640 1 -- 172.21.15.57:0/895374738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1441cf7d0 con 0x55d144494f00 2024-04-13T05:23:35.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:35.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.151+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d144495320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009cc0 tx=0x7fdbd4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21dcad53367c3906 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.152+0000 7fdbd1c00640 1 -- 172.21.15.57:0/895374738 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4005220 con 0x55d144494f00 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.152+0000 7fdbd1c00640 1 -- 172.21.15.57:0/895374738 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbd4004c50 con 0x55d144494f00 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 -- 172.21.15.57:0/895374738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 msgr2=0x55d144495320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d144495320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009cc0 tx=0x7fdbd4011810 comp rx=0 tx=0).stop 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbd1c00640 1 -- 172.21.15.57:0/895374738 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40197f0 con 0x55d144494f00 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 -- 172.21.15.57:0/895374738 shutdown_connections 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d14449aa60 0x55d14449cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 --2- 172.21.15.57:0/895374738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d144495320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 -- 172.21.15.57:0/895374738 >> 172.21.15.57:0/895374738 conn(0x55d144490470 msgr2=0x55d144490870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 -- 172.21.15.57:0/895374738 shutdown_connections 2024-04-13T05:23:35.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.153+0000 7fdbea37a740 1 -- 172.21.15.57:0/895374738 wait complete. 2024-04-13T05:23:35.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.154+0000 7fdbea37a740 1 -- start start 2024-04-13T05:23:35.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.154+0000 7fdbea37a740 1 Processor -- start 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d14449f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbea37a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d14449dab0 con 0x55d1444958f0 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbea37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1443aca90 con 0x55d14449aa60 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbea37a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d14449dc30 con 0x55d144494f00 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbe5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbe5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45518/0 (socket says 172.21.15.57:45518) 2024-04-13T05:23:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.155+0000 7fdbe4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d14449f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbe4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d14449f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:40048/0 (socket says 172.21.15.57:40048) 2024-04-13T05:23:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbe4a00640 1 -- 172.21.15.57:0/1891015738 learned_addr learned my addr 172.21.15.57:0/1891015738 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbea37a740 10 client.0 _ll_register_callbacks cb 0x55d1443ac480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:35.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1891015738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 msgr2=0x55d14449f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d14449f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 msgr2=0x55d1444a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.156+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1891015738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1441053f0 con 0x55d1444958f0 2024-04-13T05:23:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.157+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449f780 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55d144495cf0 tx=0x7fdbcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44732ad9b838fc2e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.157+0000 7fdbc6000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.157+0000 7fdbc6000640 1 -- 172.21.15.57:0/1891015738 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc004220 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.157+0000 7fdbc6000640 1 -- 172.21.15.57:0/1891015738 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdbcc0043c0 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.157+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd4004c50 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.158+0000 7fdbc6000640 1 -- 172.21.15.57:0/1891015738 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc031050 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.158+0000 7fdbea37a740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.158+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1440c50f0 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.158+0000 7fdbc6000640 1 -- 172.21.15.57:0/1891015738 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbd4004c50 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.158+0000 7fdbc6000640 1 -- 172.21.15.57:0/1891015738 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdbcc003cb0 con 0x55d1444958f0 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.158+0000 7fdbea37a740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:35.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.159+0000 7fdbea37a740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.159+0000 7fdbea37a740 20 client.16620 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.159+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14448bbb0 0x55d14448e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.159+0000 7fdbea37a740 4 client.16620 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.159+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d144392600 con 0x55d14448bbb0 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.159+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14448bbb0 0x55d14448e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.160+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14448bbb0 0x55d14448e050 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276307c20f992fc7 server_cookie=64ea86bcaebf17bf in_seq=0 out_seq=0 2024-04-13T05:23:35.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.160+0000 7fdbc6000640 10 client.16620 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbc6000640 1 -- 172.21.15.57:0/1891015738 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d144392600 con 0x55d14448bbb0 2024-04-13T05:23:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbc6000640 10 client.16620 handle_command_reply: tid=0 2024-04-13T05:23:35.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbea37a740 1 client.16620 shutdown 2024-04-13T05:23:35.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 msgr2=0x55d14449f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449f780 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55d144495cf0 tx=0x7fdbcc004730 comp rx=0 tx=0).stop 2024-04-13T05:23:35.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 shutdown_connections 2024-04-13T05:23:35.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.161+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14448bbb0 0x55d14448e050 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d14449f780 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1891015738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d14449f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 >> 172.21.15.57:0/1891015738 conn(0x55d144490470 msgr2=0x55d144486e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbc6000640 0 client.16620 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 shutdown_connections 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 1 -- 172.21.15.57:0/1891015738 wait complete. 2024-04-13T05:23:35.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.162+0000 7fdbea37a740 20 client.16620 trim_cache size 0 max 16384 2024-04-13T05:23:35.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 Processor -- start 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 -- start start 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d144493de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d1444a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0012530 con 0x55d1444958f0 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc000fda0 con 0x55d14449aa60 2024-04-13T05:23:35.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.190+0000 7fdbea37a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc000dbf0 con 0x55d144494f00 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d1444a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d1444a6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36728/0 (socket says 172.21.15.57:36728) 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1402926826 learned_addr learned my addr 172.21.15.57:0/1402926826 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbe5400640 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d144493de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1402926826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 msgr2=0x55d144493de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d144493de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1402926826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 msgr2=0x55d1444865f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444865f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.191+0000 7fdbdfe00640 1 -- 172.21.15.57:0/1402926826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d144495530 con 0x55d1444958f0 2024-04-13T05:23:35.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.192+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d1444a6780 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc01fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8979971ef5a7b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.192+0000 7fdbd1200640 1 -- 172.21.15.57:0/1402926826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc039020 con 0x55d1444958f0 2024-04-13T05:23:35.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.192+0000 7fdbd1200640 1 -- 172.21.15.57:0/1402926826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbcc01fd40 con 0x55d1444958f0 2024-04-13T05:23:35.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.192+0000 7fdbd1200640 1 -- 172.21.15.57:0/1402926826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc01eda0 con 0x55d1444958f0 2024-04-13T05:23:35.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 -- 172.21.15.57:0/1402926826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 msgr2=0x55d1444a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d1444a6780 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc01fa00 comp rx=0 tx=0).stop 2024-04-13T05:23:35.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 -- 172.21.15.57:0/1402926826 shutdown_connections 2024-04-13T05:23:35.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d14449aa60 0x55d1444865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1444958f0 0x55d1444a6780 secure :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000c00 tx=0x7fdbcc01fa00 comp rx=0 tx=0).stop 2024-04-13T05:23:35.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 --2- 172.21.15.57:0/1402926826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d144494f00 0x55d144493de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.193+0000 7fdbea37a740 1 -- 172.21.15.57:0/1402926826 >> 172.21.15.57:0/1402926826 conn(0x55d144490470 msgr2=0x55d1444936d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.194+0000 7fdbea37a740 1 -- 172.21.15.57:0/1402926826 shutdown_connections 2024-04-13T05:23:35.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.194+0000 7fdbea37a740 1 -- 172.21.15.57:0/1402926826 wait complete. 2024-04-13T05:23:35.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.195+0000 7fdbea37a740 1 -- start start 2024-04-13T05:23:35.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.195+0000 7fdbea37a740 1 Processor -- start 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.195+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d1444a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.195+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 0x55d1444a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.195+0000 7fdbea37a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1444a5cf0 0x55d1444a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbea37a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc000e260 con 0x55d14449aa60 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbea37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc000f6d0 con 0x55d144494f00 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbea37a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc000e930 con 0x55d1444a5cf0 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbea37a740 10 client.0 _ll_register_callbacks cb 0x7fdbcc011870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 0x55d1444a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 0x55d1444a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36732/0 (socket says 172.21.15.57:36732) 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 -- 172.21.15.57:0/3007567854 learned_addr learned my addr 172.21.15.57:0/3007567854 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d1444a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbe5400640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1444a5cf0 0x55d1444a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 -- 172.21.15.57:0/3007567854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1444a5cf0 msgr2=0x55d1444a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1444a5cf0 0x55d1444a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 -- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 msgr2=0x55d1444a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.196+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d1444a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbdfe00640 1 -- 172.21.15.57:0/3007567854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1444a66b0 con 0x55d14449aa60 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbe5400640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1444a5cf0 0x55d1444a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d1444a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbdfe00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 0x55d1444a2e50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc01ffa0 tx=0x7fdbcc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ad91ccd7ca7eab server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbc6000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbc6000640 1 -- 172.21.15.57:0/3007567854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc039030 con 0x55d14449aa60 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbc6000640 1 -- 172.21.15.57:0/3007567854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdbcc01ed20 con 0x55d14449aa60 2024-04-13T05:23:35.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.197+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc01fd40 con 0x55d14449aa60 2024-04-13T05:23:35.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.198+0000 7fdbc6000640 1 -- 172.21.15.57:0/3007567854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc04d050 con 0x55d14449aa60 2024-04-13T05:23:35.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.198+0000 7fdbea37a740 10 client.16626 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:35.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.198+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d14452b2b0 con 0x55d14449aa60 2024-04-13T05:23:35.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbc6000640 1 -- 172.21.15.57:0/3007567854 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbcc01fd40 con 0x55d14449aa60 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbc6000640 1 -- 172.21.15.57:0/3007567854 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdbcc034040 con 0x55d14449aa60 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbea37a740 10 client.16626 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbea37a740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbea37a740 20 client.16626 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbea37a740 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14452bc30 0x55d14452e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbea37a740 4 client.16626 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.199+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d144400a40 con 0x55d14452bc30 2024-04-13T05:23:35.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.201+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14452bc30 0x55d14452e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.201+0000 7fdbe4a00640 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14452bc30 0x55d14452e050 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db391eb79961de1 server_cookie=1c1e40889bc52119 in_seq=0 out_seq=0 2024-04-13T05:23:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.201+0000 7fdbc6000640 10 client.16626 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.202+0000 7fdbc6000640 1 -- 172.21.15.57:0/3007567854 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d144400a40 con 0x55d14452bc30 2024-04-13T05:23:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.202+0000 7fdbc6000640 10 client.16626 handle_command_reply: tid=0 2024-04-13T05:23:35.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.202+0000 7fdbea37a740 1 client.16626 shutdown 2024-04-13T05:23:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.202+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 msgr2=0x55d1444a2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 0x55d1444a2e50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc01ffa0 tx=0x7fdbcc010da0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 shutdown_connections 2024-04-13T05:23:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d14452bc30 0x55d14452e050 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1444a5cf0 0x55d1444a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbc6000640 0 client.16626 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d14449aa60 0x55d1444a2e50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 --2- 172.21.15.57:0/3007567854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d144494f00 0x55d1444a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 >> 172.21.15.57:0/3007567854 conn(0x55d144490470 msgr2=0x55d144493f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.203+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 shutdown_connections 2024-04-13T05:23:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.204+0000 7fdbea37a740 1 -- 172.21.15.57:0/3007567854 wait complete. 2024-04-13T05:23:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.204+0000 7fdbea37a740 20 client.16626 trim_cache size 0 max 16384 2024-04-13T05:23:35.207 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:23:35.207 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:23:35.207 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:23:35.207 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:23:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.204+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc80a3c00 msgr2=0x7fdbc80a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc80a3c00 0x7fdbc80a60c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdbe019b7a0 tx=0x7fdbcc00b390 comp rx=0 tx=0).stop 2024-04-13T05:23:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 msgr2=0x7fdbe01a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe01a1840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8034fd0 tx=0x7fdbd8033c40 comp rx=0 tx=0).stop 2024-04-13T05:23:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 shutdown_connections 2024-04-13T05:23:35.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdbc80a3c00 0x7fdbc80a60c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe0101aa0 0x7fdbe01a1840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdbe0101100 0x7fdbe019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 --2- 172.21.15.57:0/729637005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbe01007a0 0x7fdbe019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 >> 172.21.15.57:0/729637005 conn(0x7fdbe00fc360 msgr2=0x7fdbe00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.205+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 shutdown_connections 2024-04-13T05:23:35.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.206+0000 7fdbe5e00640 1 -- 172.21.15.57:0/729637005 wait complete. 2024-04-13T05:23:35.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:23:35.230 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:23:35.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.485+0000 7f3ada800640 1 -- 172.21.15.57:0/1479434055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 msgr2=0x7f3ad4103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.485+0000 7f3ada800640 1 --2- 172.21.15.57:0/1479434055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 0x7f3ad4103430 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009900 tx=0x7f3ac002bc40 comp rx=0 tx=0).stop 2024-04-13T05:23:35.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 -- 172.21.15.57:0/1479434055 shutdown_connections 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 --2- 172.21.15.57:0/1479434055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4103970 0x7f3ad410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 --2- 172.21.15.57:0/1479434055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 0x7f3ad4103430 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 --2- 172.21.15.57:0/1479434055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ad4101e60 0x7f3ad4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 -- 172.21.15.57:0/1479434055 >> 172.21.15.57:0/1479434055 conn(0x7f3ad40fd3d0 msgr2=0x7f3ad40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 -- 172.21.15.57:0/1479434055 shutdown_connections 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.486+0000 7f3ada800640 1 -- 172.21.15.57:0/1479434055 wait complete. 2024-04-13T05:23:35.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.487+0000 7f3ada800640 1 Processor -- start 2024-04-13T05:23:35.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.487+0000 7f3ada800640 1 -- start start 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ada800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 0x7f3ad419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ada800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 0x7f3ad419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ada800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ad4103970 0x7f3ad41a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ada800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad4075150 con 0x7f3ad4102fb0 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ada800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad4074fd0 con 0x7f3ad4101e60 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ada800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad40752d0 con 0x7f3ad4103970 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ad8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 0x7f3ad419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ad9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 0x7f3ad419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ad9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 0x7f3ad419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45548/0 (socket says 172.21.15.57:45548) 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ad9400640 1 -- 172.21.15.57:0/608530007 learned_addr learned my addr 172.21.15.57:0/608530007 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:35.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.488+0000 7f3ad9e00640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ad4103970 0x7f3ad41a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad9400640 1 -- 172.21.15.57:0/608530007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ad4103970 msgr2=0x7f3ad41a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad9400640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ad4103970 0x7f3ad41a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad9400640 1 -- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 msgr2=0x7f3ad419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad9400640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 0x7f3ad419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad9400640 1 -- 172.21.15.57:0/608530007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad4110fc0 con 0x7f3ad4101e60 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad9400640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 0x7f3ad419e700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800d950 tx=0x7f3ac800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.489+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8017070 con 0x7f3ad4101e60 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.490+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ac8009c30 con 0x7f3ad4101e60 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.490+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800f620 con 0x7f3ad4101e60 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.490+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad41112b0 con 0x7f3ad4101e60 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.490+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad4111780 con 0x7f3ad4101e60 2024-04-13T05:23:35.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.491+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad41119e0 con 0x7f3ad4101e60 2024-04-13T05:23:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.495+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ac80109a0 con 0x7f3ad4101e60 2024-04-13T05:23:35.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.497+0000 7f3ad2000640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3abc0a3c80 0x7f3abc0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:35.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.497+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3ac80d0250 con 0x7f3ad4101e60 2024-04-13T05:23:35.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.497+0000 7f3ad8a00640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3abc0a3c80 0x7f3abc0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:35.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.499+0000 7f3ad8a00640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3abc0a3c80 0x7f3abc0a6140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3ac002c250 tx=0x7f3ac0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:35.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.504+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ac8009de0 con 0x7f3ad4101e60 2024-04-13T05:23:35.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:35 smithi057 ceph-mon[32331]: pgmap v635: 129 pgs: 129 active+clean; 339 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-04-13T05:23:35.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:35 smithi189 ceph-mon[35531]: pgmap v635: 129 pgs: 129 active+clean; 339 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-04-13T05:23:35.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:35 smithi083 ceph-mon[35133]: pgmap v635: 129 pgs: 129 active+clean; 339 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 152 op/s 2024-04-13T05:23:35.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.915+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ad4111ea0 con 0x7f3ad4101e60 2024-04-13T05:23:35.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.916+0000 7f3ad2000640 1 -- 172.21.15.57:0/608530007 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11858 (secure 0 0 0) 0x7f3ac809c2c0 con 0x7f3ad4101e60 2024-04-13T05:23:35.919 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:23:35.920 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:22:16.935949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:23:35.920 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 29 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3abc0a3c80 msgr2=0x7f3abc0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3abc0a3c80 0x7f3abc0a6140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3ac002c250 tx=0x7f3ac0005fd0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 msgr2=0x7f3ad419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 0x7f3ad419e700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800d950 tx=0x7f3ac800de10 comp rx=0 tx=0).stop 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 shutdown_connections 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3abc0a3c80 0x7f3abc0a6140 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ad4103970 0x7f3ad41a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ad4102fb0 0x7f3ad419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 --2- 172.21.15.57:0/608530007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad4101e60 0x7f3ad419e700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 >> 172.21.15.57:0/608530007 conn(0x7f3ad40fd3d0 msgr2=0x7f3ad40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:35.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 shutdown_connections 2024-04-13T05:23:35.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:35.920+0000 7f3ada800640 1 -- 172.21.15.57:0/608530007 wait complete. 2024-04-13T05:23:35.945 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:23:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 -- 172.21.15.57:0/4199275954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c0749c0 msgr2=0x7f004c074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 --2- 172.21.15.57:0/4199275954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c0749c0 0x7f004c074e20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f003c009900 tx=0x7f003c02bcd0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 -- 172.21.15.57:0/4199275954 shutdown_connections 2024-04-13T05:23:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 --2- 172.21.15.57:0/4199275954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c109d60 0x7f004c10c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 --2- 172.21.15.57:0/4199275954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c0749c0 0x7f004c074e20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 --2- 172.21.15.57:0/4199275954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f004c075fa0 0x7f004c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 -- 172.21.15.57:0/4199275954 >> 172.21.15.57:0/4199275954 conn(0x7f004c0fd460 msgr2=0x7f004c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.198+0000 7f0053200640 1 -- 172.21.15.57:0/4199275954 shutdown_connections 2024-04-13T05:23:36.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.199+0000 7f0053200640 1 -- 172.21.15.57:0/4199275954 wait complete. 2024-04-13T05:23:36.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 Processor -- start 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 -- start start 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c0749c0 0x7f004c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 0x7f004c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f004c109d60 0x7f004c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f004c10fc60 con 0x7f004c075fa0 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f004c10fae0 con 0x7f004c0749c0 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.200+0000 7f0053200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f004c10fde0 con 0x7f004c109d60 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 0x7f004c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c0749c0 0x7f004c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c0749c0 0x7f004c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45568/0 (socket says 172.21.15.57:45568) 2024-04-13T05:23:36.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051e00640 1 -- 172.21.15.57:0/4072417096 learned_addr learned my addr 172.21.15.57:0/4072417096 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 0x7f004c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36756/0 (socket says 172.21.15.57:36756) 2024-04-13T05:23:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 -- 172.21.15.57:0/4072417096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f004c109d60 msgr2=0x7f004c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f004c109d60 0x7f004c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 -- 172.21.15.57:0/4072417096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c0749c0 msgr2=0x7f004c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c0749c0 0x7f004c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.201+0000 7f0051400640 1 -- 172.21.15.57:0/4072417096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c1a1e70 con 0x7f004c075fa0 2024-04-13T05:23:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.202+0000 7f0051400640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 0x7f004c19a790 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f003c02c210 tx=0x7f003c02da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.202+0000 7f0046a00640 1 -- 172.21.15.57:0/4072417096 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c043020 con 0x7f004c075fa0 2024-04-13T05:23:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.202+0000 7f0046a00640 1 -- 172.21.15.57:0/4072417096 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f003c02de10 con 0x7f004c075fa0 2024-04-13T05:23:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.202+0000 7f0046a00640 1 -- 172.21.15.57:0/4072417096 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c003bc0 con 0x7f004c075fa0 2024-04-13T05:23:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.202+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c1a2100 con 0x7f004c075fa0 2024-04-13T05:23:36.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.202+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c1a2670 con 0x7f004c075fa0 2024-04-13T05:23:36.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.206+0000 7f0046a00640 1 -- 172.21.15.57:0/4072417096 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f003c046020 con 0x7f004c075fa0 2024-04-13T05:23:36.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.206+0000 7f0057805740 1 Processor -- start 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.206+0000 7f0057805740 1 -- start start 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0057805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0057805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5e30 0x56275fada930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0057805740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fadd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0057805740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56275fa3b020 con 0x56275fadafa0 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0057805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56275f9e1950 con 0x56275fad5e30 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0057805740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56275fa36fe0 con 0x56275fad5440 2024-04-13T05:23:36.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fadd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fadd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36772/0 (socket says 172.21.15.57:36772) 2024-04-13T05:23:36.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0051400640 1 -- 172.21.15.57:0/2884552940 learned_addr learned my addr 172.21.15.57:0/2884552940 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0051e00640 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5e30 0x56275fada930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.207+0000 7f0052800640 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0046a00640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f00340a3c30 0x7f00340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0051400640 1 -- 172.21.15.57:0/2884552940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 msgr2=0x56275fad5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0051400640 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0051400640 1 -- 172.21.15.57:0/2884552940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5e30 msgr2=0x56275fada930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0051400640 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5e30 0x56275fada930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0051400640 1 -- 172.21.15.57:0/2884552940 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275f74d3f0 con 0x56275fadafa0 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0051e00640 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5e30 0x56275fada930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0052800640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f00340a3c30 0x7f00340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.208+0000 7f0046a00640 1 -- 172.21.15.57:0/4072417096 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f003c0f2940 con 0x7f004c075fa0 2024-04-13T05:23:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.209+0000 7f0051400640 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fadd430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f003c042330 tx=0x7f003c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1398447a9549cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.209+0000 7f003b000640 1 -- 172.21.15.57:0/2884552940 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0be3c0 con 0x56275fadafa0 2024-04-13T05:23:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.209+0000 7f003b000640 1 -- 172.21.15.57:0/2884552940 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f003c042170 con 0x56275fadafa0 2024-04-13T05:23:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.209+0000 7f003b000640 1 -- 172.21.15.57:0/2884552940 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0a9590 con 0x56275fadafa0 2024-04-13T05:23:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.209+0000 7f0052800640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f00340a3c30 0x7f00340a60f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56275fad6690 tx=0x7f0048004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 -- 172.21.15.57:0/2884552940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 msgr2=0x56275fadd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fadd430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f003c042330 tx=0x7f003c04f000 comp rx=0 tx=0).stop 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 -- 172.21.15.57:0/2884552940 shutdown_connections 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5e30 0x56275fada930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fadd430 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 --2- 172.21.15.57:0/2884552940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 -- 172.21.15.57:0/2884552940 >> 172.21.15.57:0/2884552940 conn(0x56275fad09b0 msgr2=0x56275fad0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 -- 172.21.15.57:0/2884552940 shutdown_connections 2024-04-13T05:23:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.210+0000 7f0057805740 1 -- 172.21.15.57:0/2884552940 wait complete. 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.211+0000 7f0057805740 1 -- start start 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.211+0000 7f0057805740 1 Processor -- start 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fadf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fadfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fadafa0 0x56275fae6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56275faddd10 con 0x56275fad5e30 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56275f9f4c80 con 0x56275fad5440 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56275fadde90 con 0x56275fadafa0 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0057805740 10 client.0 _ll_register_callbacks cb 0x56275f9f4670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fadfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fadfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36774/0 (socket says 172.21.15.57:36774) 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0051400640 1 -- 172.21.15.57:0/3337639271 learned_addr learned my addr 172.21.15.57:0/3337639271 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.212+0000 7f0051e00640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fadf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0052800640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fadafa0 0x56275fae6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0051400640 1 -- 172.21.15.57:0/3337639271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fadafa0 msgr2=0x56275fae6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0051400640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fadafa0 0x56275fae6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0051400640 1 -- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 msgr2=0x56275fadf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0051400640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fadf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0051400640 1 -- 172.21.15.57:0/3337639271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275f8177d0 con 0x56275fad5e30 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0051400640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fadfb60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f003c0bea40 tx=0x7f003c0f2c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa386e1514a74cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f002f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f002f400640 1 -- 172.21.15.57:0/3337639271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0be1e0 con 0x56275fad5e30 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f002f400640 1 -- 172.21.15.57:0/3337639271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f003c0d2fe0 con 0x56275fad5e30 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.213+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56275f9da750 con 0x56275fad5e30 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f002f400640 1 -- 172.21.15.57:0/3337639271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f003c0be3c0 con 0x56275fad5e30 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 10 client.16644 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56275faddd10 con 0x56275fad5e30 2024-04-13T05:23:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f002f400640 1 -- 172.21.15.57:0/3337639271 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56275f9da750 con 0x56275fad5e30 2024-04-13T05:23:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f002f400640 1 -- 172.21.15.57:0/3337639271 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f003c0f0120 con 0x56275fad5e30 2024-04-13T05:23:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 10 client.16644 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 10 client.16644 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:23:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 20 client.16644 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:36.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275faeb310 0x56275faed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 4 client.16644 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:36.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.214+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56275fa48a40 con 0x56275faeb310 2024-04-13T05:23:36.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.253+0000 7f0051e00640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275faeb310 0x56275faed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.257+0000 7f0051e00640 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275faeb310 0x56275faed750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21598dcc77730679 server_cookie=f9eeb14d63c04bf in_seq=0 out_seq=0 2024-04-13T05:23:36.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.257+0000 7f002f400640 10 client.16644 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:36.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f002f400640 1 -- 172.21.15.57:0/3337639271 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56275fa48a40 con 0x56275faeb310 2024-04-13T05:23:36.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f002f400640 10 client.16644 handle_command_reply: tid=0 2024-04-13T05:23:36.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 client.16644 shutdown 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 msgr2=0x56275fadfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fadfb60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f003c0bea40 tx=0x7f003c0f2c10 comp rx=0 tx=0).stop 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 shutdown_connections 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275faeb310 0x56275faed750 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fadafa0 0x56275fae6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fadfb60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 --2- 172.21.15.57:0/3337639271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fadf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f002f400640 0 client.16644 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:36.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 >> 172.21.15.57:0/3337639271 conn(0x56275fad09b0 msgr2=0x56275facfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 shutdown_connections 2024-04-13T05:23:36.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 1 -- 172.21.15.57:0/3337639271 wait complete. 2024-04-13T05:23:36.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.263+0000 7f0057805740 20 client.16644 trim_cache size 0 max 16384 2024-04-13T05:23:36.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 Processor -- start 2024-04-13T05:23:36.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 -- start start 2024-04-13T05:23:36.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fae6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fadafa0 0x56275fad7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1af440 con 0x56275fad5e30 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1accb0 con 0x56275fadafa0 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.287+0000 7f0057805740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1aab00 con 0x56275fad5440 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0052800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fadafa0 0x56275fad7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0051e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fae6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0051400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fae6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36776/0 (socket says 172.21.15.57:36776) 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0052800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fadafa0 0x56275fad7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45604/0 (socket says 172.21.15.57:45604) 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0051400640 1 -- 172.21.15.57:0/860434400 learned_addr learned my addr 172.21.15.57:0/860434400 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.288+0000 7f0051e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:40132/0 (socket says 172.21.15.57:40132) 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.289+0000 7f0051400640 1 -- 172.21.15.57:0/860434400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 msgr2=0x56275fad7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.289+0000 7f0051400640 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.289+0000 7f0051400640 1 -- 172.21.15.57:0/860434400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fadafa0 msgr2=0x56275fad7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.289+0000 7f0051400640 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fadafa0 0x56275fad7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.289+0000 7f0051400640 1 -- 172.21.15.57:0/860434400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275facc410 con 0x56275fad5e30 2024-04-13T05:23:36.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.290+0000 7f0051400640 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fae6b60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f003c006010 tx=0x7f003c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c79dfeb190f6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.291+0000 7f003a600640 1 -- 172.21.15.57:0/860434400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0be650 con 0x56275fad5e30 2024-04-13T05:23:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.291+0000 7f003a600640 1 -- 172.21.15.57:0/860434400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f003c0be880 con 0x56275fad5e30 2024-04-13T05:23:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.291+0000 7f003a600640 1 -- 172.21.15.57:0/860434400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0f00f0 con 0x56275fad5e30 2024-04-13T05:23:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 -- 172.21.15.57:0/860434400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 msgr2=0x56275fae6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fae6b60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f003c006010 tx=0x7f003c009620 comp rx=0 tx=0).stop 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 -- 172.21.15.57:0/860434400 shutdown_connections 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fadafa0 0x56275fad7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fad5e30 0x56275fae6b60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 --2- 172.21.15.57:0/860434400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5440 0x56275fad7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 -- 172.21.15.57:0/860434400 >> 172.21.15.57:0/860434400 conn(0x56275fad09b0 msgr2=0x56275facbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 -- 172.21.15.57:0/860434400 shutdown_connections 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.292+0000 7f0057805740 1 -- 172.21.15.57:0/860434400 wait complete. 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.293+0000 7f0057805740 1 -- start start 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.293+0000 7f0057805740 1 Processor -- start 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fae39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5e30 0x56275fae3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fb4f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1ab170 con 0x56275fadafa0 2024-04-13T05:23:36.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1ac5e0 con 0x56275fad5440 2024-04-13T05:23:36.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1ab840 con 0x56275fad5e30 2024-04-13T05:23:36.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0057805740 10 client.0 _ll_register_callbacks cb 0x7f003c0babb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:36.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0051e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fae39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0051400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5e30 0x56275fae3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0051e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fae39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45618/0 (socket says 172.21.15.57:45618) 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.294+0000 7f0051e00640 1 -- 172.21.15.57:0/1611585209 learned_addr learned my addr 172.21.15.57:0/1611585209 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0052800640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fb4f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0051400640 1 -- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 msgr2=0x56275fae39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0051400640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fae39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0051400640 1 -- 172.21.15.57:0/1611585209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 msgr2=0x56275fb4f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0051400640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fb4f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0051400640 1 -- 172.21.15.57:0/1611585209 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275fb4fdc0 con 0x56275fad5e30 2024-04-13T05:23:36.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.295+0000 7f0051e00640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fae39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:36.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.296+0000 7f0051400640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5e30 0x56275fae3f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f003c0bea40 tx=0x7f003c003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647d5970b1021b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.296+0000 7f002f400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:23:36.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.301+0000 7f002f400640 1 -- 172.21.15.57:0/1611585209 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c0be650 con 0x56275fad5e30 2024-04-13T05:23:36.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.301+0000 7f002f400640 1 -- 172.21.15.57:0/1611585209 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f003c0c7060 con 0x56275fad5e30 2024-04-13T05:23:36.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.302+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56275fa428b0 con 0x56275fad5e30 2024-04-13T05:23:36.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.302+0000 7f002f400640 1 -- 172.21.15.57:0/1611585209 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f003c0c72e0 con 0x56275fad5e30 2024-04-13T05:23:36.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.302+0000 7f0057805740 10 client.25966 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:36.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.302+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56275fb4fdc0 con 0x56275fad5e30 2024-04-13T05:23:36.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.302+0000 7f002f400640 1 -- 172.21.15.57:0/1611585209 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56275fa428b0 con 0x56275fad5e30 2024-04-13T05:23:36.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f002f400640 1 -- 172.21.15.57:0/1611585209 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f003c0beae0 con 0x56275fad5e30 2024-04-13T05:23:36.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0057805740 10 client.25966 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:36.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0057805740 10 client.25966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:23:36.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0057805740 20 client.25966 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:36.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0057805740 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275fb51670 0x56275fb53a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0057805740 4 client.25966 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:23:36.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56275f9be540 con 0x56275fb51670 2024-04-13T05:23:36.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.303+0000 7f0051e00640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275fb51670 0x56275fb53a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.306+0000 7f0051e00640 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275fb51670 0x56275fb53a90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39b7473ef87093b6 server_cookie=6bd3d1d28f9b1532 in_seq=0 out_seq=0 2024-04-13T05:23:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.306+0000 7f002f400640 10 client.25966 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f002f400640 1 -- 172.21.15.57:0/1611585209 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56275f9be540 con 0x56275fb51670 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f002f400640 10 client.25966 handle_command_reply: tid=0 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 client.25966 shutdown 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5e30 msgr2=0x56275fae3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5e30 0x56275fae3f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f003c0bea40 tx=0x7f003c003e60 comp rx=0 tx=0).stop 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 shutdown_connections 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56275fb51670 0x56275fb53a90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56275fadafa0 0x56275fb4f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56275fad5e30 0x56275fae3f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 --2- 172.21.15.57:0/1611585209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56275fad5440 0x56275fae39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 >> 172.21.15.57:0/1611585209 conn(0x56275fad09b0 msgr2=0x56275fade110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f002f400640 0 client.25966 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:23:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.307+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 shutdown_connections 2024-04-13T05:23:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.308+0000 7f0057805740 1 -- 172.21.15.57:0/1611585209 wait complete. 2024-04-13T05:23:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.308+0000 7f0057805740 20 client.25966 trim_cache size 0 max 16384 2024-04-13T05:23:36.311 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f00340a3c30 msgr2=0x7f00340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f00340a3c30 0x7f00340a60f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x56275fad6690 tx=0x7f0048004a40 comp rx=0 tx=0).stop 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 msgr2=0x7f004c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 0x7f004c19a790 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f003c02c210 tx=0x7f003c02da10 comp rx=0 tx=0).stop 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 shutdown_connections 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f00340a3c30 0x7f00340a60f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f004c109d60 0x7f004c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f004c075fa0 0x7f004c19a790 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 --2- 172.21.15.57:0/4072417096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f004c0749c0 0x7f004c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.309+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 >> 172.21.15.57:0/4072417096 conn(0x7f004c0fd460 msgr2=0x7f004c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.310+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 shutdown_connections 2024-04-13T05:23:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.310+0000 7f0053200640 1 -- 172.21.15.57:0/4072417096 wait complete. 2024-04-13T05:23:36.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:23:36.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:36 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/608530007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:23:36.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.586+0000 7fdba3800640 1 -- 172.21.15.57:0/1332577501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c105430 msgr2=0x7fdb9c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.586+0000 7fdba3800640 1 --2- 172.21.15.57:0/1332577501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c105430 0x7fdb9c1058b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009900 tx=0x7fdb8802bc40 comp rx=0 tx=0).stop 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 -- 172.21.15.57:0/1332577501 shutdown_connections 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 --2- 172.21.15.57:0/1332577501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c105430 0x7fdb9c1058b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 --2- 172.21.15.57:0/1332577501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105df0 0x7fdb9c10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 --2- 172.21.15.57:0/1332577501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb9c104240 0x7fdb9c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 -- 172.21.15.57:0/1332577501 >> 172.21.15.57:0/1332577501 conn(0x7fdb9c0ff7b0 msgr2=0x7fdb9c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 -- 172.21.15.57:0/1332577501 shutdown_connections 2024-04-13T05:23:36.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.587+0000 7fdba3800640 1 -- 172.21.15.57:0/1332577501 wait complete. 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 Processor -- start 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 -- start start 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 0x7fdb9c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105430 0x7fdb9c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb9c105df0 0x7fdb9c1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c10fd80 con 0x7fdb9c105df0 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c10fc00 con 0x7fdb9c104240 2024-04-13T05:23:36.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.588+0000 7fdba3800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c10ff00 con 0x7fdb9c105430 2024-04-13T05:23:36.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 0x7fdb9c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb9c105df0 0x7fdb9c1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 0x7fdb9c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45644/0 (socket says 172.21.15.57:45644) 2024-04-13T05:23:36.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 -- 172.21.15.57:0/986393540 learned_addr learned my addr 172.21.15.57:0/986393540 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdb9be00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105430 0x7fdb9c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 -- 172.21.15.57:0/986393540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105430 msgr2=0x7fdb9c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105430 0x7fdb9c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 -- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb9c105df0 msgr2=0x7fdb9c1a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb9c105df0 0x7fdb9c1a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdba0e00640 1 -- 172.21.15.57:0/986393540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c1a2080 con 0x7fdb9c104240 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.589+0000 7fdb9be00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105430 0x7fdb9c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.590+0000 7fdba0e00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 0x7fdb9c19a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000e9c0 tx=0x7fdb9000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.590+0000 7fdb99600640 1 -- 172.21.15.57:0/986393540 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000cd80 con 0x7fdb9c104240 2024-04-13T05:23:36.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.590+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c076460 con 0x7fdb9c104240 2024-04-13T05:23:36.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.590+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c076a00 con 0x7fdb9c104240 2024-04-13T05:23:36.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.591+0000 7fdb99600640 1 -- 172.21.15.57:0/986393540 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb900035f0 con 0x7fdb9c104240 2024-04-13T05:23:36.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.591+0000 7fdb99600640 1 -- 172.21.15.57:0/986393540 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90010690 con 0x7fdb9c104240 2024-04-13T05:23:36.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.595+0000 7fdba67d5740 1 Processor -- start 2024-04-13T05:23:36.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.595+0000 7fdba67d5740 1 -- start start 2024-04-13T05:23:36.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 0x563aa40a1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdb99600640 1 -- 172.21.15.57:0/986393540 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdb900037a0 con 0x7fdb9c104240 2024-04-13T05:23:36.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a6db0 0x563aa40a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba67d5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563aa4007020 con 0x563aa40a1250 2024-04-13T05:23:36.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba67d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563aa3fad950 con 0x563aa40a6db0 2024-04-13T05:23:36.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba67d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563aa4002fe0 con 0x563aa40a1c40 2024-04-13T05:23:36.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 0x563aa40a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 0x563aa40a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36804/0 (socket says 172.21.15.57:36804) 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.596+0000 7fdba1800640 1 -- 172.21.15.57:0/558008138 learned_addr learned my addr 172.21.15.57:0/558008138 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdb9be00640 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a6db0 0x563aa40a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba0e00640 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba1800640 1 -- 172.21.15.57:0/558008138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 msgr2=0x563aa40a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba1800640 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba1800640 1 -- 172.21.15.57:0/558008138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a6db0 msgr2=0x563aa40a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba1800640 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a6db0 0x563aa40a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba1800640 1 -- 172.21.15.57:0/558008138 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa3d193f0 con 0x563aa40a1250 2024-04-13T05:23:36.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.597+0000 7fdba1800640 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 0x563aa40a1670 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdb94010370 tx=0x7fdb94010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4068f8cc55a997a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.598+0000 7fdb8e000640 1 -- 172.21.15.57:0/558008138 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94016070 con 0x563aa40a1250 2024-04-13T05:23:36.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.598+0000 7fdb8e000640 1 -- 172.21.15.57:0/558008138 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb94004d20 con 0x563aa40a1250 2024-04-13T05:23:36.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.598+0000 7fdb8e000640 1 -- 172.21.15.57:0/558008138 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94005390 con 0x563aa40a1250 2024-04-13T05:23:36.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.598+0000 7fdb99600640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb840a3c30 0x7fdb840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdb99600640 1 -- 172.21.15.57:0/986393540 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdb900d0b00 con 0x7fdb9c104240 2024-04-13T05:23:36.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba0e00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb840a3c30 0x7fdb840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 -- 172.21.15.57:0/558008138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 msgr2=0x563aa40a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 0x563aa40a1670 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdb94010370 tx=0x7fdb94010830 comp rx=0 tx=0).stop 2024-04-13T05:23:36.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 -- 172.21.15.57:0/558008138 shutdown_connections 2024-04-13T05:23:36.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a6db0 0x563aa40a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 --2- 172.21.15.57:0/558008138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a1250 0x563aa40a1670 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 -- 172.21.15.57:0/558008138 >> 172.21.15.57:0/558008138 conn(0x563aa409c7c0 msgr2=0x563aa409cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.599+0000 7fdba67d5740 1 -- 172.21.15.57:0/558008138 shutdown_connections 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.600+0000 7fdba0e00640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb840a3c30 0x7fdb840a60f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdb90098e40 tx=0x7fdb900d4540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.600+0000 7fdba67d5740 1 -- 172.21.15.57:0/558008138 wait complete. 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.600+0000 7fdba67d5740 1 -- start start 2024-04-13T05:23:36.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 Processor -- start 2024-04-13T05:23:36.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1250 0x563aa40ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 0x563aa40abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563aa40a9d60 con 0x563aa40a6db0 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563aa3fc0c80 con 0x563aa40a1c40 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.601+0000 7fdba67d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563aa40a9ee0 con 0x563aa40a1250 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.602+0000 7fdba67d5740 10 client.0 _ll_register_callbacks cb 0x563aa3fc0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.602+0000 7fdb9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 0x563aa40abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.602+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.602+0000 7fdb9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 0x563aa40abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45656/0 (socket says 172.21.15.57:45656) 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.602+0000 7fdb9be00640 1 -- 172.21.15.57:0/2315140655 learned_addr learned my addr 172.21.15.57:0/2315140655 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.602+0000 7fdba1800640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1250 0x563aa40ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb9be00640 1 -- 172.21.15.57:0/2315140655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1250 msgr2=0x563aa40ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb9be00640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1250 0x563aa40ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb9be00640 1 -- 172.21.15.57:0/2315140655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 msgr2=0x563aa40b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb9be00640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb9be00640 1 -- 172.21.15.57:0/2315140655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa3de37d0 con 0x563aa40a1c40 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdba1800640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb9be00640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 0x563aa40abd00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb88005940 tx=0x7fdb880042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5705cdb920284f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.603+0000 7fdb82a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.604+0000 7fdb82a00640 1 -- 172.21.15.57:0/2315140655 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880044e0 con 0x563aa40a1c40 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.604+0000 7fdb82a00640 1 -- 172.21.15.57:0/2315140655 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdb88005450 con 0x563aa40a1c40 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.604+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb94004d20 con 0x563aa40a1c40 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.604+0000 7fdb82a00640 1 -- 172.21.15.57:0/2315140655 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb880046c0 con 0x563aa40a1c40 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 10 client.36068 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563aa40b6150 con 0x563aa40a1c40 2024-04-13T05:23:36.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdb82a00640 1 -- 172.21.15.57:0/2315140655 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb94004d20 con 0x563aa40a1c40 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdb82a00640 1 -- 172.21.15.57:0/2315140655 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdb88003860 con 0x563aa40a1c40 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 10 client.36068 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 10 client.36068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 20 client.36068 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa40b7810 0x563aa40b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 4 client.36068 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.605+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aa3fa6750 con 0x563aa40b7810 2024-04-13T05:23:36.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.606+0000 7fdba1800640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa40b7810 0x563aa40b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.606+0000 7fdba1800640 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa40b7810 0x563aa40b9c70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e5e262b54c22c0 server_cookie=b97ed085835f628d in_seq=0 out_seq=0 2024-04-13T05:23:36.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.607+0000 7fdb82a00640 10 client.36068 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:36.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.607+0000 7fdb82a00640 1 -- 172.21.15.57:0/2315140655 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563aa3fa6750 con 0x563aa40b7810 2024-04-13T05:23:36.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.607+0000 7fdb82a00640 10 client.36068 handle_command_reply: tid=0 2024-04-13T05:23:36.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 client.36068 shutdown 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 msgr2=0x563aa40abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 0x563aa40abd00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb88005940 tx=0x7fdb880042f0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 shutdown_connections 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa40b7810 0x563aa40b9c70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1c40 0x563aa40abd00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2315140655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1250 0x563aa40ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 >> 172.21.15.57:0/2315140655 conn(0x563aa409c7c0 msgr2=0x563aa409ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdb82a00640 0 client.36068 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:36.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 shutdown_connections 2024-04-13T05:23:36.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.608+0000 7fdba67d5740 1 -- 172.21.15.57:0/2315140655 wait complete. 2024-04-13T05:23:36.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.609+0000 7fdba67d5740 20 client.36068 trim_cache size 0 max 16384 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.630+0000 7fdba67d5740 1 Processor -- start 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.630+0000 7fdba67d5740 1 -- start start 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba67d5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0125c0 con 0x563aa40a6db0 2024-04-13T05:23:36.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba67d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c00fe30 con 0x563aa40a1250 2024-04-13T05:23:36.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba67d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c00dc80 con 0x563aa40a1c40 2024-04-13T05:23:36.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40aa850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36834/0 (socket says 172.21.15.57:36834) 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdba1800640 1 -- 172.21.15.57:0/2178543419 learned_addr learned my addr 172.21.15.57:0/2178543419 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.631+0000 7fdb9be00640 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.632+0000 7fdba1800640 1 -- 172.21.15.57:0/2178543419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 msgr2=0x563aa40b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.632+0000 7fdba1800640 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.632+0000 7fdba1800640 1 -- 172.21.15.57:0/2178543419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 msgr2=0x563aa40aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.632+0000 7fdba1800640 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.632+0000 7fdba1800640 1 -- 172.21.15.57:0/2178543419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aa40ac720 con 0x563aa40a6db0 2024-04-13T05:23:36.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.632+0000 7fdba1800640 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40aa850 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009c40 tx=0x7fdb94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d4c58cc899fec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.633+0000 7fdb8d600640 1 -- 172.21.15.57:0/2178543419 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94016040 con 0x563aa40a6db0 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.633+0000 7fdb8d600640 1 -- 172.21.15.57:0/2178543419 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb9402b5b0 con 0x563aa40a6db0 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.633+0000 7fdb8d600640 1 -- 172.21.15.57:0/2178543419 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94012b50 con 0x563aa40a6db0 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 -- 172.21.15.57:0/2178543419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 msgr2=0x563aa40aa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40aa850 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009c40 tx=0x7fdb94009bb0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 -- 172.21.15.57:0/2178543419 shutdown_connections 2024-04-13T05:23:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40aa850 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 --2- 172.21.15.57:0/2178543419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 -- 172.21.15.57:0/2178543419 >> 172.21.15.57:0/2178543419 conn(0x563aa409c7c0 msgr2=0x563aa40a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 -- 172.21.15.57:0/2178543419 shutdown_connections 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.634+0000 7fdba67d5740 1 -- 172.21.15.57:0/2178543419 wait complete. 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.635+0000 7fdba67d5740 1 -- start start 2024-04-13T05:23:36.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.635+0000 7fdba67d5740 1 Processor -- start 2024-04-13T05:23:36.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.635+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba67d5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba67d5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c00e2f0 con 0x563aa40a6db0 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba67d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c00f760 con 0x563aa40a1250 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba67d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c00e9c0 con 0x563aa40a1c40 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba67d5740 10 client.0 _ll_register_callbacks cb 0x7fdb8802cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36848/0 (socket says 172.21.15.57:36848) 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 -- 172.21.15.57:0/1589410667 learned_addr learned my addr 172.21.15.57:0/1589410667 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdb9be00640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 -- 172.21.15.57:0/1589410667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 msgr2=0x563aa40b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 -- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 msgr2=0x563aa40b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.636+0000 7fdba1800640 1 -- 172.21.15.57:0/1589410667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aa40b1d90 con 0x563aa40a6db0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.637+0000 7fdba1800640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b17a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009b80 tx=0x7fdb940097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47ec5bb6bd79591 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.637+0000 7fdb82a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.637+0000 7fdb82a00640 1 -- 172.21.15.57:0/1589410667 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94016040 con 0x563aa40a6db0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.637+0000 7fdb82a00640 1 -- 172.21.15.57:0/1589410667 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdb9402b100 con 0x563aa40a6db0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.637+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9402b5b0 con 0x563aa40a6db0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdb82a00640 1 -- 172.21.15.57:0/1589410667 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb9402b300 con 0x563aa40a6db0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563aa411ddf0 con 0x563aa40a6db0 2024-04-13T05:23:36.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdb82a00640 1 -- 172.21.15.57:0/1589410667 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb9402b5b0 con 0x563aa40a6db0 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdb82a00640 1 -- 172.21.15.57:0/1589410667 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdb9400f040 con 0x563aa40a6db0 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 10 client.16668 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 20 client.16668 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa411e450 0x563aa4120870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 4 client.16668 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:23:36.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.638+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563aa4014a40 con 0x563aa411e450 2024-04-13T05:23:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.639+0000 7fdb9be00640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa411e450 0x563aa4120870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.640+0000 7fdb9be00640 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa411e450 0x563aa4120870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d55de36ab886e948 server_cookie=9f0ef0bb9f84390b in_seq=0 out_seq=0 2024-04-13T05:23:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.640+0000 7fdb82a00640 10 client.16668 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.641+0000 7fdb82a00640 1 -- 172.21.15.57:0/1589410667 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563aa4014a40 con 0x563aa411e450 2024-04-13T05:23:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.641+0000 7fdb82a00640 10 client.16668 handle_command_reply: tid=0 2024-04-13T05:23:36.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.641+0000 7fdba67d5740 1 client.16668 shutdown 2024-04-13T05:23:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 msgr2=0x563aa40b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b17a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009b80 tx=0x7fdb940097a0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 shutdown_connections 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x563aa411e450 0x563aa4120870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563aa40a6db0 0x563aa40b17a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aa40a1c40 0x563aa40b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdb82a00640 0 client.16668 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 --2- 172.21.15.57:0/1589410667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aa40a1250 0x563aa40b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 >> 172.21.15.57:0/1589410667 conn(0x563aa409c7c0 msgr2=0x563aa409b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.642+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 shutdown_connections 2024-04-13T05:23:36.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.643+0000 7fdba67d5740 1 -- 172.21.15.57:0/1589410667 wait complete. 2024-04-13T05:23:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.643+0000 7fdba67d5740 20 client.16668 trim_cache size 0 max 16384 2024-04-13T05:23:36.646 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.643+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb840a3c30 msgr2=0x7fdb840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.643+0000 7fdba3800640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb840a3c30 0x7fdb840a60f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdb90098e40 tx=0x7fdb900d4540 comp rx=0 tx=0).stop 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.643+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 msgr2=0x7fdb9c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.643+0000 7fdba3800640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 0x7fdb9c19a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000e9c0 tx=0x7fdb9000ee80 comp rx=0 tx=0).stop 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 shutdown_connections 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fdb840a3c30 0x7fdb840a60f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdb9c105df0 0x7fdb9c1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb9c105430 0x7fdb9c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 --2- 172.21.15.57:0/986393540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb9c104240 0x7fdb9c19a460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 >> 172.21.15.57:0/986393540 conn(0x7fdb9c0ff7b0 msgr2=0x7fdb9c108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 shutdown_connections 2024-04-13T05:23:36.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.644+0000 7fdba3800640 1 -- 172.21.15.57:0/986393540 wait complete. 2024-04-13T05:23:36.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:36 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/608530007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:23:36.668 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:23:36.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:36 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/608530007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 -- 172.21.15.57:0/1202469024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238104100 msgr2=0x7f9238104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 --2- 172.21.15.57:0/1202469024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238104100 0x7f9238104520 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f922c009be0 tx=0x7f922c02bef0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 -- 172.21.15.57:0/1202469024 shutdown_connections 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 --2- 172.21.15.57:0/1202469024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92381052f0 0x7f9238105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 --2- 172.21.15.57:0/1202469024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9238105cb0 0x7f923810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 --2- 172.21.15.57:0/1202469024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238104100 0x7f9238104520 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 -- 172.21.15.57:0/1202469024 >> 172.21.15.57:0/1202469024 conn(0x7f92380ff670 msgr2=0x7f9238101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 -- 172.21.15.57:0/1202469024 shutdown_connections 2024-04-13T05:23:36.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.928+0000 7f923f000640 1 -- 172.21.15.57:0/1202469024 wait complete. 2024-04-13T05:23:36.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 Processor -- start 2024-04-13T05:23:36.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 -- start start 2024-04-13T05:23:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238104100 0x7f923819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92381052f0 0x7f923819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 0x7f92381a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f923810fc90 con 0x7f9238105cb0 2024-04-13T05:23:36.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f923810fb10 con 0x7f9238104100 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.930+0000 7f923f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f923810fe10 con 0x7f92381052f0 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 0x7f92381a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 0x7f92381a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36860/0 (socket says 172.21.15.57:36860) 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 -- 172.21.15.57:0/1568935735 learned_addr learned my addr 172.21.15.57:0/1568935735 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923dc00640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238104100 0x7f923819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923d200640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92381052f0 0x7f923819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 -- 172.21.15.57:0/1568935735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92381052f0 msgr2=0x7f923819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92381052f0 0x7f923819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 -- 172.21.15.57:0/1568935735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238104100 msgr2=0x7f923819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238104100 0x7f923819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.931+0000 7f923e600640 1 -- 172.21.15.57:0/1568935735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92381a1f70 con 0x7f9238105cb0 2024-04-13T05:23:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.932+0000 7f923e600640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 0x7f92381a1900 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f923000e970 tx=0x7f923000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.933+0000 7f9236000640 1 -- 172.21.15.57:0/1568935735 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000ccc0 con 0x7f9238105cb0 2024-04-13T05:23:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.933+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92381a2260 con 0x7f9238105cb0 2024-04-13T05:23:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.933+0000 7f9236000640 1 -- 172.21.15.57:0/1568935735 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92300054d0 con 0x7f9238105cb0 2024-04-13T05:23:36.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.933+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92381a27b0 con 0x7f9238105cb0 2024-04-13T05:23:36.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.935+0000 7f9236000640 1 -- 172.21.15.57:0/1568935735 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000ccc0 con 0x7f9238105cb0 2024-04-13T05:23:36.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.937+0000 7f9243450740 1 Processor -- start 2024-04-13T05:23:36.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9236000640 1 -- 172.21.15.57:0/1568935735 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9230018430 con 0x7f9238105cb0 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 -- start start 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d1e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d1e7b0 0x562fa4d20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562fa4c76020 con 0x562fa4d18c50 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fa4c1c950 con 0x562fa4d19640 2024-04-13T05:23:36.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.939+0000 7f9243450740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562fa4c71fe0 con 0x562fa4d1e7b0 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d19070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36876/0 (socket says 172.21.15.57:36876) 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 -- 172.21.15.57:0/1584973253 learned_addr learned my addr 172.21.15.57:0/1584973253 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923dc00640 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d1e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 -- 172.21.15.57:0/1584973253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d1e7b0 msgr2=0x562fa4d20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d1e7b0 0x562fa4d20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 -- 172.21.15.57:0/1584973253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 msgr2=0x562fa4d1e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d1e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.940+0000 7f923d200640 1 -- 172.21.15.57:0/1584973253 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa49883f0 con 0x562fa4d18c50 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f923d200640 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d19070 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9224009d40 tx=0x7f9224013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2772fe9382a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f9236000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92200a3e30 0x7f92200a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f9236000640 1 -- 172.21.15.57:0/1568935735 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f92300d07a0 con 0x7f9238105cb0 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f922aa00640 1 -- 172.21.15.57:0/1584973253 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922401c470 con 0x562fa4d18c50 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f922aa00640 1 -- 172.21.15.57:0/1584973253 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9224017070 con 0x562fa4d18c50 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f922aa00640 1 -- 172.21.15.57:0/1584973253 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224010850 con 0x562fa4d18c50 2024-04-13T05:23:36.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.941+0000 7f923dc00640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92200a3e30 0x7f92200a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 -- 172.21.15.57:0/1584973253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 msgr2=0x562fa4d19070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f923dc00640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92200a3e30 0x7f92200a62f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f922c02c500 tx=0x7f922c034f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d19070 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9224009d40 tx=0x7f9224013830 comp rx=0 tx=0).stop 2024-04-13T05:23:36.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 -- 172.21.15.57:0/1584973253 shutdown_connections 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d1e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d1e7b0 0x562fa4d20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 --2- 172.21.15.57:0/1584973253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d19070 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 -- 172.21.15.57:0/1584973253 >> 172.21.15.57:0/1584973253 conn(0x562fa4d0b9b0 msgr2=0x562fa4d0bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 -- 172.21.15.57:0/1584973253 shutdown_connections 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.943+0000 7f9243450740 1 -- 172.21.15.57:0/1584973253 wait complete. 2024-04-13T05:23:36.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.944+0000 7f9243450740 1 -- start start 2024-04-13T05:23:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.944+0000 7f9243450740 1 Processor -- start 2024-04-13T05:23:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.944+0000 7f9243450740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d23170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f9243450740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d19640 0x562fa4d236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f9243450740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d1e7b0 0x562fa4d2a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f9243450740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562fa4d21760 con 0x562fa4d18c50 2024-04-13T05:23:36.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f9243450740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562fa4c2fc80 con 0x562fa4d1e7b0 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f9243450740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562fa4d218e0 con 0x562fa4d19640 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f9243450740 10 client.0 _ll_register_callbacks cb 0x562fa4c2f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.945+0000 7f923dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d23170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d23170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36892/0 (socket says 172.21.15.57:36892) 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 -- 172.21.15.57:0/4182271003 learned_addr learned my addr 172.21.15.57:0/4182271003 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923d200640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d19640 0x562fa4d236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923e600640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d1e7b0 0x562fa4d2a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 -- 172.21.15.57:0/4182271003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d19640 msgr2=0x562fa4d236b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d19640 0x562fa4d236b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 -- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d1e7b0 msgr2=0x562fa4d2a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d1e7b0 0x562fa4d2a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.946+0000 7f923dc00640 1 -- 172.21.15.57:0/4182271003 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa4a527d0 con 0x562fa4d18c50 2024-04-13T05:23:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.947+0000 7f923dc00640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d23170 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f922c0099a0 tx=0x7f922c02bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b0ee3cdb108815 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.947+0000 7f921f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.947+0000 7f921f400640 1 -- 172.21.15.57:0/4182271003 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c051020 con 0x562fa4d18c50 2024-04-13T05:23:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.947+0000 7f921f400640 1 -- 172.21.15.57:0/4182271003 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f922c033760 con 0x562fa4d18c50 2024-04-13T05:23:36.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.947+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9224017070 con 0x562fa4d18c50 2024-04-13T05:23:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.948+0000 7f921f400640 1 -- 172.21.15.57:0/4182271003 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f922c05c050 con 0x562fa4d18c50 2024-04-13T05:23:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.948+0000 7f9243450740 10 client.16680 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.948+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fa4d2db00 con 0x562fa4d18c50 2024-04-13T05:23:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f921f400640 1 -- 172.21.15.57:0/4182271003 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f922c02cd60 con 0x562fa4d18c50 2024-04-13T05:23:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f921f400640 1 -- 172.21.15.57:0/4182271003 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f922c04d620 con 0x562fa4d18c50 2024-04-13T05:23:36.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f9243450740 10 client.16680 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f9243450740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f9243450740 20 client.16680 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f9243450740 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4d316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f9243450740 4 client.16680 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9224017070 con 0x562fa4d2f240 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.949+0000 7f923d200640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4d316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.950+0000 7f923d200640 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4d316c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b755fec0005e939 server_cookie=b08aa027f80985d7 in_seq=0 out_seq=0 2024-04-13T05:23:36.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.950+0000 7f921f400640 10 client.16680 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f921f400640 1 -- 172.21.15.57:0/4182271003 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f9224017070 con 0x562fa4d2f240 2024-04-13T05:23:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f921f400640 10 client.16680 handle_command_reply: tid=0 2024-04-13T05:23:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f9243450740 1 client.16680 shutdown 2024-04-13T05:23:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 msgr2=0x562fa4d23170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f9243450740 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d23170 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f922c0099a0 tx=0x7f922c02bef0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 shutdown_connections 2024-04-13T05:23:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f9243450740 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4d316c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f921f400640 0 client.16680 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:36.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.952+0000 7f9243450740 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d1e7b0 0x562fa4d2a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f9243450740 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d19640 0x562fa4d236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f9243450740 1 --2- 172.21.15.57:0/4182271003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d18c50 0x562fa4d23170 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 >> 172.21.15.57:0/4182271003 conn(0x562fa4d0b9b0 msgr2=0x562fa4d0acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 shutdown_connections 2024-04-13T05:23:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f9243450740 1 -- 172.21.15.57:0/4182271003 wait complete. 2024-04-13T05:23:36.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.953+0000 7f9243450740 20 client.16680 trim_cache size 0 max 16384 2024-04-13T05:23:36.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.977+0000 7f9243450740 1 Processor -- start 2024-04-13T05:23:36.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.977+0000 7f9243450740 1 -- start start 2024-04-13T05:23:36.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f9243450740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d23610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f9243450740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d2a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f9243450740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d0bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f9243450740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9218012540 con 0x562fa4d1e7b0 2024-04-13T05:23:36.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f9243450740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921800fdb0 con 0x562fa4d19640 2024-04-13T05:23:36.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f9243450740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f921800dc00 con 0x562fa4d18c50 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f923d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d2a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f923d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d2a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45718/0 (socket says 172.21.15.57:45718) 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f923d200640 1 -- 172.21.15.57:0/3519885961 learned_addr learned my addr 172.21.15.57:0/3519885961 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.978+0000 7f923dc00640 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d23610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923e600640 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d0bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923d200640 1 -- 172.21.15.57:0/3519885961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 msgr2=0x562fa4d23610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923d200640 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d23610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923d200640 1 -- 172.21.15.57:0/3519885961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 msgr2=0x562fa4d0bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923d200640 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d0bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923d200640 1 -- 172.21.15.57:0/3519885961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa4d0c2d0 con 0x562fa4d19640 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.979+0000 7f923e600640 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d0bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.980+0000 7f923d200640 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d2a6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9224013d30 tx=0x7f9224009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0be648284b7e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.980+0000 7f922a000640 1 -- 172.21.15.57:0/3519885961 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240050b0 con 0x562fa4d19640 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.980+0000 7f922a000640 1 -- 172.21.15.57:0/3519885961 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9224017070 con 0x562fa4d19640 2024-04-13T05:23:36.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 -- 172.21.15.57:0/3519885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 msgr2=0x562fa4d2a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d2a6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9224013d30 tx=0x7f9224009850 comp rx=0 tx=0).stop 2024-04-13T05:23:36.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 -- 172.21.15.57:0/3519885961 shutdown_connections 2024-04-13T05:23:36.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d2a6b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d0bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 --2- 172.21.15.57:0/3519885961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d23610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 -- 172.21.15.57:0/3519885961 >> 172.21.15.57:0/3519885961 conn(0x562fa4d0b9b0 msgr2=0x562fa4d06b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.981+0000 7f9243450740 1 -- 172.21.15.57:0/3519885961 shutdown_connections 2024-04-13T05:23:36.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.982+0000 7f9243450740 1 -- 172.21.15.57:0/3519885961 wait complete. 2024-04-13T05:23:36.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.982+0000 7f9243450740 1 -- start start 2024-04-13T05:23:36.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.982+0000 7f9243450740 1 Processor -- start 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d27c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f921800e270 con 0x562fa4d1e7b0 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921800f6e0 con 0x562fa4d19640 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f921800e940 con 0x562fa4d18c50 2024-04-13T05:23:36.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.983+0000 7f9243450740 10 client.0 _ll_register_callbacks cb 0x7f922c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:36.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.984+0000 7f923e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.984+0000 7f923e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d29190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36912/0 (socket says 172.21.15.57:36912) 2024-04-13T05:23:36.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.984+0000 7f923e600640 1 -- 172.21.15.57:0/680965633 learned_addr learned my addr 172.21.15.57:0/680965633 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.984+0000 7f923dc00640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d27c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.984+0000 7f923d200640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923e600640 1 -- 172.21.15.57:0/680965633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 msgr2=0x562fa4d27c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923e600640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d27c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923e600640 1 -- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 msgr2=0x562fa4d28180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923e600640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d28180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923e600640 1 -- 172.21.15.57:0/680965633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa4d29780 con 0x562fa4d1e7b0 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923dc00640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d27c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923d200640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d28180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:36.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f923e600640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d29190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f923009c380 tx=0x7f923000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739a4a22de8b15f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:36.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.985+0000 7f921f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f921f400640 1 -- 172.21.15.57:0/680965633 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923009c680 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f921f400640 1 -- 172.21.15.57:0/680965633 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f92300d0440 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9224017070 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f921f400640 1 -- 172.21.15.57:0/680965633 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92300a0a30 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f9243450740 10 client.16686 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fa4da6850 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.986+0000 7f921f400640 1 -- 172.21.15.57:0/680965633 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9224017070 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f921f400640 1 -- 172.21.15.57:0/680965633 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f92300984f0 con 0x562fa4d1e7b0 2024-04-13T05:23:36.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f9243450740 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f9243450740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f9243450740 20 client.16686 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f9243450740 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4da9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f9243450740 4 client.16686 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.987+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fa4c15750 con 0x562fa4d2f240 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.988+0000 7f923dc00640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4da9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:36.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.988+0000 7f923dc00640 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4da9070 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff6dfd1d1e90165 server_cookie=4df8a20244c73127 in_seq=0 out_seq=0 2024-04-13T05:23:36.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.988+0000 7f921f400640 10 client.16686 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.989+0000 7f921f400640 1 -- 172.21.15.57:0/680965633 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fa4c15750 con 0x562fa4d2f240 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.989+0000 7f921f400640 10 client.16686 handle_command_reply: tid=0 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.989+0000 7f9243450740 1 client.16686 shutdown 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.989+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 msgr2=0x562fa4d29190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.989+0000 7f9243450740 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d29190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f923009c380 tx=0x7f923000ce60 comp rx=0 tx=0).stop 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 shutdown_connections 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562fa4d2f240 0x562fa4da9070 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562fa4d19640 0x562fa4d28180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562fa4d1e7b0 0x562fa4d29190 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 --2- 172.21.15.57:0/680965633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562fa4d18c50 0x562fa4d27c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f921f400640 0 client.16686 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 >> 172.21.15.57:0/680965633 conn(0x562fa4d0b9b0 msgr2=0x562fa4d22bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 shutdown_connections 2024-04-13T05:23:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 1 -- 172.21.15.57:0/680965633 wait complete. 2024-04-13T05:23:36.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.990+0000 7f9243450740 20 client.16686 trim_cache size 0 max 16384 2024-04-13T05:23:36.994 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.991+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92200a3e30 msgr2=0x7f92200a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.991+0000 7f923f000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92200a3e30 0x7f92200a62f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f922c02c500 tx=0x7f922c034f90 comp rx=0 tx=0).stop 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.991+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 msgr2=0x7f92381a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.991+0000 7f923f000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 0x7f92381a1900 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f923000e970 tx=0x7f923000ee30 comp rx=0 tx=0).stop 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 shutdown_connections 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f92200a3e30 0x7f92200a62f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9238105cb0 0x7f92381a1900 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92381052f0 0x7f923819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 --2- 172.21.15.57:0/1568935735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9238104100 0x7f923819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 >> 172.21.15.57:0/1568935735 conn(0x7f92380ff670 msgr2=0x7f9238108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 shutdown_connections 2024-04-13T05:23:36.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:36.992+0000 7f923f000640 1 -- 172.21.15.57:0/1568935735 wait complete. 2024-04-13T05:23:37.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:37 smithi189 ceph-mon[35531]: pgmap v636: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-04-13T05:23:37.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:37 smithi083 ceph-mon[35133]: pgmap v636: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-04-13T05:23:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:37 smithi057 ceph-mon[32331]: pgmap v636: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-04-13T05:23:38.017 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:23:38.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:23:38.018 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:23:38.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 -- 172.21.15.57:0/24932268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958104240 msgr2=0x7f6958104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 --2- 172.21.15.57:0/24932268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958104240 0x7f6958104660 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6948009900 tx=0x7f694802bc30 comp rx=0 tx=0).stop 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 -- 172.21.15.57:0/24932268 shutdown_connections 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 --2- 172.21.15.57:0/24932268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958105430 0x7f69581058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 --2- 172.21.15.57:0/24932268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6958105df0 0x7f695810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 --2- 172.21.15.57:0/24932268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958104240 0x7f6958104660 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 -- 172.21.15.57:0/24932268 >> 172.21.15.57:0/24932268 conn(0x7f69580ff7b0 msgr2=0x7f6958101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 -- 172.21.15.57:0/24932268 shutdown_connections 2024-04-13T05:23:38.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.280+0000 7f695ec00640 1 -- 172.21.15.57:0/24932268 wait complete. 2024-04-13T05:23:38.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.281+0000 7f695ec00640 1 Processor -- start 2024-04-13T05:23:38.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 -- start start 2024-04-13T05:23:38.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 0x7f695819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6958105430 0x7f695819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958105df0 0x7f69581a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f695810fdd0 con 0x7f6958105df0 2024-04-13T05:23:38.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f695810fc50 con 0x7f6958104240 2024-04-13T05:23:38.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.282+0000 7f695ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f695810ff50 con 0x7f6958105430 2024-04-13T05:23:38.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 0x7f695819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 0x7f695819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45754/0 (socket says 172.21.15.57:45754) 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 -- 172.21.15.57:0/1753889434 learned_addr learned my addr 172.21.15.57:0/1753889434 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957400640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6958105430 0x7f695819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f695cc00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958105df0 0x7f69581a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 -- 172.21.15.57:0/1753889434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6958105430 msgr2=0x7f695819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6958105430 0x7f695819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 -- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958105df0 msgr2=0x7f69581a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958105df0 0x7f69581a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.283+0000 7f6957e00640 1 -- 172.21.15.57:0/1753889434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581a64a0 con 0x7f6958104240 2024-04-13T05:23:38.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.284+0000 7f6957e00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 0x7f695819e880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6948000c00 tx=0x7f6948004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.284+0000 7f6954c00640 1 -- 172.21.15.57:0/1753889434 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948043020 con 0x7f6958104240 2024-04-13T05:23:38.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.284+0000 7f6954c00640 1 -- 172.21.15.57:0/1753889434 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69480052b0 con 0x7f6958104240 2024-04-13T05:23:38.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.284+0000 7f6954c00640 1 -- 172.21.15.57:0/1753889434 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948003c70 con 0x7f6958104240 2024-04-13T05:23:38.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.284+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581a6730 con 0x7f6958104240 2024-04-13T05:23:38.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.285+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581a6c80 con 0x7f6958104240 2024-04-13T05:23:38.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.288+0000 7f69619ee740 1 Processor -- start 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 -- start start 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a41e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5c70 0x5622a41ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a41ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5622a414bbc0 con 0x5622a41e5c70 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622a40f2660 con 0x5622a41eade0 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f69619ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622a4147b80 con 0x5622a41e5280 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.289+0000 7f6954c00640 1 -- 172.21.15.57:0/1753889434 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6948046020 con 0x7f6958104240 2024-04-13T05:23:38.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a41ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a41ed270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45756/0 (socket says 172.21.15.57:45756) 2024-04-13T05:23:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957400640 1 -- 172.21.15.57:0/3731514942 learned_addr learned my addr 172.21.15.57:0/3731514942 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957e00640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5c70 0x5622a41ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f695cc00640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a41e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957e00640 1 -- 172.21.15.57:0/3731514942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 msgr2=0x5622a41e56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957e00640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a41e56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957e00640 1 -- 172.21.15.57:0/3731514942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 msgr2=0x5622a41ed270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957e00640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a41ed270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.290+0000 7f6957e00640 1 -- 172.21.15.57:0/3731514942 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a4115220 con 0x5622a41e5c70 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.291+0000 7f695cc00640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a41e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.291+0000 7f6957400640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a41ed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.291+0000 7f6957e00640 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5c70 0x5622a41ea770 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f69480f0220 tx=0x7f69480f6bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e111827f04da6eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.292+0000 7f6954c00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f693c0a3c00 0x7f693c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.292+0000 7f6954c00640 1 -- 172.21.15.57:0/1753889434 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f69480f2b80 con 0x7f6958104240 2024-04-13T05:23:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.292+0000 7f694d200640 1 -- 172.21.15.57:0/3731514942 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480bcba0 con 0x5622a41e5c70 2024-04-13T05:23:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.292+0000 7f694d200640 1 -- 172.21.15.57:0/3731514942 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6948104070 con 0x5622a41e5c70 2024-04-13T05:23:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.292+0000 7f694d200640 1 -- 172.21.15.57:0/3731514942 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480be3c0 con 0x5622a41e5c70 2024-04-13T05:23:38.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.292+0000 7f695cc00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f693c0a3c00 0x7f693c0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 -- 172.21.15.57:0/3731514942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5c70 msgr2=0x5622a41ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f695cc00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f693c0a3c00 0x7f693c0a60c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5622a41e64d0 tx=0x7f6950009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5c70 0x5622a41ea770 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f69480f0220 tx=0x7f69480f6bb0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 -- 172.21.15.57:0/3731514942 shutdown_connections 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a41ed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5c70 0x5622a41ea770 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 --2- 172.21.15.57:0/3731514942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a41e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 -- 172.21.15.57:0/3731514942 >> 172.21.15.57:0/3731514942 conn(0x5622a41e07f0 msgr2=0x5622a41e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 -- 172.21.15.57:0/3731514942 shutdown_connections 2024-04-13T05:23:38.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.294+0000 7f69619ee740 1 -- 172.21.15.57:0/3731514942 wait complete. 2024-04-13T05:23:38.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.295+0000 7f69619ee740 1 -- start start 2024-04-13T05:23:38.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.295+0000 7f69619ee740 1 Processor -- start 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.296+0000 7f69619ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a42004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.296+0000 7f69619ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5c70 0x5622a4200a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f69619ee740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4207a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f69619ee740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5622a41feb60 con 0x5622a41eade0 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f69619ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622a4105a30 con 0x5622a41e5c70 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f69619ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622a41fece0 con 0x5622a41e5280 2024-04-13T05:23:38.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f6957e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a42004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f69619ee740 10 client.0 _ll_register_callbacks cb 0x5622a3e5dc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:38.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f6957e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a42004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:40294/0 (socket says 172.21.15.57:40294) 2024-04-13T05:23:38.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f6957e00640 1 -- 172.21.15.57:0/606020316 learned_addr learned my addr 172.21.15.57:0/606020316 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f695cc00640 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4207a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f695cc00640 1 -- 172.21.15.57:0/606020316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 msgr2=0x5622a42004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f695cc00640 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a42004c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f695cc00640 1 -- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5c70 msgr2=0x5622a4200a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f695cc00640 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5c70 0x5622a4200a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.297+0000 7f695cc00640 1 -- 172.21.15.57:0/606020316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a3f267e0 con 0x5622a41eade0 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.298+0000 7f695cc00640 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4207a40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6950005fd0 tx=0x7f6950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503c04c667cc4dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.298+0000 7f6941600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:38.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.299+0000 7f6941600640 1 -- 172.21.15.57:0/606020316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950022070 con 0x5622a41eade0 2024-04-13T05:23:38.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.299+0000 7f6941600640 1 -- 172.21.15.57:0/606020316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f695000fd00 con 0x5622a41eade0 2024-04-13T05:23:38.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.299+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6948104070 con 0x5622a41eade0 2024-04-13T05:23:38.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.302+0000 7f6941600640 1 -- 172.21.15.57:0/606020316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6950027460 con 0x5622a41eade0 2024-04-13T05:23:38.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.302+0000 7f69619ee740 10 client.16698 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:38.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.302+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622a3f86f70 con 0x5622a41eade0 2024-04-13T05:23:38.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.302+0000 7f6941600640 1 -- 172.21.15.57:0/606020316 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6948104070 con 0x5622a41eade0 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f6941600640 1 -- 172.21.15.57:0/606020316 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f695000f050 con 0x5622a41eade0 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f69619ee740 10 client.16698 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f69619ee740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f69619ee740 20 client.16698 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f69619ee740 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a420c390 0x5622a420e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f69619ee740 4 client.16698 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.303+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a41595e0 con 0x5622a420c390 2024-04-13T05:23:38.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.304+0000 7f6957e00640 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a420c390 0x5622a420e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.305+0000 7f6957e00640 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a420c390 0x5622a420e7c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4947268c174c1a3b server_cookie=10618110a13b12b9 in_seq=0 out_seq=0 2024-04-13T05:23:38.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.305+0000 7f6941600640 10 client.16698 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f6941600640 1 -- 172.21.15.57:0/606020316 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622a41595e0 con 0x5622a420c390 2024-04-13T05:23:38.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f6941600640 10 client.16698 handle_command_reply: tid=0 2024-04-13T05:23:38.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 client.16698 shutdown 2024-04-13T05:23:38.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 msgr2=0x5622a4207a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4207a40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6950005fd0 tx=0x7f6950004a40 comp rx=0 tx=0).stop 2024-04-13T05:23:38.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 shutdown_connections 2024-04-13T05:23:38.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a420c390 0x5622a420e7c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5c70 0x5622a4200a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4207a40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 --2- 172.21.15.57:0/606020316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5280 0x5622a42004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 >> 172.21.15.57:0/606020316 conn(0x5622a41e07f0 msgr2=0x5622a41df9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.306+0000 7f6941600640 0 client.16698 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.307+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 shutdown_connections 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.307+0000 7f69619ee740 1 -- 172.21.15.57:0/606020316 wait complete. 2024-04-13T05:23:38.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.307+0000 7f69619ee740 20 client.16698 trim_cache size 0 max 16384 2024-04-13T05:23:38.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.340+0000 7f69619ee740 1 Processor -- start 2024-04-13T05:23:38.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.340+0000 7f69619ee740 1 -- start start 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f69619ee740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 0x5622a41e4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f69619ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4207a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f69619ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a4213080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f69619ee740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f69581ab290 con 0x5622a41e5280 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f69619ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a8b00 con 0x5622a41eade0 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f69619ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f695819d7c0 con 0x5622a41e5c70 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f6957e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 0x5622a41e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f6957e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 0x5622a41e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36942/0 (socket says 172.21.15.57:36942) 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f6957e00640 1 -- 172.21.15.57:0/1801048569 learned_addr learned my addr 172.21.15.57:0/1801048569 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.341+0000 7f695cc00640 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a4213080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957400640 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4207a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957e00640 1 -- 172.21.15.57:0/1801048569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 msgr2=0x5622a4207a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957e00640 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4207a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957e00640 1 -- 172.21.15.57:0/1801048569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 msgr2=0x5622a4213080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957e00640 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a4213080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957e00640 1 -- 172.21.15.57:0/1801048569 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a3e1e170 con 0x5622a41e5280 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f695cc00640 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a4213080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6957e00640 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 0x5622a41e4160 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f69480f0280 tx=0x7f69480df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b82bca99a7af34 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.342+0000 7f6943e00640 1 -- 172.21.15.57:0/1801048569 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480be5a0 con 0x5622a41e5280 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.343+0000 7f6943e00640 1 -- 172.21.15.57:0/1801048569 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6948104040 con 0x5622a41e5280 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.343+0000 7f6943e00640 1 -- 172.21.15.57:0/1801048569 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480be830 con 0x5622a41e5280 2024-04-13T05:23:38.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 -- 172.21.15.57:0/1801048569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 msgr2=0x5622a41e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 0x5622a41e4160 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f69480f0280 tx=0x7f69480df040 comp rx=0 tx=0).stop 2024-04-13T05:23:38.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 -- 172.21.15.57:0/1801048569 shutdown_connections 2024-04-13T05:23:38.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41eade0 0x5622a4213080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4207a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 --2- 172.21.15.57:0/1801048569 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41e5280 0x5622a41e4160 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.344+0000 7f69619ee740 1 -- 172.21.15.57:0/1801048569 >> 172.21.15.57:0/1801048569 conn(0x5622a41e07f0 msgr2=0x5622a41e3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.345+0000 7f69619ee740 1 -- 172.21.15.57:0/1801048569 shutdown_connections 2024-04-13T05:23:38.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.345+0000 7f69619ee740 1 -- 172.21.15.57:0/1801048569 wait complete. 2024-04-13T05:23:38.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.345+0000 7f69619ee740 1 -- start start 2024-04-13T05:23:38.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.346+0000 7f69619ee740 1 Processor -- start 2024-04-13T05:23:38.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.346+0000 7f69619ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 0x5622a4205310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.346+0000 7f69619ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4205850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f69619ee740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4206860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f69619ee740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a6fc0 con 0x5622a41eade0 2024-04-13T05:23:38.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f69619ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a8430 con 0x5622a41e5280 2024-04-13T05:23:38.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f69619ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a7690 con 0x5622a41e5c70 2024-04-13T05:23:38.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f6957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 0x5622a4205310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f69619ee740 10 client.0 _ll_register_callbacks cb 0x7f69500093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f6957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 0x5622a4205310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45790/0 (socket says 172.21.15.57:45790) 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f6957e00640 1 -- 172.21.15.57:0/3772995386 learned_addr learned my addr 172.21.15.57:0/3772995386 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f6957e00640 1 -- 172.21.15.57:0/3772995386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 msgr2=0x5622a4205850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.347+0000 7f6957e00640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4205850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.348+0000 7f6957e00640 1 -- 172.21.15.57:0/3772995386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 msgr2=0x5622a4206860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.348+0000 7f695cc00640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4206860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.348+0000 7f6957e00640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4206860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.348+0000 7f6957e00640 1 -- 172.21.15.57:0/3772995386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a4106be0 con 0x5622a41e5280 2024-04-13T05:23:38.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.348+0000 7f695cc00640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4206860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:38.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.350+0000 7f6957e00640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 0x5622a4205310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69480096c0 tx=0x7f69480fb5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3c0e758ec4aae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.350+0000 7f6941600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:38.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.350+0000 7f6941600640 1 -- 172.21.15.57:0/3772995386 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480bac70 con 0x5622a41e5280 2024-04-13T05:23:38.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.350+0000 7f6941600640 1 -- 172.21.15.57:0/3772995386 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f69480f6e00 con 0x5622a41e5280 2024-04-13T05:23:38.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.350+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6948104040 con 0x5622a41e5280 2024-04-13T05:23:38.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.351+0000 7f6941600640 1 -- 172.21.15.57:0/3772995386 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69480bcba0 con 0x5622a41e5280 2024-04-13T05:23:38.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.351+0000 7f69619ee740 10 client.36104 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:38.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.351+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5622a4272900 con 0x5622a41e5280 2024-04-13T05:23:38.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.351+0000 7f6941600640 1 -- 172.21.15.57:0/3772995386 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6948088070 con 0x5622a41e5280 2024-04-13T05:23:38.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f6941600640 1 -- 172.21.15.57:0/3772995386 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f69480f0300 con 0x5622a41e5280 2024-04-13T05:23:38.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f69619ee740 10 client.36104 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:38.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f69619ee740 10 client.36104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:38.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f69619ee740 20 client.36104 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:38.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f69619ee740 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a4272eb0 0x5622a42752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f69619ee740 4 client.36104 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:23:38.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5622a4153450 con 0x5622a4272eb0 2024-04-13T05:23:38.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.352+0000 7f6957400640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a4272eb0 0x5622a42752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.353+0000 7f6957400640 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a4272eb0 0x5622a42752d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b926ee8471518ae9 server_cookie=85c1dafe5d1b9c2a in_seq=0 out_seq=0 2024-04-13T05:23:38.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.354+0000 7f6941600640 10 client.36104 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f6941600640 1 -- 172.21.15.57:0/3772995386 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5622a4153450 con 0x5622a4272eb0 2024-04-13T05:23:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f6941600640 10 client.36104 handle_command_reply: tid=0 2024-04-13T05:23:38.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 client.36104 shutdown 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 msgr2=0x5622a4205310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 0x5622a4205310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69480096c0 tx=0x7f69480fb5d0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 shutdown_connections 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5622a4272eb0 0x5622a42752d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5622a41eade0 0x5622a4206860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a41e5c70 0x5622a4205850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f6941600640 0 client.36104 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 --2- 172.21.15.57:0/3772995386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a41e5280 0x5622a4205310 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.358+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 >> 172.21.15.57:0/3772995386 conn(0x5622a41e07f0 msgr2=0x5622a41e42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.359+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 shutdown_connections 2024-04-13T05:23:38.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.359+0000 7f69619ee740 1 -- 172.21.15.57:0/3772995386 wait complete. 2024-04-13T05:23:38.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.359+0000 7f69619ee740 20 client.36104 trim_cache size 0 max 16384 2024-04-13T05:23:38.362 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:23:38.362 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:23:38.363 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "5dfbee8f-a7d4-4137-ba87-b2e6915db00e", 2024-04-13T05:23:38.363 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:23:38.363 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:23:38.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f693c0a3c00 msgr2=0x7f693c0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f693c0a3c00 0x7f693c0a60c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5622a41e64d0 tx=0x7f6950009500 comp rx=0 tx=0).stop 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 msgr2=0x7f695819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 0x7f695819e880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6948000c00 tx=0x7f6948004270 comp rx=0 tx=0).stop 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 shutdown_connections 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f693c0a3c00 0x7f693c0a60c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6958105df0 0x7f69581a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6958105430 0x7f695819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 --2- 172.21.15.57:0/1753889434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6958104240 0x7f695819e880 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 >> 172.21.15.57:0/1753889434 conn(0x7f69580ff7b0 msgr2=0x7f6958078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 shutdown_connections 2024-04-13T05:23:38.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.360+0000 7f695ec00640 1 -- 172.21.15.57:0/1753889434 wait complete. 2024-04-13T05:23:38.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:23:38.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:23:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.653+0000 7f3a57e00640 1 -- 172.21.15.57:0/1999567206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a5810a0b0 msgr2=0x7f3a58110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.653+0000 7f3a57e00640 1 --2- 172.21.15.57:0/1999567206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a5810a0b0 0x7f3a58110940 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009ca0 tx=0x7f3a5002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 -- 172.21.15.57:0/1999567206 shutdown_connections 2024-04-13T05:23:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 --2- 172.21.15.57:0/1999567206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a5810a0b0 0x7f3a58110940 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 --2- 172.21.15.57:0/1999567206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a581096f0 0x7f3a58109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 --2- 172.21.15.57:0/1999567206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a58108500 0x7f3a58108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 -- 172.21.15.57:0/1999567206 >> 172.21.15.57:0/1999567206 conn(0x7f3a58077490 msgr2=0x7f3a580798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 -- 172.21.15.57:0/1999567206 shutdown_connections 2024-04-13T05:23:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.654+0000 7f3a57e00640 1 -- 172.21.15.57:0/1999567206 wait complete. 2024-04-13T05:23:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 Processor -- start 2024-04-13T05:23:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 -- start start 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a58108500 0x7f3a5807f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 0x7f3a5807fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a5810a0b0 0x7f3a581ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58114040 con 0x7f3a581096f0 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58113ec0 con 0x7f3a58108500 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.656+0000 7f3a57e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581141c0 con 0x7f3a5810a0b0 2024-04-13T05:23:38.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.657+0000 7f3a4e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 0x7f3a5807fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.657+0000 7f3a4e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 0x7f3a5807fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36966/0 (socket says 172.21.15.57:36966) 2024-04-13T05:23:38.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.657+0000 7f3a4e000640 1 -- 172.21.15.57:0/70689805 learned_addr learned my addr 172.21.15.57:0/70689805 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.657+0000 7f3a56a00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a58108500 0x7f3a5807f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a57400640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a5810a0b0 0x7f3a581ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a4e000640 1 -- 172.21.15.57:0/70689805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a5810a0b0 msgr2=0x7f3a581ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a4e000640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a5810a0b0 0x7f3a581ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a4e000640 1 -- 172.21.15.57:0/70689805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a58108500 msgr2=0x7f3a5807f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a4e000640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a58108500 0x7f3a5807f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a4e000640 1 -- 172.21.15.57:0/70689805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a581aec70 con 0x7f3a581096f0 2024-04-13T05:23:38.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.658+0000 7f3a4e000640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 0x7f3a5807fd10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000d770 tx=0x7f3a4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.659+0000 7f3a4fe00640 1 -- 172.21.15.57:0/70689805 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40017070 con 0x7f3a581096f0 2024-04-13T05:23:38.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.659+0000 7f3a4fe00640 1 -- 172.21.15.57:0/70689805 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a400054d0 con 0x7f3a581096f0 2024-04-13T05:23:38.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.659+0000 7f3a4fe00640 1 -- 172.21.15.57:0/70689805 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40004cf0 con 0x7f3a581096f0 2024-04-13T05:23:38.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.659+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a581aee40 con 0x7f3a581096f0 2024-04-13T05:23:38.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.659+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a581af250 con 0x7f3a581096f0 2024-04-13T05:23:38.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.667+0000 7f3a60a03740 1 Processor -- start 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.669+0000 7f3a60a03740 1 -- start start 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.669+0000 7f3a4fe00640 1 -- 172.21.15.57:0/70689805 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3a40004e90 con 0x7f3a581096f0 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.669+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d4e6f0 0x55a817d4eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d53be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d54250 0x55a817d566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a60a03740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a817cb5020 con 0x55a817d4e6f0 2024-04-13T05:23:38.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a60a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a817c5b6d0 con 0x55a817d4f0e0 2024-04-13T05:23:38.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a60a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a817cb0fe0 con 0x55a817d54250 2024-04-13T05:23:38.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d53be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d53be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45816/0 (socket says 172.21.15.57:45816) 2024-04-13T05:23:38.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a56a00640 1 -- 172.21.15.57:0/2873840233 learned_addr learned my addr 172.21.15.57:0/2873840233 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.670+0000 7f3a57400640 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d4e6f0 0x55a817d4eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a4e000640 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d54250 0x55a817d566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a57400640 1 -- 172.21.15.57:0/2873840233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d54250 msgr2=0x55a817d566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a57400640 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d54250 0x55a817d566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a57400640 1 -- 172.21.15.57:0/2873840233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 msgr2=0x55a817d53be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a57400640 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d53be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a57400640 1 -- 172.21.15.57:0/2873840233 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a817a917d0 con 0x55a817d4e6f0 2024-04-13T05:23:38.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.671+0000 7f3a57400640 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d4e6f0 0x55a817d4eb10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3a50037040 tx=0x7f3a50005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e7aa19ad03841f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.672+0000 7f3a47e00640 1 -- 172.21.15.57:0/2873840233 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50043020 con 0x55a817d4e6f0 2024-04-13T05:23:38.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.673+0000 7f3a47e00640 1 -- 172.21.15.57:0/2873840233 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a50032db0 con 0x55a817d4e6f0 2024-04-13T05:23:38.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.673+0000 7f3a47e00640 1 -- 172.21.15.57:0/2873840233 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50039040 con 0x55a817d4e6f0 2024-04-13T05:23:38.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.673+0000 7f3a4fe00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a3c0a3bb0 0x7f3a3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.673+0000 7f3a4fe00640 1 -- 172.21.15.57:0/70689805 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3a400cff10 con 0x7f3a581096f0 2024-04-13T05:23:38.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.673+0000 7f3a56a00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a3c0a3bb0 0x7f3a3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.674+0000 7f3a60a03740 1 -- 172.21.15.57:0/2873840233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d4e6f0 msgr2=0x55a817d4eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.674+0000 7f3a60a03740 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d4e6f0 0x55a817d4eb10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3a50037040 tx=0x7f3a50005fa0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.674+0000 7f3a56a00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a3c0a3bb0 0x7f3a3c0a6070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55a817d4f4e0 tx=0x7f3a4800b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.675+0000 7f3a60a03740 1 -- 172.21.15.57:0/2873840233 shutdown_connections 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.675+0000 7f3a60a03740 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d53be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.675+0000 7f3a60a03740 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d54250 0x55a817d566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.675+0000 7f3a60a03740 1 --2- 172.21.15.57:0/2873840233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d4e6f0 0x55a817d4eb10 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.675+0000 7f3a60a03740 1 -- 172.21.15.57:0/2873840233 >> 172.21.15.57:0/2873840233 conn(0x55a817d49c60 msgr2=0x55a817d4a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.676+0000 7f3a60a03740 1 -- 172.21.15.57:0/2873840233 shutdown_connections 2024-04-13T05:23:38.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.676+0000 7f3a60a03740 1 -- 172.21.15.57:0/2873840233 wait complete. 2024-04-13T05:23:38.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.676+0000 7f3a60a03740 1 -- start start 2024-04-13T05:23:38.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.677+0000 7f3a60a03740 1 Processor -- start 2024-04-13T05:23:38.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 0x55a817d58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a817d571a0 con 0x55a817d54250 2024-04-13T05:23:38.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a817c6ea90 con 0x55a817d4e6f0 2024-04-13T05:23:38.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a817d57320 con 0x55a817d4f0e0 2024-04-13T05:23:38.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.678+0000 7f3a60a03740 10 client.0 _ll_register_callbacks cb 0x55a817c6e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:38.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 0x55a817d58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 0x55a817d58a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:45820/0 (socket says 172.21.15.57:45820) 2024-04-13T05:23:38.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 -- 172.21.15.57:0/217009662 learned_addr learned my addr 172.21.15.57:0/217009662 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a4e000640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 -- 172.21.15.57:0/217009662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 msgr2=0x55a817d58f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a57400640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 -- 172.21.15.57:0/217009662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 msgr2=0x55a817d5ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a56a00640 1 -- 172.21.15.57:0/217009662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8179c73f0 con 0x55a817d4e6f0 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.679+0000 7f3a4e000640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.680+0000 7f3a57400640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.680+0000 7f3a44c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:23:38.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.680+0000 7f3a56a00640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 0x55a817d58a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3a48010660 tx=0x7f3a480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f141289994a01279 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.681+0000 7f3a44c00640 1 -- 172.21.15.57:0/217009662 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48004220 con 0x55a817d4e6f0 2024-04-13T05:23:38.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.681+0000 7f3a44c00640 1 -- 172.21.15.57:0/217009662 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3a480043c0 con 0x55a817d4e6f0 2024-04-13T05:23:38.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.681+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a50032db0 con 0x55a817d4e6f0 2024-04-13T05:23:38.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.682+0000 7f3a44c00640 1 -- 172.21.15.57:0/217009662 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a4801f780 con 0x55a817d4e6f0 2024-04-13T05:23:38.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.682+0000 7f3a60a03740 10 client.36116 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:38.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.682+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a8179870f0 con 0x55a817d4e6f0 2024-04-13T05:23:38.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a44c00640 1 -- 172.21.15.57:0/217009662 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a50032db0 con 0x55a817d4e6f0 2024-04-13T05:23:38.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a44c00640 1 -- 172.21.15.57:0/217009662 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a48003c60 con 0x55a817d4e6f0 2024-04-13T05:23:38.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a60a03740 10 client.36116 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:38.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a60a03740 10 client.36116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:38.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a60a03740 20 client.36116 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:38.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a60a03740 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817d648d0 0x55a817d66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.683+0000 7f3a60a03740 4 client.36116 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:23:38.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.684+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a817c54600 con 0x55a817d648d0 2024-04-13T05:23:38.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.685+0000 7f3a4e000640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817d648d0 0x55a817d66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.687+0000 7f3a4e000640 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817d648d0 0x55a817d66d90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b489091651872a server_cookie=639b7a4ada872773 in_seq=0 out_seq=0 2024-04-13T05:23:38.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.688+0000 7f3a44c00640 10 client.36116 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.689+0000 7f3a44c00640 1 -- 172.21.15.57:0/217009662 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a817c54600 con 0x55a817d648d0 2024-04-13T05:23:38.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.689+0000 7f3a44c00640 10 client.36116 handle_command_reply: tid=0 2024-04-13T05:23:38.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.689+0000 7f3a60a03740 1 client.36116 shutdown 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 msgr2=0x55a817d58a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 0x55a817d58a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3a48010660 tx=0x7f3a480046c0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 shutdown_connections 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817d648d0 0x55a817d66d90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a44c00640 0 client.36116 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 --2- 172.21.15.57:0/217009662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4e6f0 0x55a817d58a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 >> 172.21.15.57:0/217009662 conn(0x55a817d49c60 msgr2=0x55a817d48e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.690+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 shutdown_connections 2024-04-13T05:23:38.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.691+0000 7f3a60a03740 1 -- 172.21.15.57:0/217009662 wait complete. 2024-04-13T05:23:38.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.691+0000 7f3a60a03740 20 client.36116 trim_cache size 0 max 16384 2024-04-13T05:23:38.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.720+0000 7f3a60a03740 1 Processor -- start 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.720+0000 7f3a60a03740 1 -- start start 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d648d0 0x55a817d58b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a60a03740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581bc150 con 0x55a817d54250 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a60a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581b99c0 con 0x55a817d648d0 2024-04-13T05:23:38.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a60a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581b7810 con 0x55a817d4f0e0 2024-04-13T05:23:38.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:40340/0 (socket says 172.21.15.57:40340) 2024-04-13T05:23:38.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.721+0000 7f3a56a00640 1 -- 172.21.15.57:0/136869047 learned_addr learned my addr 172.21.15.57:0/136869047 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.722+0000 7f3a4e000640 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.722+0000 7f3a57400640 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d648d0 0x55a817d58b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.723+0000 7f3a57400640 1 -- 172.21.15.57:0/136869047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 msgr2=0x55a817d58610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.723+0000 7f3a57400640 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.723+0000 7f3a57400640 1 -- 172.21.15.57:0/136869047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 msgr2=0x55a817d5ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.723+0000 7f3a57400640 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.723+0000 7f3a57400640 1 -- 172.21.15.57:0/136869047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a817d4a9d0 con 0x55a817d648d0 2024-04-13T05:23:38.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.723+0000 7f3a57400640 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d648d0 0x55a817d58b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004af0 tx=0x7f3a50000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615ba09b87adab8f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.724+0000 7f3a47400640 1 -- 172.21.15.57:0/136869047 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50043030 con 0x55a817d648d0 2024-04-13T05:23:38.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.724+0000 7f3a47400640 1 -- 172.21.15.57:0/136869047 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a50032db0 con 0x55a817d648d0 2024-04-13T05:23:38.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.724+0000 7f3a47400640 1 -- 172.21.15.57:0/136869047 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50039070 con 0x55a817d648d0 2024-04-13T05:23:38.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 -- 172.21.15.57:0/136869047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d648d0 msgr2=0x55a817d58b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d648d0 0x55a817d58b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004af0 tx=0x7f3a50000f40 comp rx=0 tx=0).stop 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 -- 172.21.15.57:0/136869047 shutdown_connections 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d648d0 0x55a817d58b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 --2- 172.21.15.57:0/136869047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d4f0e0 0x55a817d58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.725+0000 7f3a60a03740 1 -- 172.21.15.57:0/136869047 >> 172.21.15.57:0/136869047 conn(0x55a817d49c60 msgr2=0x55a817d504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.726+0000 7f3a60a03740 1 -- 172.21.15.57:0/136869047 shutdown_connections 2024-04-13T05:23:38.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.726+0000 7f3a60a03740 1 -- 172.21.15.57:0/136869047 wait complete. 2024-04-13T05:23:38.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.726+0000 7f3a60a03740 1 -- start start 2024-04-13T05:23:38.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 Processor -- start 2024-04-13T05:23:38.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d5b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d648d0 0x55a817d5e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581b7e80 con 0x55a817d54250 2024-04-13T05:23:38.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581b92f0 con 0x55a817d4f0e0 2024-04-13T05:23:38.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a581b8550 con 0x55a817d648d0 2024-04-13T05:23:38.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.727+0000 7f3a60a03740 10 client.0 _ll_register_callbacks cb 0x7f3a4800ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:23:38.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.728+0000 7f3a4e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.728+0000 7f3a4e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36986/0 (socket says 172.21.15.57:36986) 2024-04-13T05:23:38.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.728+0000 7f3a4e000640 1 -- 172.21.15.57:0/1243254608 learned_addr learned my addr 172.21.15.57:0/1243254608 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.728+0000 7f3a56a00640 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d5b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a4e000640 1 -- 172.21.15.57:0/1243254608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d648d0 msgr2=0x55a817d5e740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a4e000640 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d648d0 0x55a817d5e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a4e000640 1 -- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 msgr2=0x55a817d5b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a4e000640 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d5b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a4e000640 1 -- 172.21.15.57:0/1243254608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a817d5ed30 con 0x55a817d54250 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a4e000640 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5b720 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3a400982b0 tx=0x7f3a400cd540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315c60da0a4442f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a44c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.729+0000 7f3a44c00640 1 -- 172.21.15.57:0/1243254608 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40020040 con 0x55a817d54250 2024-04-13T05:23:38.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.730+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a50032db0 con 0x55a817d54250 2024-04-13T05:23:38.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.730+0000 7f3a44c00640 1 -- 172.21.15.57:0/1243254608 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3a400984c0 con 0x55a817d54250 2024-04-13T05:23:38.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.730+0000 7f3a44c00640 1 -- 172.21.15.57:0/1243254608 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a40009c30 con 0x55a817d54250 2024-04-13T05:23:38.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.730+0000 7f3a60a03740 10 client.16716 fetch_fsmap learned FSMap version 29 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.730+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a817d600b0 con 0x55a817d54250 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a44c00640 1 -- 172.21.15.57:0/1243254608 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a50032db0 con 0x55a817d54250 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a44c00640 1 -- 172.21.15.57:0/1243254608 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a4009b2a0 con 0x55a817d54250 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a60a03740 10 client.16716 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a60a03740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a60a03740 20 client.16716 populate_metadata read hostname 'smithi057' 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a60a03740 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817ddc580 0x55a817dde9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a60a03740 4 client.16716 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.731+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a817cc2a40 con 0x55a817ddc580 2024-04-13T05:23:38.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.732+0000 7f3a56a00640 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817ddc580 0x55a817dde9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:23:38.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.733+0000 7f3a56a00640 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817ddc580 0x55a817dde9a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=840c4e8077bcc5a3 server_cookie=7d44f1c67da83f9e in_seq=0 out_seq=0 2024-04-13T05:23:38.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.733+0000 7f3a44c00640 10 client.16716 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.736+0000 7f3a44c00640 1 -- 172.21.15.57:0/1243254608 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55a817cc2a40 con 0x55a817ddc580 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.736+0000 7f3a44c00640 10 client.16716 handle_command_reply: tid=0 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.736+0000 7f3a60a03740 1 client.16716 shutdown 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.736+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 msgr2=0x55a817d5b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.736+0000 7f3a60a03740 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5b720 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3a400982b0 tx=0x7f3a400cd540 comp rx=0 tx=0).stop 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 shutdown_connections 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a817ddc580 0x55a817dde9a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a817d648d0 0x55a817d5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a817d54250 0x55a817d5b720 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 --2- 172.21.15.57:0/1243254608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a817d4f0e0 0x55a817d5b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 >> 172.21.15.57:0/1243254608 conn(0x55a817d49c60 msgr2=0x55a817d585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a44c00640 0 client.16716 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:23:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 shutdown_connections 2024-04-13T05:23:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 1 -- 172.21.15.57:0/1243254608 wait complete. 2024-04-13T05:23:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.737+0000 7f3a60a03740 20 client.16716 trim_cache size 0 max 16384 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1456 inodes in the stack)", 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: "5dfbee8f-a7d4-4137-ba87-b2e6915db00e": { 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "5dfbee8f-a7d4-4137-ba87-b2e6915db00e", 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:23:38.741 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.738+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a3c0a3bb0 msgr2=0x7f3a3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.738+0000 7f3a57e00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a3c0a3bb0 0x7f3a3c0a6070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55a817d4f4e0 tx=0x7f3a4800b580 comp rx=0 tx=0).stop 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.738+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 msgr2=0x7f3a5807fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.738+0000 7f3a57e00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 0x7f3a5807fd10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000d770 tx=0x7f3a4000dc30 comp rx=0 tx=0).stop 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 shutdown_connections 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3a3c0a3bb0 0x7f3a3c0a6070 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a5810a0b0 0x7f3a581ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a581096f0 0x7f3a5807fd10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 --2- 172.21.15.57:0/70689805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a58108500 0x7f3a5807f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:23:38.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 >> 172.21.15.57:0/70689805 conn(0x7f3a58077490 msgr2=0x7f3a5810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:23:38.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 shutdown_connections 2024-04-13T05:23:38.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:23:38.739+0000 7f3a57e00640 1 -- 172.21.15.57:0/70689805 wait complete. 2024-04-13T05:23:38.765 INFO:tasks.cephfs.filesystem:scrub status for tag:5dfbee8f-a7d4-4137-ba87-b2e6915db00e - {'path': '/', 'tag': '5dfbee8f-a7d4-4137-ba87-b2e6915db00e', 'options': 'recursive,force'} 2024-04-13T05:23:39.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:39 smithi189 ceph-mon[35531]: pgmap v637: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:23:39.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:39 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:39.638 INFO:tasks.workunit.client.0.smithi057.stdout: 51 7629 34 4762 144 2663 79 2024-04-13T05:23:39.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:39 smithi083 ceph-mon[35133]: pgmap v637: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:23:39.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:39 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:39.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:39 smithi057 ceph-mon[32331]: pgmap v637: 129 pgs: 129 active+clean; 343 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-13T05:23:39.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:39 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:40.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:40 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:23:40.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:40.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:40.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:40.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:40 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:23:40.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:40.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:40.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:40.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:40.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:40 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:41 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:41 smithi189 ceph-mon[35531]: pgmap v638: 129 pgs: 129 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-04-13T05:23:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:41 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:23:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:41 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:41.652 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:41 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:41.652 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:41 smithi083 ceph-mon[35133]: pgmap v638: 129 pgs: 129 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-04-13T05:23:41.652 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:41 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:23:41.653 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:41 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:41.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:41 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:41.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:41 smithi057 ceph-mon[32331]: pgmap v638: 129 pgs: 129 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-04-13T05:23:41.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:41 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:23:41.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:41 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:42.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:42 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:42.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:42 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:42.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:42 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:23:43.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:43 smithi189 ceph-mon[35531]: pgmap v639: 129 pgs: 129 active+clean; 352 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-04-13T05:23:43.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:43 smithi083 ceph-mon[35133]: pgmap v639: 129 pgs: 129 active+clean; 352 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-04-13T05:23:43.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:43 smithi057 ceph-mon[32331]: pgmap v639: 129 pgs: 129 active+clean; 352 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-04-13T05:23:45.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:45 smithi083 ceph-mon[35133]: pgmap v640: 129 pgs: 129 active+clean; 355 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-04-13T05:23:45.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:45 smithi057 ceph-mon[32331]: pgmap v640: 129 pgs: 129 active+clean; 355 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-04-13T05:23:45.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:45 smithi189 ceph-mon[35531]: pgmap v640: 129 pgs: 129 active+clean; 355 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-04-13T05:23:47.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:47 smithi083 ceph-mon[35133]: pgmap v641: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-04-13T05:23:47.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:47 smithi057 ceph-mon[32331]: pgmap v641: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-04-13T05:23:47.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:47 smithi189 ceph-mon[35531]: pgmap v641: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-04-13T05:23:49.638 INFO:tasks.workunit.client.0.smithi057.stdout: 52 8820 156 5585 45 2551 70 2024-04-13T05:23:49.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:49 smithi083 ceph-mon[35133]: pgmap v642: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-04-13T05:23:49.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:49 smithi057 ceph-mon[32331]: pgmap v642: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-04-13T05:23:49.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:49 smithi189 ceph-mon[35531]: pgmap v642: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-04-13T05:23:51.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:51 smithi083 ceph-mon[35133]: pgmap v643: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-13T05:23:51.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:51 smithi057 ceph-mon[32331]: pgmap v643: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-13T05:23:51.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:51 smithi189 ceph-mon[35531]: pgmap v643: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-04-13T05:23:52.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:52 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:23:52.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:52 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:23:52.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:52 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:23:52.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:52 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:23:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:52 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:23:52.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:52 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:23:53.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:53 smithi057 ceph-mon[32331]: pgmap v644: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-13T05:23:53.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:53 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:23:53.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:53 smithi083 ceph-mon[35133]: pgmap v644: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-13T05:23:53.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:53 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:23:53.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:53 smithi189 ceph-mon[35531]: pgmap v644: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-13T05:23:53.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:53 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:23:55.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:55 smithi083 ceph-mon[35133]: pgmap v645: 129 pgs: 129 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2024-04-13T05:23:55.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:55 smithi057 ceph-mon[32331]: pgmap v645: 129 pgs: 129 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2024-04-13T05:23:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:55 smithi189 ceph-mon[35531]: pgmap v645: 129 pgs: 129 active+clean; 374 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2024-04-13T05:23:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:57 smithi083 ceph-mon[35133]: pgmap v646: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2024-04-13T05:23:57.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:57 smithi057 ceph-mon[32331]: pgmap v646: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2024-04-13T05:23:57.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:57 smithi189 ceph-mon[35531]: pgmap v646: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2024-04-13T05:23:59.635 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:23:59 smithi189 ceph-mon[35531]: pgmap v647: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-04-13T05:23:59.638 INFO:tasks.workunit.client.0.smithi057.stdout: 53 8420 52 5663 95 2398 112 2024-04-13T05:23:59.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:23:59 smithi083 ceph-mon[35133]: pgmap v647: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-04-13T05:23:59.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:23:59 smithi057 ceph-mon[32331]: pgmap v647: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-04-13T05:24:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:01 smithi083 ceph-mon[35133]: pgmap v648: 129 pgs: 129 active+clean; 386 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-04-13T05:24:01.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:01 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:24:01.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:01 smithi057 ceph-mon[32331]: pgmap v648: 129 pgs: 129 active+clean; 386 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-04-13T05:24:01.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:01 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:24:01.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:01 smithi189 ceph-mon[35531]: pgmap v648: 129 pgs: 129 active+clean; 386 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-04-13T05:24:01.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:01 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:24:02.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:02 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:02 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:02.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:02 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-13T05:24:03.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: pgmap v649: 129 pgs: 129 active+clean; 386 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-13T05:24:03.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-13T05:24:03.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-13T05:24:03.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-13T05:24:03.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-13T05:24:03.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:24:03.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:24:03.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: pgmap v649: 129 pgs: 129 active+clean; 386 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:24:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:24:03.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: pgmap v649: 129 pgs: 129 active+clean; 386 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-13T05:24:03.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-13T05:24:03.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-13T05:24:03.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-13T05:24:03.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-13T05:24:03.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:24:03.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:24:03.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:24:04.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:04 smithi083 ceph-mon[35133]: pgmap v650: 129 pgs: 129 active+clean; 388 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-04-13T05:24:04.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:04 smithi057 ceph-mon[32331]: pgmap v650: 129 pgs: 129 active+clean; 388 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-04-13T05:24:04.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:04 smithi189 ceph-mon[35531]: pgmap v650: 129 pgs: 129 active+clean; 388 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-04-13T05:24:07.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:06 smithi083 ceph-mon[35133]: pgmap v651: 129 pgs: 129 active+clean; 393 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-04-13T05:24:07.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:06 smithi057 ceph-mon[32331]: pgmap v651: 129 pgs: 129 active+clean; 393 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-04-13T05:24:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:06 smithi189 ceph-mon[35531]: pgmap v651: 129 pgs: 129 active+clean; 393 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-04-13T05:24:08.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:24:08.767 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:24:09.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 -- 172.21.15.57:0/1566042063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b4101aa0 msgr2=0x7fa9b4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/1566042063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b4101aa0 0x7fa9b4101f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009860 tx=0x7fa9a402bb70 comp rx=0 tx=0).stop 2024-04-13T05:24:09.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 -- 172.21.15.57:0/1566042063 shutdown_connections 2024-04-13T05:24:09.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/1566042063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b4101aa0 0x7fa9b4101f20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/1566042063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101100 0x7fa9b4101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/1566042063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b41007a0 0x7fa9b4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.033+0000 7fa9b8a00640 1 -- 172.21.15.57:0/1566042063 >> 172.21.15.57:0/1566042063 conn(0x7fa9b40fc360 msgr2=0x7fa9b40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.034+0000 7fa9b8a00640 1 -- 172.21.15.57:0/1566042063 shutdown_connections 2024-04-13T05:24:09.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.034+0000 7fa9b8a00640 1 -- 172.21.15.57:0/1566042063 wait complete. 2024-04-13T05:24:09.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 Processor -- start 2024-04-13T05:24:09.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 -- start start 2024-04-13T05:24:09.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b41007a0 0x7fa9b419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b4101100 0x7fa9b419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 0x7fa9b41a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b410fc70 con 0x7fa9b4101aa0 2024-04-13T05:24:09.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b410faf0 con 0x7fa9b41007a0 2024-04-13T05:24:09.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b410fdf0 con 0x7fa9b4101100 2024-04-13T05:24:09.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.036+0000 7fa9b3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 0x7fa9b41a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 0x7fa9b41a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59970/0 (socket says 172.21.15.57:59970) 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 -- 172.21.15.57:0/2732524723 learned_addr learned my addr 172.21.15.57:0/2732524723 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3400640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b41007a0 0x7fa9b419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b4101100 0x7fa9b419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 -- 172.21.15.57:0/2732524723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b4101100 msgr2=0x7fa9b419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b4101100 0x7fa9b419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 -- 172.21.15.57:0/2732524723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b41007a0 msgr2=0x7fa9b419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b41007a0 0x7fa9b419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b3e00640 1 -- 172.21.15.57:0/2732524723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41a1e10 con 0x7fa9b4101aa0 2024-04-13T05:24:09.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.037+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b4101100 0x7fa9b419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:09.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.038+0000 7fa9b3e00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 0x7fa9b41a17a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4034d50 tx=0x7fa9a402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.039+0000 7fa9abe00640 1 -- 172.21.15.57:0/2732524723 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4039040 con 0x7fa9b4101aa0 2024-04-13T05:24:09.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.039+0000 7fa9abe00640 1 -- 172.21.15.57:0/2732524723 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a4042070 con 0x7fa9b4101aa0 2024-04-13T05:24:09.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.039+0000 7fa9abe00640 1 -- 172.21.15.57:0/2732524723 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40035f0 con 0x7fa9b4101aa0 2024-04-13T05:24:09.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.039+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41a20a0 con 0x7fa9b4101aa0 2024-04-13T05:24:09.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.039+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b41a2540 con 0x7fa9b4101aa0 2024-04-13T05:24:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.045+0000 7fa9bce14740 1 Processor -- start 2024-04-13T05:24:09.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.045+0000 7fa9abe00640 1 -- 172.21.15.57:0/2732524723 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa9a40046c0 con 0x7fa9b4101aa0 2024-04-13T05:24:09.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.045+0000 7fa9bce14740 1 -- start start 2024-04-13T05:24:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.046+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 0x56237de24320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.046+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de248f0 0x56237de293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.046+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de29a60 0x56237de2bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.046+0000 7fa9bce14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56237dd82020 con 0x56237de248f0 2024-04-13T05:24:09.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.046+0000 7fa9bce14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56237dd286d0 con 0x56237de23f00 2024-04-13T05:24:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.046+0000 7fa9bce14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56237dd7dfe0 con 0x56237de29a60 2024-04-13T05:24:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.047+0000 7fa9b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 0x56237de24320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.047+0000 7fa9b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 0x56237de24320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47536/0 (socket says 172.21.15.57:47536) 2024-04-13T05:24:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.047+0000 7fa9b2a00640 1 -- 172.21.15.57:0/608006041 learned_addr learned my addr 172.21.15.57:0/608006041 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.047+0000 7fa9b3400640 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de248f0 0x56237de293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.047+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de29a60 0x56237de2bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.047+0000 7fa9abe00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9980a3d40 0x7fa9980a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b2a00640 1 -- 172.21.15.57:0/608006041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de29a60 msgr2=0x56237de2bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de29a60 0x56237de2bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b2a00640 1 -- 172.21.15.57:0/608006041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de248f0 msgr2=0x56237de293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de248f0 0x56237de293f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b2a00640 1 -- 172.21.15.57:0/608006041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56237db5e7d0 con 0x56237de23f00 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b3e00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9980a3d40 0x7fa9980a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9abe00640 1 -- 172.21.15.57:0/2732524723 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa9a40f21a0 con 0x7fa9b4101aa0 2024-04-13T05:24:09.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 0x56237de24320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa99c014a10 tx=0x7fa99c00b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31bfbcf97f4bd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.048+0000 7fa9a3e00640 1 -- 172.21.15.57:0/608006041 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0142d0 con 0x56237de23f00 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.049+0000 7fa9a3e00640 1 -- 172.21.15.57:0/608006041 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa99c00fc10 con 0x56237de23f00 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.049+0000 7fa9a3e00640 1 -- 172.21.15.57:0/608006041 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0129c0 con 0x56237de23f00 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 -- 172.21.15.57:0/608006041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 msgr2=0x56237de24320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 0x56237de24320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa99c014a10 tx=0x7fa99c00b520 comp rx=0 tx=0).stop 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9b3e00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9980a3d40 0x7fa9980a6200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40ef8a0 tx=0x7fa9a40886c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 -- 172.21.15.57:0/608006041 shutdown_connections 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de29a60 0x56237de2bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de248f0 0x56237de293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 --2- 172.21.15.57:0/608006041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de23f00 0x56237de24320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 -- 172.21.15.57:0/608006041 >> 172.21.15.57:0/608006041 conn(0x56237de1f470 msgr2=0x56237de1f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 -- 172.21.15.57:0/608006041 shutdown_connections 2024-04-13T05:24:09.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.050+0000 7fa9bce14740 1 -- 172.21.15.57:0/608006041 wait complete. 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 -- start start 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 Processor -- start 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de2e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56237de2c9b0 con 0x56237de29a60 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56237dd3ba90 con 0x56237de248f0 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.051+0000 7fa9bce14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56237de2cb30 con 0x56237de23f00 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9bce14740 10 client.0 _ll_register_callbacks cb 0x56237dd3b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:09.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de2e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47552/0 (socket says 172.21.15.57:47552) 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 -- 172.21.15.57:0/3147798114 learned_addr learned my addr 172.21.15.57:0/3147798114 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 -- 172.21.15.57:0/3147798114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 msgr2=0x56237de2e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de2e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 -- 172.21.15.57:0/3147798114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 msgr2=0x56237de35730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de35730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b3400640 1 -- 172.21.15.57:0/3147798114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56237da943f0 con 0x56237de248f0 2024-04-13T05:24:09.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.052+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.053+0000 7fa9b3400640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de2e6f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac010b40 tx=0x7fa9ac00af80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4dccc1f4449e651 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.053+0000 7fa9a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.053+0000 7fa9a0c00640 1 -- 172.21.15.57:0/3147798114 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00fd70 con 0x56237de248f0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.053+0000 7fa9a0c00640 1 -- 172.21.15.57:0/3147798114 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa9ac004b20 con 0x56237de248f0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.054+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99c00fc10 con 0x56237de248f0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.054+0000 7fa9a0c00640 1 -- 172.21.15.57:0/3147798114 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac004d20 con 0x56237de248f0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.054+0000 7fa9bce14740 10 client.36134 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.054+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56237da540f0 con 0x56237de248f0 2024-04-13T05:24:09.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.054+0000 7fa9a0c00640 1 -- 172.21.15.57:0/3147798114 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa99c00fc10 con 0x56237de248f0 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9a0c00640 1 -- 172.21.15.57:0/3147798114 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa9ac0046a0 con 0x56237de248f0 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9bce14740 10 client.36134 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9bce14740 10 client.36134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9bce14740 20 client.36134 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9bce14740 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de3a020 0x56237de3c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9bce14740 4 client.36134 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56237dd21600 con 0x56237de3a020 2024-04-13T05:24:09.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.055+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de3a020 0x56237de3c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.056+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de3a020 0x56237de3c4c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7111050a653ab7 server_cookie=173a7672a1832e13 in_seq=0 out_seq=0 2024-04-13T05:24:09.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.056+0000 7fa9a0c00640 10 client.36134 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:09.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.057+0000 7fa9a0c00640 1 -- 172.21.15.57:0/3147798114 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56237dd21600 con 0x56237de3a020 2024-04-13T05:24:09.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.057+0000 7fa9a0c00640 10 client.36134 handle_command_reply: tid=0 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.057+0000 7fa9bce14740 1 client.36134 shutdown 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.057+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 msgr2=0x56237de2e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.057+0000 7fa9bce14740 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de2e6f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac010b40 tx=0x7fa9ac00af80 comp rx=0 tx=0).stop 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 shutdown_connections 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de3a020 0x56237de3c4c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de2e6f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 --2- 172.21.15.57:0/3147798114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de2e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 >> 172.21.15.57:0/3147798114 conn(0x56237de1f470 msgr2=0x56237de15e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9a0c00640 0 client.36134 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 shutdown_connections 2024-04-13T05:24:09.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 1 -- 172.21.15.57:0/3147798114 wait complete. 2024-04-13T05:24:09.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.058+0000 7fa9bce14740 20 client.36134 trim_cache size 0 max 16384 2024-04-13T05:24:09.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.085+0000 7fa9bce14740 1 Processor -- start 2024-04-13T05:24:09.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 -- start start 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de25c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de26150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41ab0a0 con 0x56237de29a60 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41a8910 con 0x56237de248f0 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.086+0000 7fa9bce14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41a6780 con 0x56237de23f00 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de356f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47560/0 (socket says 172.21.15.57:47560) 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 -- 172.21.15.57:0/2338707106 learned_addr learned my addr 172.21.15.57:0/2338707106 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de26150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de25c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 -- 172.21.15.57:0/2338707106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 msgr2=0x56237de25c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de25c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 -- 172.21.15.57:0/2338707106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 msgr2=0x56237de26150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de26150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.087+0000 7fa9b3400640 1 -- 172.21.15.57:0/2338707106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56237de201e0 con 0x56237de248f0 2024-04-13T05:24:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.088+0000 7fa9b3400640 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de356f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00ff70 tx=0x7fa9ac00ffa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49e4fb6e88db8bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.088+0000 7fa9a3400640 1 -- 172.21.15.57:0/2338707106 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac027070 con 0x56237de248f0 2024-04-13T05:24:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.088+0000 7fa9a3400640 1 -- 172.21.15.57:0/2338707106 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9ac028040 con 0x56237de248f0 2024-04-13T05:24:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.088+0000 7fa9a3400640 1 -- 172.21.15.57:0/2338707106 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004000 con 0x56237de248f0 2024-04-13T05:24:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.089+0000 7fa9bce14740 1 -- 172.21.15.57:0/2338707106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 msgr2=0x56237de356f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.089+0000 7fa9bce14740 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de356f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00ff70 tx=0x7fa9ac00ffa0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 -- 172.21.15.57:0/2338707106 shutdown_connections 2024-04-13T05:24:09.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de356f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de26150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 --2- 172.21.15.57:0/2338707106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de25c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 -- 172.21.15.57:0/2338707106 >> 172.21.15.57:0/2338707106 conn(0x56237de1f470 msgr2=0x56237de2e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 -- 172.21.15.57:0/2338707106 shutdown_connections 2024-04-13T05:24:09.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 -- 172.21.15.57:0/2338707106 wait complete. 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.090+0000 7fa9bce14740 1 -- start start 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 Processor -- start 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de30d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de31260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41a6dd0 con 0x56237de29a60 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41a8240 con 0x56237de248f0 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41a74a0 con 0x56237de23f00 2024-04-13T05:24:09.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.091+0000 7fa9bce14740 10 client.0 _ll_register_callbacks cb 0x7fa9ac010780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de34210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60018/0 (socket says 172.21.15.57:60018) 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 -- 172.21.15.57:0/31919970 learned_addr learned my addr 172.21.15.57:0/31919970 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b3400640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de31260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de30d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 -- 172.21.15.57:0/31919970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 msgr2=0x56237de30d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de30d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 -- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 msgr2=0x56237de31260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.092+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de31260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.093+0000 7fa9b2a00640 1 -- 172.21.15.57:0/31919970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56237de34800 con 0x56237de29a60 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.093+0000 7fa9b2a00640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de34210 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00bb40 tx=0x7fa99c00b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a806e4be1932ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.093+0000 7fa9a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.093+0000 7fa9a0c00640 1 -- 172.21.15.57:0/31919970 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c014320 con 0x56237de29a60 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.093+0000 7fa9a0c00640 1 -- 172.21.15.57:0/31919970 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa99c0050c0 con 0x56237de29a60 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.093+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac028040 con 0x56237de29a60 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.094+0000 7fa9a0c00640 1 -- 172.21.15.57:0/31919970 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa99c0052c0 con 0x56237de29a60 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.094+0000 7fa9bce14740 10 client.16731 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:09.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.094+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56237de1a6f0 con 0x56237de29a60 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9a0c00640 1 -- 172.21.15.57:0/31919970 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9ac028040 con 0x56237de29a60 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9a0c00640 1 -- 172.21.15.57:0/31919970 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa99c00f7c0 con 0x56237de29a60 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9bce14740 10 client.16731 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9bce14740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9bce14740 20 client.16731 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9bce14740 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de1ab90 0x56237de1cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9bce14740 4 client.16731 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:24:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.095+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56237dd8fa40 con 0x56237de1ab90 2024-04-13T05:24:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.096+0000 7fa9b3400640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de1ab90 0x56237de1cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9b3400640 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de1ab90 0x56237de1cfb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d50076f8d263d7 server_cookie=55e5d941f29ef1c9 in_seq=0 out_seq=0 2024-04-13T05:24:09.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9a0c00640 10 client.16731 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:09.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9a0c00640 1 -- 172.21.15.57:0/31919970 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56237dd8fa40 con 0x56237de1ab90 2024-04-13T05:24:09.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9a0c00640 10 client.16731 handle_command_reply: tid=0 2024-04-13T05:24:09.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9bce14740 1 client.16731 shutdown 2024-04-13T05:24:09.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 msgr2=0x56237de34210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.097+0000 7fa9bce14740 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de34210 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00bb40 tx=0x7fa99c00b520 comp rx=0 tx=0).stop 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 shutdown_connections 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56237de1ab90 0x56237de1cfb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237de248f0 0x56237de31260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56237de29a60 0x56237de34210 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 --2- 172.21.15.57:0/31919970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56237de23f00 0x56237de30d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 >> 172.21.15.57:0/31919970 conn(0x56237de1f470 msgr2=0x56237de2de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9a0c00640 0 client.16731 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 shutdown_connections 2024-04-13T05:24:09.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 1 -- 172.21.15.57:0/31919970 wait complete. 2024-04-13T05:24:09.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.098+0000 7fa9bce14740 20 client.16731 trim_cache size 0 max 16384 2024-04-13T05:24:09.102 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:24:09.102 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:24:09.102 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:24:09.102 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9980a3d40 msgr2=0x7fa9980a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9980a3d40 0x7fa9980a6200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40ef8a0 tx=0x7fa9a40886c0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 msgr2=0x7fa9b41a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 0x7fa9b41a17a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4034d50 tx=0x7fa9a402c740 comp rx=0 tx=0).stop 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 shutdown_connections 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9980a3d40 0x7fa9980a6200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9b4101aa0 0x7fa9b41a17a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9b4101100 0x7fa9b419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 --2- 172.21.15.57:0/2732524723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b41007a0 0x7fa9b419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.099+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 >> 172.21.15.57:0/2732524723 conn(0x7fa9b40fc360 msgr2=0x7fa9b40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.100+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 shutdown_connections 2024-04-13T05:24:09.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.100+0000 7fa9b8a00640 1 -- 172.21.15.57:0/2732524723 wait complete. 2024-04-13T05:24:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:08 smithi057 ceph-mon[32331]: pgmap v652: 129 pgs: 129 active+clean; 393 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-13T05:24:09.123 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:24:09.123 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:24:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:08 smithi083 ceph-mon[35133]: pgmap v652: 129 pgs: 129 active+clean; 393 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-13T05:24:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:08 smithi189 ceph-mon[35531]: pgmap v652: 129 pgs: 129 active+clean; 393 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-13T05:24:09.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1612750322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8077bc0 msgr2=0x7f1fd8078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1612750322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8077bc0 0x7f1fd8078020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc402bc60 comp rx=0 tx=0).stop 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1612750322 shutdown_connections 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1612750322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8077bc0 0x7f1fd8078020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1612750322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fd8078560 0x7f1fd8078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1612750322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8076980 0x7f1fd8076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1612750322 >> 172.21.15.57:0/1612750322 conn(0x7f1fd80ff7b0 msgr2=0x7f1fd8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1612750322 shutdown_connections 2024-04-13T05:24:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.381+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1612750322 wait complete. 2024-04-13T05:24:09.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.382+0000 7f1fdfa00640 1 Processor -- start 2024-04-13T05:24:09.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 -- start start 2024-04-13T05:24:09.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 0x7f1fd819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fd8077bc0 0x7f1fd819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8078560 0x7f1fd81a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8114200 con 0x7f1fd8078560 2024-04-13T05:24:09.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8114080 con 0x7f1fd8076980 2024-04-13T05:24:09.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.383+0000 7f1fdfa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8114380 con 0x7f1fd8077bc0 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdda00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8078560 0x7f1fd81a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 0x7f1fd819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 0x7f1fd819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47580/0 (socket says 172.21.15.57:47580) 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 -- 172.21.15.57:0/1335717403 learned_addr learned my addr 172.21.15.57:0/1335717403 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fd7e00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fd8077bc0 0x7f1fd819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 -- 172.21.15.57:0/1335717403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fd8077bc0 msgr2=0x7f1fd819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fd8077bc0 0x7f1fd819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 -- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8078560 msgr2=0x7f1fd81a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8078560 0x7f1fd81a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdd000640 1 -- 172.21.15.57:0/1335717403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd81a64c0 con 0x7f1fd8076980 2024-04-13T05:24:09.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.384+0000 7f1fdda00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8078560 0x7f1fd81a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:09.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.385+0000 7f1fdd000640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 0x7f1fd819e8a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00e9f0 tx=0x7f1fcc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.385+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00cd80 con 0x7f1fd8076980 2024-04-13T05:24:09.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.385+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fcc0108e0 con 0x7f1fd8076980 2024-04-13T05:24:09.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.385+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc018920 con 0x7f1fd8076980 2024-04-13T05:24:09.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.385+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd81a67b0 con 0x7f1fd8076980 2024-04-13T05:24:09.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.386+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd81a6d30 con 0x7f1fd8076980 2024-04-13T05:24:09.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.387+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd8110d90 con 0x7f1fd8076980 2024-04-13T05:24:09.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.389+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1fcc010a90 con 0x7f1fd8076980 2024-04-13T05:24:09.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.391+0000 7f1fd5600640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1fc00a3c30 0x7f1fc00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:09.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.391+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1fcc0d0b10 con 0x7f1fd8076980 2024-04-13T05:24:09.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.392+0000 7f1fd7e00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1fc00a3c30 0x7f1fc00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:09.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.393+0000 7f1fd7e00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1fc00a3c30 0x7f1fc00a60f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fd819fdb0 tx=0x7f1fc4004bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:09.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.397+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fcc09d0f0 con 0x7f1fd8076980 2024-04-13T05:24:09.638 INFO:tasks.workunit.client.0.smithi057.stdout: 54 7851 50 5384 61 2676 95 2024-04-13T05:24:09.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.820+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fd802d5e0 con 0x7f1fd8076980 2024-04-13T05:24:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:24:09.824 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:22:16.935949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:24:09.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.822+0000 7f1fd5600640 1 -- 172.21.15.57:0/1335717403 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11858 (secure 0 0 0) 0x7f1fcc09c9f0 con 0x7f1fd8076980 2024-04-13T05:24:09.825 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 29 2024-04-13T05:24:09.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1fc00a3c30 msgr2=0x7f1fc00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1fc00a3c30 0x7f1fc00a60f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fd819fdb0 tx=0x7f1fc4004bb0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 msgr2=0x7f1fd819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 0x7f1fd819e8a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00e9f0 tx=0x7f1fcc00eeb0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 shutdown_connections 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1fc00a3c30 0x7f1fc00a60f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fd8078560 0x7f1fd81a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fd8077bc0 0x7f1fd819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 --2- 172.21.15.57:0/1335717403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fd8076980 0x7f1fd819e8a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 >> 172.21.15.57:0/1335717403 conn(0x7f1fd80ff7b0 msgr2=0x7f1fd8100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 shutdown_connections 2024-04-13T05:24:09.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:09.825+0000 7f1fdfa00640 1 -- 172.21.15.57:0/1335717403 wait complete. 2024-04-13T05:24:09.850 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:24:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 -- 172.21.15.57:0/3542358910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 msgr2=0x7f6180103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 --2- 172.21.15.57:0/3542358910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f6180103500 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f616c009900 tx=0x7f616c02bc40 comp rx=0 tx=0).stop 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 -- 172.21.15.57:0/3542358910 shutdown_connections 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 --2- 172.21.15.57:0/3542358910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6180103a40 0x7f618010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 --2- 172.21.15.57:0/3542358910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f6180103500 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 --2- 172.21.15.57:0/3542358910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6180101ea0 0x7f61801022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.111+0000 7f6186c00640 1 -- 172.21.15.57:0/3542358910 >> 172.21.15.57:0/3542358910 conn(0x7f61800fd460 msgr2=0x7f61800ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.112+0000 7f6186c00640 1 -- 172.21.15.57:0/3542358910 shutdown_connections 2024-04-13T05:24:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.112+0000 7f6186c00640 1 -- 172.21.15.57:0/3542358910 wait complete. 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.113+0000 7f6186c00640 1 Processor -- start 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.113+0000 7f6186c00640 1 -- start start 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.113+0000 7f6186c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6180101ea0 0x7f618019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.113+0000 7f6186c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f618019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.113+0000 7f6186c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6180103a40 0x7f61801a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6186c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f618010daa0 con 0x7f6180103080 2024-04-13T05:24:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6186c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f618010d920 con 0x7f6180103a40 2024-04-13T05:24:10.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6186c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f618010dc20 con 0x7f6180101ea0 2024-04-13T05:24:10.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6184e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f618019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6184e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f618019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60042/0 (socket says 172.21.15.57:60042) 2024-04-13T05:24:10.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6184e00640 1 -- 172.21.15.57:0/4267833362 learned_addr learned my addr 172.21.15.57:0/4267833362 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6185800640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6180101ea0 0x7f618019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.114+0000 7f6186200640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6180103a40 0x7f61801a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.115+0000 7f6184e00640 1 -- 172.21.15.57:0/4267833362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6180101ea0 msgr2=0x7f618019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.115+0000 7f6184e00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6180101ea0 0x7f618019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.115+0000 7f6184e00640 1 -- 172.21.15.57:0/4267833362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6180103a40 msgr2=0x7f61801a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.115+0000 7f6184e00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6180103a40 0x7f61801a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.115+0000 7f6184e00640 1 -- 172.21.15.57:0/4267833362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61801a1fa0 con 0x7f6180103080 2024-04-13T05:24:10.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.115+0000 7f6184e00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f618019a8c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f616c006010 tx=0x7f616c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.116+0000 7f617e000640 1 -- 172.21.15.57:0/4267833362 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c043020 con 0x7f6180103080 2024-04-13T05:24:10.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.116+0000 7f617e000640 1 -- 172.21.15.57:0/4267833362 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f616c004440 con 0x7f6180103080 2024-04-13T05:24:10.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.116+0000 7f617e000640 1 -- 172.21.15.57:0/4267833362 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c003bf0 con 0x7f6180103080 2024-04-13T05:24:10.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.116+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61801a2230 con 0x7f6180103080 2024-04-13T05:24:10.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.116+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61801a27b0 con 0x7f6180103080 2024-04-13T05:24:10.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.121+0000 7f618b191740 1 Processor -- start 2024-04-13T05:24:10.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.121+0000 7f618b191740 1 -- start start 2024-04-13T05:24:10.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.121+0000 7f618b191740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 0x5577d229e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f618b191740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229e640 0x5577d22a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f618b191740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22a37b0 0x5577d22a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f618b191740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5577d21fb020 con 0x5577d229dc50 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f618b191740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577d21a1950 con 0x5577d22a37b0 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f618b191740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577d21f6fe0 con 0x5577d229e640 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f617e000640 1 -- 172.21.15.57:0/4267833362 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f616c046020 con 0x7f6180103080 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f6186200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 0x5577d229e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f6186200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 0x5577d229e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60046/0 (socket says 172.21.15.57:60046) 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f6186200640 1 -- 172.21.15.57:0/3621975243 learned_addr learned my addr 172.21.15.57:0/3621975243 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f6185800640 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229e640 0x5577d22a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.122+0000 7f6184e00640 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22a37b0 0x5577d22a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.123+0000 7f6186200640 1 -- 172.21.15.57:0/3621975243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229e640 msgr2=0x5577d22a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.123+0000 7f6186200640 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229e640 0x5577d22a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.123+0000 7f6186200640 1 -- 172.21.15.57:0/3621975243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22a37b0 msgr2=0x5577d22a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.123+0000 7f6186200640 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22a37b0 0x5577d22a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.123+0000 7f6186200640 1 -- 172.21.15.57:0/3621975243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d1f0d3f0 con 0x5577d229dc50 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.124+0000 7f6186200640 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 0x5577d229e070 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6178009f20 tx=0x7f617800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0245dd626a4190 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.124+0000 7f6172a00640 1 -- 172.21.15.57:0/3621975243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617801a070 con 0x5577d229dc50 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.124+0000 7f6172a00640 1 -- 172.21.15.57:0/3621975243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61780158c0 con 0x5577d229dc50 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.124+0000 7f6172a00640 1 -- 172.21.15.57:0/3621975243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617800a820 con 0x5577d229dc50 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.124+0000 7f617e000640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f61680a3d00 0x7f61680a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.125+0000 7f617e000640 1 -- 172.21.15.57:0/4267833362 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f616c0f2b20 con 0x7f6180103080 2024-04-13T05:24:10.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.125+0000 7f6185800640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f61680a3d00 0x7f61680a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 -- 172.21.15.57:0/3621975243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 msgr2=0x5577d229e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 0x5577d229e070 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6178009f20 tx=0x7f617800c8b0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 -- 172.21.15.57:0/3621975243 shutdown_connections 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22a37b0 0x5577d22a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229e640 0x5577d22a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 --2- 172.21.15.57:0/3621975243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229dc50 0x5577d229e070 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.126+0000 7f618b191740 1 -- 172.21.15.57:0/3621975243 >> 172.21.15.57:0/3621975243 conn(0x5577d22991c0 msgr2=0x5577d22995c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.127+0000 7f6185800640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f61680a3d00 0x7f61680a61c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5577d229ea40 tx=0x7f6174005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.127+0000 7f618b191740 1 -- 172.21.15.57:0/3621975243 shutdown_connections 2024-04-13T05:24:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.127+0000 7f618b191740 1 -- 172.21.15.57:0/3621975243 wait complete. 2024-04-13T05:24:10.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.128+0000 7f618b191740 1 -- start start 2024-04-13T05:24:10.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.128+0000 7f618b191740 1 Processor -- start 2024-04-13T05:24:10.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d229e640 0x5577d22a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d22a37b0 0x5577d22af870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5577d22a6760 con 0x5577d22a37b0 2024-04-13T05:24:10.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577d21b4c80 con 0x5577d229e640 2024-04-13T05:24:10.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577d22a68e0 con 0x5577d229dc50 2024-04-13T05:24:10.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f618b191740 10 client.0 _ll_register_callbacks cb 0x5577d21b4670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f6185800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.129+0000 7f6185800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22a82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:53530/0 (socket says 172.21.15.57:53530) 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6185800640 1 -- 172.21.15.57:0/815536146 learned_addr learned my addr 172.21.15.57:0/815536146 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6184e00640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d229e640 0x5577d22a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6186200640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d22a37b0 0x5577d22af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6186200640 1 -- 172.21.15.57:0/815536146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 msgr2=0x5577d22a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6186200640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22a82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6186200640 1 -- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d229e640 msgr2=0x5577d22a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6186200640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d229e640 0x5577d22a8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.130+0000 7f6186200640 1 -- 172.21.15.57:0/815536146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d1fd77d0 con 0x5577d22a37b0 2024-04-13T05:24:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.131+0000 7f6186200640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d22a37b0 0x5577d22af870 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x5577d229eea0 tx=0x7f617800c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2d9efe5d8d85ce server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.131+0000 7f6167400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.132+0000 7f6167400640 1 -- 172.21.15.57:0/815536146 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178025060 con 0x5577d22a37b0 2024-04-13T05:24:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.132+0000 7f6167400640 1 -- 172.21.15.57:0/815536146 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f617800a5a0 con 0x5577d22a37b0 2024-04-13T05:24:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.132+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61780158c0 con 0x5577d22a37b0 2024-04-13T05:24:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.133+0000 7f6167400640 1 -- 172.21.15.57:0/815536146 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f617801a040 con 0x5577d22a37b0 2024-04-13T05:24:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.133+0000 7f618b191740 10 client.16749 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.133+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577d22b2be0 con 0x5577d22a37b0 2024-04-13T05:24:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.133+0000 7f6167400640 1 -- 172.21.15.57:0/815536146 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61780158c0 con 0x5577d22a37b0 2024-04-13T05:24:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f6167400640 1 -- 172.21.15.57:0/815536146 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f617801d070 con 0x5577d22a37b0 2024-04-13T05:24:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f618b191740 10 client.16749 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f618b191740 10 client.16749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:24:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f618b191740 20 client.16749 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f618b191740 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22b3f90 0x5577d22b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f618b191740 4 client.16749 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.134+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577d219a750 con 0x5577d22b3f90 2024-04-13T05:24:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.135+0000 7f6185800640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22b3f90 0x5577d22b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.136+0000 7f6185800640 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22b3f90 0x5577d22b6380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c41e6a83414f2c86 server_cookie=2844915dba358a21 in_seq=0 out_seq=0 2024-04-13T05:24:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.136+0000 7f6167400640 10 client.16749 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:10.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.137+0000 7f6167400640 1 -- 172.21.15.57:0/815536146 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577d219a750 con 0x5577d22b3f90 2024-04-13T05:24:10.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.137+0000 7f6167400640 10 client.16749 handle_command_reply: tid=0 2024-04-13T05:24:10.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.137+0000 7f618b191740 1 client.16749 shutdown 2024-04-13T05:24:10.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d22a37b0 msgr2=0x5577d22af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d22a37b0 0x5577d22af870 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x5577d229eea0 tx=0x7f617800c350 comp rx=0 tx=0).stop 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 shutdown_connections 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22b3f90 0x5577d22b6380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d229e640 0x5577d22a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d22a37b0 0x5577d22af870 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 --2- 172.21.15.57:0/815536146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 >> 172.21.15.57:0/815536146 conn(0x5577d22991c0 msgr2=0x5577d228fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.138+0000 7f6167400640 0 client.16749 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.139+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 shutdown_connections 2024-04-13T05:24:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.139+0000 7f618b191740 1 -- 172.21.15.57:0/815536146 wait complete. 2024-04-13T05:24:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.139+0000 7f618b191740 20 client.16749 trim_cache size 0 max 16384 2024-04-13T05:24:10.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.159+0000 7f618b191740 1 Processor -- start 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.159+0000 7f618b191740 1 -- start start 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.159+0000 7f618b191740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.159+0000 7f618b191740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f618b191740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f618b191740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f61801ab290 con 0x5577d229e640 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f618b191740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61801a8b00 con 0x5577d22991c0 2024-04-13T05:24:10.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f618b191740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61801a6950 con 0x5577d229dc50 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6186200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47608/0 (socket says 172.21.15.57:47608) 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 -- 172.21.15.57:0/1297982801 learned_addr learned my addr 172.21.15.57:0/1297982801 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6184e00640 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 -- 172.21.15.57:0/1297982801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 msgr2=0x5577d22af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 -- 172.21.15.57:0/1297982801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 msgr2=0x5577d22a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6185800640 1 -- 172.21.15.57:0/1297982801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d22a0510 con 0x5577d22991c0 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.160+0000 7f6186200640 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.161+0000 7f6185800640 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22a6cb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f617401d670 tx=0x7f6174019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289c32302ad5bed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.161+0000 7f6172000640 1 -- 172.21.15.57:0/1297982801 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174031070 con 0x5577d22991c0 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.162+0000 7f6172000640 1 -- 172.21.15.57:0/1297982801 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f617401dde0 con 0x5577d22991c0 2024-04-13T05:24:10.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.162+0000 7f6172000640 1 -- 172.21.15.57:0/1297982801 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617402a9e0 con 0x5577d22991c0 2024-04-13T05:24:10.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.162+0000 7f618b191740 1 -- 172.21.15.57:0/1297982801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 msgr2=0x5577d22a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.162+0000 7f618b191740 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22a6cb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f617401d670 tx=0x7f6174019000 comp rx=0 tx=0).stop 2024-04-13T05:24:10.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 -- 172.21.15.57:0/1297982801 shutdown_connections 2024-04-13T05:24:10.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 --2- 172.21.15.57:0/1297982801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22a6cb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 -- 172.21.15.57:0/1297982801 >> 172.21.15.57:0/1297982801 conn(0x5577d22a7eb0 msgr2=0x5577d22a82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 -- 172.21.15.57:0/1297982801 shutdown_connections 2024-04-13T05:24:10.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 -- 172.21.15.57:0/1297982801 wait complete. 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.163+0000 7f618b191740 1 -- start start 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 Processor -- start 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22ad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22ae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f61801a6fc0 con 0x5577d229e640 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61801a8430 con 0x5577d22991c0 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61801a7690 con 0x5577d229dc50 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f618b191740 10 client.0 _ll_register_callbacks cb 0x7f61780157b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f6185800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f6185800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47624/0 (socket says 172.21.15.57:47624) 2024-04-13T05:24:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f6186200640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6186200640 1 -- 172.21.15.57:0/3715892984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 msgr2=0x5577d22ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6186200640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6186200640 1 -- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 msgr2=0x5577d22acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6186200640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.164+0000 7f6185800640 1 -- 172.21.15.57:0/3715892984 learned_addr learned my addr 172.21.15.57:0/3715892984 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6186200640 1 -- 172.21.15.57:0/3715892984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d22ae860 con 0x5577d229e640 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6185800640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6186200640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22ae270 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f617800a010 tx=0x7f61780071d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5a96320d0b5f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.165+0000 7f6167400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.166+0000 7f6167400640 1 -- 172.21.15.57:0/3715892984 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178025040 con 0x5577d229e640 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.166+0000 7f6167400640 1 -- 172.21.15.57:0/3715892984 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f617802a420 con 0x5577d229e640 2024-04-13T05:24:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.166+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617401dde0 con 0x5577d229e640 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.166+0000 7f6167400640 1 -- 172.21.15.57:0/3715892984 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f617803c050 con 0x5577d229e640 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.166+0000 7f618b191740 10 client.16755 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.166+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577d22afb60 con 0x5577d229e640 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f6167400640 1 -- 172.21.15.57:0/3715892984 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f617401dde0 con 0x5577d229e640 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f6167400640 1 -- 172.21.15.57:0/3715892984 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f617801d040 con 0x5577d229e640 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f618b191740 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f618b191740 10 client.16755 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f618b191740 20 client.16755 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f618b191740 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22a37b0 0x5577d231c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f618b191740 4 client.16755 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:24:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.167+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577d2208a40 con 0x5577d22a37b0 2024-04-13T05:24:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.168+0000 7f6185800640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22a37b0 0x5577d231c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.169+0000 7f6185800640 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22a37b0 0x5577d231c9b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdd5e34cd327f1c4 server_cookie=acc56c84779bea36 in_seq=0 out_seq=0 2024-04-13T05:24:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.169+0000 7f6167400640 10 client.16755 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:10.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.171+0000 7f6167400640 1 -- 172.21.15.57:0/3715892984 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577d2208a40 con 0x5577d22a37b0 2024-04-13T05:24:10.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.171+0000 7f6167400640 10 client.16755 handle_command_reply: tid=0 2024-04-13T05:24:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.171+0000 7f618b191740 1 client.16755 shutdown 2024-04-13T05:24:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 msgr2=0x5577d22ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22ae270 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f617800a010 tx=0x7f61780071d0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 shutdown_connections 2024-04-13T05:24:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5577d22a37b0 0x5577d231c9b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f6167400640 0 client.16755 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5577d229e640 0x5577d22ae270 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577d229dc50 0x5577d22ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 --2- 172.21.15.57:0/3715892984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577d22991c0 0x5577d22acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 >> 172.21.15.57:0/3715892984 conn(0x5577d22a7eb0 msgr2=0x5577d22a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.172+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 shutdown_connections 2024-04-13T05:24:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.173+0000 7f618b191740 1 -- 172.21.15.57:0/3715892984 wait complete. 2024-04-13T05:24:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.173+0000 7f618b191740 20 client.16755 trim_cache size 0 max 16384 2024-04-13T05:24:10.176 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:24:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f61680a3d00 msgr2=0x7f61680a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f61680a3d00 0x7f61680a61c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5577d229ea40 tx=0x7f6174005f90 comp rx=0 tx=0).stop 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 msgr2=0x7f618019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f618019a8c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f616c006010 tx=0x7f616c004040 comp rx=0 tx=0).stop 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 shutdown_connections 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f61680a3d00 0x7f61680a61c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6180103a40 0x7f61801a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6180103080 0x7f618019a8c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 --2- 172.21.15.57:0/4267833362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6180101ea0 0x7f618019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 >> 172.21.15.57:0/4267833362 conn(0x7f61800fd460 msgr2=0x7f61800741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.174+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 shutdown_connections 2024-04-13T05:24:10.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.175+0000 7f6186c00640 1 -- 172.21.15.57:0/4267833362 wait complete. 2024-04-13T05:24:10.196 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1335717403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:24:10.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:24:10.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1335717403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:24:10.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1335717403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:24:10.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.466+0000 7f87ce800640 1 -- 172.21.15.57:0/2240310259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 msgr2=0x7f87c8105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.466+0000 7f87ce800640 1 --2- 172.21.15.57:0/2240310259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c8105770 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009900 tx=0x7f87b402bc40 comp rx=0 tx=0).stop 2024-04-13T05:24:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.467+0000 7f87ce800640 1 -- 172.21.15.57:0/2240310259 shutdown_connections 2024-04-13T05:24:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.467+0000 7f87ce800640 1 --2- 172.21.15.57:0/2240310259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87c8105cb0 0x7f87c810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.467+0000 7f87ce800640 1 --2- 172.21.15.57:0/2240310259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c8105770 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.467+0000 7f87ce800640 1 --2- 172.21.15.57:0/2240310259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8104100 0x7f87c8104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.467+0000 7f87ce800640 1 -- 172.21.15.57:0/2240310259 >> 172.21.15.57:0/2240310259 conn(0x7f87c80ff670 msgr2=0x7f87c8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.468+0000 7f87ce800640 1 -- 172.21.15.57:0/2240310259 shutdown_connections 2024-04-13T05:24:10.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.468+0000 7f87ce800640 1 -- 172.21.15.57:0/2240310259 wait complete. 2024-04-13T05:24:10.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.469+0000 7f87ce800640 1 Processor -- start 2024-04-13T05:24:10.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.469+0000 7f87ce800640 1 -- start start 2024-04-13T05:24:10.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87c8104100 0x7f87c8079d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87ce800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c807a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87ce800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8105cb0 0x7f87c80768e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87ce800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810fc90 con 0x7f87c81052f0 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87ce800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810fb10 con 0x7f87c8104100 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87ce800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810fe10 con 0x7f87c8105cb0 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87cca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c807a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87cca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c807a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60064/0 (socket says 172.21.15.57:60064) 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87cca00640 1 -- 172.21.15.57:0/3821009256 learned_addr learned my addr 172.21.15.57:0/3821009256 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.470+0000 7f87cd400640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87c8104100 0x7f87c8079d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.471+0000 7f87cde00640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8105cb0 0x7f87c80768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.471+0000 7f87cca00640 1 -- 172.21.15.57:0/3821009256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8105cb0 msgr2=0x7f87c80768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.471+0000 7f87cca00640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8105cb0 0x7f87c80768e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.471+0000 7f87cca00640 1 -- 172.21.15.57:0/3821009256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87c8104100 msgr2=0x7f87c8079d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.471+0000 7f87cca00640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87c8104100 0x7f87c8079d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.471+0000 7f87cca00640 1 -- 172.21.15.57:0/3821009256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c80771a0 con 0x7f87c81052f0 2024-04-13T05:24:10.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.472+0000 7f87cca00640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c807a2b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.473+0000 7f87c6000640 1 -- 172.21.15.57:0/3821009256 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4043020 con 0x7f87c81052f0 2024-04-13T05:24:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.473+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c81a6770 con 0x7f87c81052f0 2024-04-13T05:24:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.473+0000 7f87c6000640 1 -- 172.21.15.57:0/3821009256 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87b40054c0 con 0x7f87c81052f0 2024-04-13T05:24:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.473+0000 7f87c6000640 1 -- 172.21.15.57:0/3821009256 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4003d70 con 0x7f87c81052f0 2024-04-13T05:24:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.473+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c81a6b70 con 0x7f87c81052f0 2024-04-13T05:24:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.477+0000 7f87d2e14740 1 Processor -- start 2024-04-13T05:24:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 -- start start 2024-04-13T05:24:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c855860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c85d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c7bb020 con 0x55ca6c855e30 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c761950 con 0x55ca6c855440 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.478+0000 7f87d2e14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c7b6fe0 con 0x55ca6c85afa0 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c855860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c855860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47648/0 (socket says 172.21.15.57:47648) 2024-04-13T05:24:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cde00640 1 -- 172.21.15.57:0/1255475916 learned_addr learned my addr 172.21.15.57:0/1255475916 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87c6000640 1 -- 172.21.15.57:0/3821009256 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f87b4046020 con 0x7f87c81052f0 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cde00640 1 -- 172.21.15.57:0/1255475916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 msgr2=0x55ca6c85d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cde00640 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c85d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.479+0000 7f87cde00640 1 -- 172.21.15.57:0/1255475916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 msgr2=0x55ca6c85a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.480+0000 7f87cde00640 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.480+0000 7f87cde00640 1 -- 172.21.15.57:0/1255475916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6c4cd3f0 con 0x55ca6c855440 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.480+0000 7f87cd400640 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.480+0000 7f87cde00640 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c855860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009fa0 tx=0x7f87c000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a75caa72cc10a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.481+0000 7f87ba600640 1 -- 172.21.15.57:0/1255475916 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0015070 con 0x55ca6c855440 2024-04-13T05:24:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.481+0000 7f87ba600640 1 -- 172.21.15.57:0/1255475916 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c001ab10 con 0x55ca6c855440 2024-04-13T05:24:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.481+0000 7f87ba600640 1 -- 172.21.15.57:0/1255475916 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0011d00 con 0x55ca6c855440 2024-04-13T05:24:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.481+0000 7f87c6000640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87b00a3c30 0x7f87b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.481+0000 7f87c6000640 1 -- 172.21.15.57:0/3821009256 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f87b40f2be0 con 0x7f87c81052f0 2024-04-13T05:24:10.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87cd400640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87b00a3c30 0x7f87b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 -- 172.21.15.57:0/1255475916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 msgr2=0x55ca6c855860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c855860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009fa0 tx=0x7f87c000f7f0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 -- 172.21.15.57:0/1255475916 shutdown_connections 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c85d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 --2- 172.21.15.57:0/1255475916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c855860 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.482+0000 7f87d2e14740 1 -- 172.21.15.57:0/1255475916 >> 172.21.15.57:0/1255475916 conn(0x55ca6c8509b0 msgr2=0x55ca6c850db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.483+0000 7f87d2e14740 1 -- 172.21.15.57:0/1255475916 shutdown_connections 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.483+0000 7f87d2e14740 1 -- 172.21.15.57:0/1255475916 wait complete. 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.483+0000 7f87cd400640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87b00a3c30 0x7f87b00a60f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55ca6c851920 tx=0x7f87bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.483+0000 7f87d2e14740 1 -- start start 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.483+0000 7f87d2e14740 1 Processor -- start 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 0x55ca6c866f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c85df50 con 0x55ca6c855e30 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c774c80 con 0x55ca6c85afa0 2024-04-13T05:24:10.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ca6c85e0d0 con 0x55ca6c855440 2024-04-13T05:24:10.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87cde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 0x55ca6c866f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.484+0000 7f87d2e14740 10 client.0 _ll_register_callbacks cb 0x55ca6c774670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:10.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.485+0000 7f87cde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 0x55ca6c866f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47654/0 (socket says 172.21.15.57:47654) 2024-04-13T05:24:10.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.485+0000 7f87cde00640 1 -- 172.21.15.57:0/942337225 learned_addr learned my addr 172.21.15.57:0/942337225 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.485+0000 7f87cd400640 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.486+0000 7f87cde00640 1 -- 172.21.15.57:0/942337225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 msgr2=0x55ca6c85fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.486+0000 7f87cde00640 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.486+0000 7f87cde00640 1 -- 172.21.15.57:0/942337225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 msgr2=0x55ca6c85ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.486+0000 7f87cde00640 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.486+0000 7f87cde00640 1 -- 172.21.15.57:0/942337225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6c5977d0 con 0x55ca6c85afa0 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.486+0000 7f87cde00640 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 0x55ca6c866f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000f80 tx=0x7f87c0012cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7559b666d4472f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.487+0000 7f87aea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:10.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.487+0000 7f87aea00640 1 -- 172.21.15.57:0/942337225 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0029060 con 0x55ca6c85afa0 2024-04-13T05:24:10.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.487+0000 7f87aea00640 1 -- 172.21.15.57:0/942337225 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f87c0004c50 con 0x55ca6c85afa0 2024-04-13T05:24:10.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.487+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c001ab10 con 0x55ca6c85afa0 2024-04-13T05:24:10.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.488+0000 7f87aea00640 1 -- 172.21.15.57:0/942337225 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87c0015050 con 0x55ca6c85afa0 2024-04-13T05:24:10.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.488+0000 7f87d2e14740 10 client.36173 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:10.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.488+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca6c86a350 con 0x55ca6c85afa0 2024-04-13T05:24:10.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87aea00640 1 -- 172.21.15.57:0/942337225 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c001ab10 con 0x55ca6c85afa0 2024-04-13T05:24:10.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87aea00640 1 -- 172.21.15.57:0/942337225 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87c0005280 con 0x55ca6c85afa0 2024-04-13T05:24:10.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87d2e14740 10 client.36173 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:10.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87d2e14740 10 client.36173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:24:10.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87d2e14740 20 client.36173 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:10.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87d2e14740 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c86b750 0x55ca6c86db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87d2e14740 4 client.36173 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:10.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.489+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca6c75a750 con 0x55ca6c86b750 2024-04-13T05:24:10.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.490+0000 7f87cd400640 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c86b750 0x55ca6c86db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.492+0000 7f87cd400640 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c86b750 0x55ca6c86db40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=302e82de3a6a22e4 server_cookie=fe7eaf026430b568 in_seq=0 out_seq=0 2024-04-13T05:24:10.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.492+0000 7f87aea00640 10 client.36173 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:10.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.493+0000 7f87aea00640 1 -- 172.21.15.57:0/942337225 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca6c75a750 con 0x55ca6c86b750 2024-04-13T05:24:10.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.493+0000 7f87aea00640 10 client.36173 handle_command_reply: tid=0 2024-04-13T05:24:10.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.493+0000 7f87d2e14740 1 client.36173 shutdown 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.493+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 msgr2=0x55ca6c866f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.493+0000 7f87d2e14740 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 0x55ca6c866f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87c0000f80 tx=0x7f87c0012cc0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 shutdown_connections 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c86b750 0x55ca6c86db40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c85afa0 0x55ca6c866f90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c85ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 --2- 172.21.15.57:0/942337225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 >> 172.21.15.57:0/942337225 conn(0x55ca6c8509b0 msgr2=0x55ca6c84fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87aea00640 0 client.36173 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:10.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 shutdown_connections 2024-04-13T05:24:10.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 1 -- 172.21.15.57:0/942337225 wait complete. 2024-04-13T05:24:10.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.494+0000 7f87d2e14740 20 client.36173 trim_cache size 0 max 16384 2024-04-13T05:24:10.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.519+0000 7f87d2e14740 1 Processor -- start 2024-04-13T05:24:10.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.519+0000 7f87d2e14740 1 -- start start 2024-04-13T05:24:10.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.519+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.519+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855e30 0x55ca6c866f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 0x55ca6c85ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87d2e14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b3940 con 0x55ca6c85afa0 2024-04-13T05:24:10.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87d2e14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b11b0 con 0x55ca6c855e30 2024-04-13T05:24:10.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87d2e14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81aefc0 con 0x55ca6c855440 2024-04-13T05:24:10.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87cde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 0x55ca6c85ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87cde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 0x55ca6c85ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60098/0 (socket says 172.21.15.57:60098) 2024-04-13T05:24:10.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87cde00640 1 -- 172.21.15.57:0/4201434292 learned_addr learned my addr 172.21.15.57:0/4201434292 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.520+0000 7f87cd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.521+0000 7f87cca00640 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855e30 0x55ca6c866f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.521+0000 7f87cde00640 1 -- 172.21.15.57:0/4201434292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 msgr2=0x55ca6c85fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.521+0000 7f87cde00640 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.521+0000 7f87cde00640 1 -- 172.21.15.57:0/4201434292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855e30 msgr2=0x55ca6c866f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.521+0000 7f87cde00640 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855e30 0x55ca6c866f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.521+0000 7f87cde00640 1 -- 172.21.15.57:0/4201434292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6c855080 con 0x55ca6c85afa0 2024-04-13T05:24:10.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.522+0000 7f87cde00640 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 0x55ca6c85ff50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006010 tx=0x7f87c0005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd974214348d6daf server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.523+0000 7f87b9c00640 1 -- 172.21.15.57:0/4201434292 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0015050 con 0x55ca6c85afa0 2024-04-13T05:24:10.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.523+0000 7f87b9c00640 1 -- 172.21.15.57:0/4201434292 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c0012de0 con 0x55ca6c85afa0 2024-04-13T05:24:10.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.523+0000 7f87b9c00640 1 -- 172.21.15.57:0/4201434292 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c001a460 con 0x55ca6c85afa0 2024-04-13T05:24:10.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 -- 172.21.15.57:0/4201434292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 msgr2=0x55ca6c85ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 0x55ca6c85ff50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006010 tx=0x7f87c0005cd0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 -- 172.21.15.57:0/4201434292 shutdown_connections 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855e30 0x55ca6c866f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c85afa0 0x55ca6c85ff50 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 --2- 172.21.15.57:0/4201434292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c855440 0x55ca6c85fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 -- 172.21.15.57:0/4201434292 >> 172.21.15.57:0/4201434292 conn(0x55ca6c8509b0 msgr2=0x55ca6c85e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 -- 172.21.15.57:0/4201434292 shutdown_connections 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.524+0000 7f87d2e14740 1 -- 172.21.15.57:0/4201434292 wait complete. 2024-04-13T05:24:10.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.525+0000 7f87d2e14740 1 -- start start 2024-04-13T05:24:10.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.525+0000 7f87d2e14740 1 Processor -- start 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.525+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c863ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c864530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87d2e14740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c865540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87d2e14740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81af670 con 0x55ca6c855e30 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87d2e14740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81b0ae0 con 0x55ca6c855440 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87d2e14740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87c81afd40 con 0x55ca6c85afa0 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87d2e14740 10 client.0 _ll_register_callbacks cb 0x7f87c00114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87cca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c864530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87cca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c864530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60114/0 (socket says 172.21.15.57:60114) 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87cca00640 1 -- 172.21.15.57:0/3957981458 learned_addr learned my addr 172.21.15.57:0/3957981458 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.526+0000 7f87cd400640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c863ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cde00640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c865540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cca00640 1 -- 172.21.15.57:0/3957981458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 msgr2=0x55ca6c865540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cca00640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c865540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cca00640 1 -- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 msgr2=0x55ca6c863ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cca00640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c863ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cca00640 1 -- 172.21.15.57:0/3957981458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6c865b30 con 0x55ca6c855e30 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87cca00640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c864530 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006010 tx=0x7f87b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9f09fa5e167d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.527+0000 7f87aea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.528+0000 7f87aea00640 1 -- 172.21.15.57:0/3957981458 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b40bf2e0 con 0x55ca6c855e30 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.528+0000 7f87aea00640 1 -- 172.21.15.57:0/3957981458 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f87b40035f0 con 0x55ca6c855e30 2024-04-13T05:24:10.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.528+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c0012de0 con 0x55ca6c855e30 2024-04-13T05:24:10.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.528+0000 7f87aea00640 1 -- 172.21.15.57:0/3957981458 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87b40bebe0 con 0x55ca6c855e30 2024-04-13T05:24:10.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.528+0000 7f87d2e14740 10 client.16770 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:10.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.528+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca6c866ea0 con 0x55ca6c855e30 2024-04-13T05:24:10.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87aea00640 1 -- 172.21.15.57:0/3957981458 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87b40c2f90 con 0x55ca6c855e30 2024-04-13T05:24:10.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87aea00640 1 -- 172.21.15.57:0/3957981458 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87b40be030 con 0x55ca6c855e30 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87d2e14740 10 client.16770 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87d2e14740 10 client.16770 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87d2e14740 20 client.16770 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87d2e14740 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c8d2110 0x55ca6c8d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87d2e14740 4 client.16770 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.529+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f87c0012de0 con 0x55ca6c8d2110 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.530+0000 7f87cd400640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c8d2110 0x55ca6c8d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.531+0000 7f87cd400640 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c8d2110 0x55ca6c8d4530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f8ee55dfeaf3242 server_cookie=a71b1be42d1e7481 in_seq=0 out_seq=0 2024-04-13T05:24:10.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.531+0000 7f87aea00640 10 client.16770 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:10.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.532+0000 7f87aea00640 1 -- 172.21.15.57:0/3957981458 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f87c0012de0 con 0x55ca6c8d2110 2024-04-13T05:24:10.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.532+0000 7f87aea00640 10 client.16770 handle_command_reply: tid=0 2024-04-13T05:24:10.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.532+0000 7f87d2e14740 1 client.16770 shutdown 2024-04-13T05:24:10.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 msgr2=0x55ca6c864530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c864530 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006010 tx=0x7f87b4005d50 comp rx=0 tx=0).stop 2024-04-13T05:24:10.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 shutdown_connections 2024-04-13T05:24:10.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55ca6c8d2110 0x55ca6c8d4530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca6c85afa0 0x55ca6c865540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ca6c855e30 0x55ca6c864530 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87aea00640 0 client.16770 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:10.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 --2- 172.21.15.57:0/3957981458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca6c855440 0x55ca6c863ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.533+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 >> 172.21.15.57:0/3957981458 conn(0x55ca6c8509b0 msgr2=0x55ca6c8573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.534+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 shutdown_connections 2024-04-13T05:24:10.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.534+0000 7f87d2e14740 1 -- 172.21.15.57:0/3957981458 wait complete. 2024-04-13T05:24:10.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.534+0000 7f87d2e14740 20 client.16770 trim_cache size 0 max 16384 2024-04-13T05:24:10.537 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:24:10.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87b00a3c30 msgr2=0x7f87b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87b00a3c30 0x7f87b00a60f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55ca6c851920 tx=0x7f87bc00b390 comp rx=0 tx=0).stop 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 msgr2=0x7f87c807a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c807a2b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b40042f0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 shutdown_connections 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87b00a3c30 0x7f87b00a60f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87c8105cb0 0x7f87c80768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87c81052f0 0x7f87c807a2b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 --2- 172.21.15.57:0/3821009256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87c8104100 0x7f87c8079d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 >> 172.21.15.57:0/3821009256 conn(0x7f87c80ff670 msgr2=0x7f87c8108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 shutdown_connections 2024-04-13T05:24:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.536+0000 7f87ce800640 1 -- 172.21.15.57:0/3821009256 wait complete. 2024-04-13T05:24:10.562 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 -- 172.21.15.57:0/1781066567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4078560 msgr2=0x7fe2b4078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/1781066567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4078560 0x7fe2b4078e00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009ca0 tx=0x7fe2ac02c100 comp rx=0 tx=0).stop 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 -- 172.21.15.57:0/1781066567 shutdown_connections 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/1781066567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4078560 0x7fe2b4078e00 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/1781066567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b4077bc0 0x7fe2b4078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/1781066567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b4076980 0x7fe2b4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 -- 172.21.15.57:0/1781066567 >> 172.21.15.57:0/1781066567 conn(0x7fe2b40ff770 msgr2=0x7fe2b4101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 -- 172.21.15.57:0/1781066567 shutdown_connections 2024-04-13T05:24:10.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.817+0000 7fe2bbe00640 1 -- 172.21.15.57:0/1781066567 wait complete. 2024-04-13T05:24:10.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.818+0000 7fe2bbe00640 1 Processor -- start 2024-04-13T05:24:10.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 -- start start 2024-04-13T05:24:10.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 0x7fe2b419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b4077bc0 0x7fe2b419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b4078560 0x7fe2b41a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b4114160 con 0x7fe2b4076980 2024-04-13T05:24:10.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b4113fe0 con 0x7fe2b4077bc0 2024-04-13T05:24:10.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2bbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41142e0 con 0x7fe2b4078560 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2b9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 0x7fe2b419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b4077bc0 0x7fe2b419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2b9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 0x7fe2b419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60140/0 (socket says 172.21.15.57:60140) 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.819+0000 7fe2b9400640 1 -- 172.21.15.57:0/32210036 learned_addr learned my addr 172.21.15.57:0/32210036 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b4078560 0x7fe2b41a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9400640 1 -- 172.21.15.57:0/32210036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b4078560 msgr2=0x7fe2b41a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9400640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b4078560 0x7fe2b41a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9400640 1 -- 172.21.15.57:0/32210036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b4077bc0 msgr2=0x7fe2b419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9400640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b4077bc0 0x7fe2b419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9400640 1 -- 172.21.15.57:0/32210036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b41a6380 con 0x7fe2b4076980 2024-04-13T05:24:10.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.820+0000 7fe2b9400640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 0x7fe2b419e760 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e9f0 tx=0x7fe2a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.821+0000 7fe2b2000640 1 -- 172.21.15.57:0/32210036 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800cd80 con 0x7fe2b4076980 2024-04-13T05:24:10.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.821+0000 7fe2b2000640 1 -- 172.21.15.57:0/32210036 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a80108e0 con 0x7fe2b4076980 2024-04-13T05:24:10.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.821+0000 7fe2b2000640 1 -- 172.21.15.57:0/32210036 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8018920 con 0x7fe2b4076980 2024-04-13T05:24:10.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.821+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b41a6670 con 0x7fe2b4076980 2024-04-13T05:24:10.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.821+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41a6b70 con 0x7fe2b4076980 2024-04-13T05:24:10.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.825+0000 7fe2be922740 1 Processor -- start 2024-04-13T05:24:10.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.825+0000 7fe2be922740 1 -- start start 2024-04-13T05:24:10.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2be922740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec803a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2b2000640 1 -- 172.21.15.57:0/32210036 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe2a8010a90 con 0x7fe2b4076980 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2be922740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec803f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2be922740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec8042240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2be922740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559ec7fa0020 con 0x559ec803ac40 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec803a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec803a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47694/0 (socket says 172.21.15.57:47694) 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.826+0000 7fe2b9e00640 1 -- 172.21.15.57:0/3607721283 learned_addr learned my addr 172.21.15.57:0/3607721283 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.827+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ec7f46950 con 0x559ec803a250 2024-04-13T05:24:10.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.827+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ec7f9bfe0 con 0x559ec803fdb0 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.827+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec803f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.827+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec8042240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.827+0000 7fe2b2000640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe29c0a3c30 0x7fe29c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b2000640 1 -- 172.21.15.57:0/32210036 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe2a80d0bc0 con 0x7fe2b4076980 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9400640 1 -- 172.21.15.57:0/3607721283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 msgr2=0x559ec8042240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec8042240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9400640 1 -- 172.21.15.57:0/3607721283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 msgr2=0x559ec803a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec803a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9400640 1 -- 172.21.15.57:0/3607721283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ec7cb23f0 con 0x559ec803ac40 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec803a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:10.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.828+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec803f740 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ce260 tx=0x7fe2a809c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcf6a6f816d9c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.829+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe29c0a3c30 0x7fe29c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.829+0000 7fe2a6600640 1 -- 172.21.15.57:0/3607721283 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a809d1a0 con 0x559ec803ac40 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.829+0000 7fe2a6600640 1 -- 172.21.15.57:0/3607721283 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a809caa0 con 0x559ec803ac40 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.829+0000 7fe2a6600640 1 -- 172.21.15.57:0/3607721283 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8099110 con 0x559ec803ac40 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 msgr2=0x559ec803f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec803f740 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ce260 tx=0x7fe2a809c930 comp rx=0 tx=0).stop 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 shutdown_connections 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec8042240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec803f740 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 --2- 172.21.15.57:0/3607721283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec803a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 >> 172.21.15.57:0/3607721283 conn(0x559ec80357c0 msgr2=0x559ec8035bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.830+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe29c0a3c30 0x7fe29c0a60f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac032380 tx=0x7fe2ac0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.831+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 shutdown_connections 2024-04-13T05:24:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.831+0000 7fe2be922740 1 -- 172.21.15.57:0/3607721283 wait complete. 2024-04-13T05:24:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.831+0000 7fe2be922740 1 -- start start 2024-04-13T05:24:10.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.831+0000 7fe2be922740 1 Processor -- start 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8044700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec8044c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec804bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559ec8042ca0 con 0x559ec803ac40 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ec7f59c80 con 0x559ec803a250 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ec8042e20 con 0x559ec803fdb0 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2be922740 10 client.0 _ll_register_callbacks cb 0x559ec7f59670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec8044c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec8044c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60154/0 (socket says 172.21.15.57:60154) 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2b8a00640 1 -- 172.21.15.57:0/4074768017 learned_addr learned my addr 172.21.15.57:0/4074768017 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.832+0000 7fe2b9400640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8044700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b8a00640 1 -- 172.21.15.57:0/4074768017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 msgr2=0x559ec804bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec804bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec804bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b8a00640 1 -- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 msgr2=0x559ec8044700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8044700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b8a00640 1 -- 172.21.15.57:0/4074768017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ec7d7c7d0 con 0x559ec803ac40 2024-04-13T05:24:10.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec8044c40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0004650 tx=0x7fe2a00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c21305dbd8043a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe29aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:10.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.833+0000 7fe2b9400640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8044700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:10.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.834+0000 7fe29aa00640 1 -- 172.21.15.57:0/4074768017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0011910 con 0x559ec803ac40 2024-04-13T05:24:10.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.834+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a809caa0 con 0x559ec803ac40 2024-04-13T05:24:10.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.834+0000 7fe29aa00640 1 -- 172.21.15.57:0/4074768017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe2a0011ab0 con 0x559ec803ac40 2024-04-13T05:24:10.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.835+0000 7fe29aa00640 1 -- 172.21.15.57:0/4074768017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2a0011cb0 con 0x559ec803ac40 2024-04-13T05:24:10.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.835+0000 7fe2be922740 10 client.16782 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.835+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ec804f110 con 0x559ec803ac40 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.835+0000 7fe29aa00640 1 -- 172.21.15.57:0/4074768017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a809caa0 con 0x559ec803ac40 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.835+0000 7fe29aa00640 1 -- 172.21.15.57:0/4074768017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a0018070 con 0x559ec803ac40 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2be922740 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2be922740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2be922740 20 client.16782 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2be922740 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec8050780 0x559ec8052c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2be922740 4 client.16782 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ec7f3f750 con 0x559ec8050780 2024-04-13T05:24:10.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.836+0000 7fe2b9400640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec8050780 0x559ec8052c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.837+0000 7fe2b9400640 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec8050780 0x559ec8052c20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e3eae0ff15d0d4 server_cookie=8e52802ddb72f4d4 in_seq=0 out_seq=0 2024-04-13T05:24:10.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.837+0000 7fe29aa00640 10 client.16782 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.839+0000 7fe29aa00640 1 -- 172.21.15.57:0/4074768017 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ec7f3f750 con 0x559ec8050780 2024-04-13T05:24:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.839+0000 7fe29aa00640 10 client.16782 handle_command_reply: tid=0 2024-04-13T05:24:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.839+0000 7fe2be922740 1 client.16782 shutdown 2024-04-13T05:24:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 msgr2=0x559ec8044c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec8044c40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0004650 tx=0x7fe2a00039c0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 shutdown_connections 2024-04-13T05:24:10.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec8050780 0x559ec8052c20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803fdb0 0x559ec804bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803ac40 0x559ec8044c40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 --2- 172.21.15.57:0/4074768017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8044700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 >> 172.21.15.57:0/4074768017 conn(0x559ec80357c0 msgr2=0x559ec8034a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe29aa00640 0 client.16782 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:10.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 shutdown_connections 2024-04-13T05:24:10.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 1 -- 172.21.15.57:0/4074768017 wait complete. 2024-04-13T05:24:10.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.840+0000 7fe2be922740 20 client.16782 trim_cache size 0 max 16384 2024-04-13T05:24:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.863+0000 7fe2be922740 1 Processor -- start 2024-04-13T05:24:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.863+0000 7fe2be922740 1 -- start start 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.863+0000 7fe2be922740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803a250 0x559ec803c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2be922740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec803c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2be922740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec804bc40 0x559ec80341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2be922740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41b3a90 con 0x559ec803fdb0 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2be922740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41b1300 con 0x559ec804bc40 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2be922740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41af150 con 0x559ec803a250 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec803c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec803c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60168/0 (socket says 172.21.15.57:60168) 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2b8a00640 1 -- 172.21.15.57:0/2814189401 learned_addr learned my addr 172.21.15.57:0/2814189401 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec804bc40 0x559ec80341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2b9400640 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803a250 0x559ec803c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.864+0000 7fe2b8a00640 1 -- 172.21.15.57:0/2814189401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803a250 msgr2=0x559ec803c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803a250 0x559ec803c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2b8a00640 1 -- 172.21.15.57:0/2814189401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec804bc40 msgr2=0x559ec80341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec804bc40 0x559ec80341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2b8a00640 1 -- 172.21.15.57:0/2814189401 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ec804ef30 con 0x559ec803fdb0 2024-04-13T05:24:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec803c680 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00195c0 tx=0x7fe2a0004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb121db858976c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2a5c00640 1 -- 172.21.15.57:0/2814189401 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0011910 con 0x559ec803fdb0 2024-04-13T05:24:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2a5c00640 1 -- 172.21.15.57:0/2814189401 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a002d070 con 0x559ec803fdb0 2024-04-13T05:24:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.865+0000 7fe2a5c00640 1 -- 172.21.15.57:0/2814189401 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0019d40 con 0x559ec803fdb0 2024-04-13T05:24:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.866+0000 7fe2be922740 1 -- 172.21.15.57:0/2814189401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 msgr2=0x559ec803c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.866+0000 7fe2be922740 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec803c680 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00195c0 tx=0x7fe2a0004000 comp rx=0 tx=0).stop 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.866+0000 7fe2be922740 1 -- 172.21.15.57:0/2814189401 shutdown_connections 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.866+0000 7fe2be922740 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec804bc40 0x559ec80341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.866+0000 7fe2be922740 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec803c680 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.866+0000 7fe2be922740 1 --2- 172.21.15.57:0/2814189401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec803a250 0x559ec803c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.867+0000 7fe2be922740 1 -- 172.21.15.57:0/2814189401 >> 172.21.15.57:0/2814189401 conn(0x559ec80357c0 msgr2=0x559ec8035ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.867+0000 7fe2be922740 1 -- 172.21.15.57:0/2814189401 shutdown_connections 2024-04-13T05:24:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.867+0000 7fe2be922740 1 -- 172.21.15.57:0/2814189401 wait complete. 2024-04-13T05:24:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 -- start start 2024-04-13T05:24:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 Processor -- start 2024-04-13T05:24:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8049360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec80498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec804bc40 0x559ec804a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41af7c0 con 0x559ec803fdb0 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41b0c30 con 0x559ec803a250 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41afe90 con 0x559ec804bc40 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.868+0000 7fe2be922740 10 client.0 _ll_register_callbacks cb 0x7fe2a000ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec80498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec80498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60178/0 (socket says 172.21.15.57:60178) 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 -- 172.21.15.57:0/3707357209 learned_addr learned my addr 172.21.15.57:0/3707357209 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:10.872 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8049360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec804bc40 0x559ec804a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 -- 172.21.15.57:0/3707357209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec804bc40 msgr2=0x559ec804a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec804bc40 0x559ec804a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 -- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 msgr2=0x559ec8049360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8049360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.869+0000 7fe2b8a00640 1 -- 172.21.15.57:0/3707357209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ec804aea0 con 0x559ec803fdb0 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.870+0000 7fe2b9e00640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec804bc40 0x559ec804a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.870+0000 7fe2b8a00640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec80498a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000c00 tx=0x7fe2a0019c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29406ad16348c2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.870+0000 7fe29aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.871+0000 7fe29aa00640 1 -- 172.21.15.57:0/3707357209 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000c4b0 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.871+0000 7fe29aa00640 1 -- 172.21.15.57:0/3707357209 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe2a0005460 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.871+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a002d070 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.871+0000 7fe29aa00640 1 -- 172.21.15.57:0/3707357209 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2a0042050 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.871+0000 7fe2be922740 10 client.16788 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.871+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ec80b72d0 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe29aa00640 1 -- 172.21.15.57:0/3707357209 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a002d070 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe29aa00640 1 -- 172.21.15.57:0/3707357209 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a0018070 con 0x559ec803fdb0 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe2be922740 10 client.16788 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe2be922740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe2be922740 20 client.16788 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe2be922740 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec80b7ca0 0x559ec80ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe2be922740 4 client.16788 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:24:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.872+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ec7fada40 con 0x559ec80b7ca0 2024-04-13T05:24:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.873+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec80b7ca0 0x559ec80ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.873+0000 7fe2b9400640 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec80b7ca0 0x559ec80ba0c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce8591880c312f3 server_cookie=30fc863d61b0fbd3 in_seq=0 out_seq=0 2024-04-13T05:24:10.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.873+0000 7fe29aa00640 10 client.16788 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe29aa00640 1 -- 172.21.15.57:0/3707357209 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ec7fada40 con 0x559ec80b7ca0 2024-04-13T05:24:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe29aa00640 10 client.16788 handle_command_reply: tid=0 2024-04-13T05:24:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 client.16788 shutdown 2024-04-13T05:24:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 msgr2=0x559ec80498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec80498a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000c00 tx=0x7fe2a0019c70 comp rx=0 tx=0).stop 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 shutdown_connections 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559ec80b7ca0 0x559ec80ba0c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ec804bc40 0x559ec804a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ec803fdb0 0x559ec80498a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 --2- 172.21.15.57:0/3707357209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ec803a250 0x559ec8049360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.874+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 >> 172.21.15.57:0/3707357209 conn(0x559ec80357c0 msgr2=0x559ec80431f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.875+0000 7fe29aa00640 0 client.16788 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.875+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 shutdown_connections 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.875+0000 7fe2be922740 1 -- 172.21.15.57:0/3707357209 wait complete. 2024-04-13T05:24:10.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.875+0000 7fe2be922740 20 client.16788 trim_cache size 0 max 16384 2024-04-13T05:24:10.879 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:24:10.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe29c0a3c30 msgr2=0x7fe29c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe29c0a3c30 0x7fe29c0a60f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac032380 tx=0x7fe2ac0095c0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 msgr2=0x7fe2b419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 0x7fe2b419e760 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800e9f0 tx=0x7fe2a800eeb0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 shutdown_connections 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe29c0a3c30 0x7fe29c0a60f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b4077bc0 0x7fe2b419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b4078560 0x7fe2b41a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 --2- 172.21.15.57:0/32210036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2b4076980 0x7fe2b419e760 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 >> 172.21.15.57:0/32210036 conn(0x7fe2b40ff770 msgr2=0x7fe2b4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.876+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 shutdown_connections 2024-04-13T05:24:10.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:10.877+0000 7fe2bbe00640 1 -- 172.21.15.57:0/32210036 wait complete. 2024-04-13T05:24:11.153 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:10 smithi083 ceph-mon[35133]: pgmap v653: 129 pgs: 129 active+clean; 398 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 133 op/s 2024-04-13T05:24:11.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:10 smithi057 ceph-mon[32331]: pgmap v653: 129 pgs: 129 active+clean; 398 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 133 op/s 2024-04-13T05:24:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:10 smithi189 ceph-mon[35531]: pgmap v653: 129 pgs: 129 active+clean; 398 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 133 op/s 2024-04-13T05:24:11.904 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:24:11.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:24:11.904 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:24:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.167+0000 7feba1a00640 1 -- 172.21.15.57:0/292505196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 msgr2=0x7feb9c10a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.167+0000 7feba1a00640 1 --2- 172.21.15.57:0/292505196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c10a450 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009ca0 tx=0x7feb8c02c100 comp rx=0 tx=0).stop 2024-04-13T05:24:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 -- 172.21.15.57:0/292505196 shutdown_connections 2024-04-13T05:24:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 --2- 172.21.15.57:0/292505196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c103200 0x7feb9c103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 --2- 172.21.15.57:0/292505196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c10a450 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 --2- 172.21.15.57:0/292505196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9c102010 0x7feb9c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 -- 172.21.15.57:0/292505196 >> 172.21.15.57:0/292505196 conn(0x7feb9c0fd5a0 msgr2=0x7feb9c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 -- 172.21.15.57:0/292505196 shutdown_connections 2024-04-13T05:24:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.168+0000 7feba1a00640 1 -- 172.21.15.57:0/292505196 wait complete. 2024-04-13T05:24:12.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.169+0000 7feba1a00640 1 Processor -- start 2024-04-13T05:24:12.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.169+0000 7feba1a00640 1 -- start start 2024-04-13T05:24:12.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feba1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c102010 0x7feb9c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feba1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9c103200 0x7feb9c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feba1a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feba1a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c10db10 con 0x7feb9c103bc0 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feba1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c10d990 con 0x7feb9c102010 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feba1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c10dc90 con 0x7feb9c103200 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c102010 0x7feb9c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feb9b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c102010 0x7feb9c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47730/0 (socket says 172.21.15.57:47730) 2024-04-13T05:24:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.170+0000 7feb9aa00640 1 -- 172.21.15.57:0/2242900978 learned_addr learned my addr 172.21.15.57:0/2242900978 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.171+0000 7feb9a000640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9c103200 0x7feb9c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.171+0000 7feb9b400640 1 -- 172.21.15.57:0/2242900978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9c103200 msgr2=0x7feb9c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.171+0000 7feb9b400640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9c103200 0x7feb9c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.171+0000 7feb9b400640 1 -- 172.21.15.57:0/2242900978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c102010 msgr2=0x7feb9c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.171+0000 7feb9b400640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c102010 0x7feb9c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.171+0000 7feb9b400640 1 -- 172.21.15.57:0/2242900978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c19fe60 con 0x7feb9c103bc0 2024-04-13T05:24:12.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.172+0000 7feb9b400640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c19f7f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feb8c037040 tx=0x7feb8c035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.172+0000 7feb93400640 1 -- 172.21.15.57:0/2242900978 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c0327c0 con 0x7feb9c103bc0 2024-04-13T05:24:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.172+0000 7feb93400640 1 -- 172.21.15.57:0/2242900978 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c032960 con 0x7feb9c103bc0 2024-04-13T05:24:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.172+0000 7feb93400640 1 -- 172.21.15.57:0/2242900978 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004840 con 0x7feb9c103bc0 2024-04-13T05:24:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.172+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c0741f0 con 0x7feb9c103bc0 2024-04-13T05:24:12.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.173+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c074630 con 0x7feb9c103bc0 2024-04-13T05:24:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.176+0000 7feba49bb740 1 Processor -- start 2024-04-13T05:24:12.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.177+0000 7feb93400640 1 -- 172.21.15.57:0/2242900978 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feb8c04d060 con 0x7feb9c103bc0 2024-04-13T05:24:12.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.177+0000 7feba49bb740 1 -- start start 2024-04-13T05:24:12.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.178+0000 7feba49bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c7226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.178+0000 7feba49bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c727770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.178+0000 7feba49bb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c72a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.178+0000 7feba49bb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c29c688bc0 con 0x55c29c727de0 2024-04-13T05:24:12.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.178+0000 7feba49bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c29c62f660 con 0x55c29c722280 2024-04-13T05:24:12.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.178+0000 7feba49bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c29c684b80 con 0x55c29c722c70 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c727770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c727770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:53716/0 (socket says 172.21.15.57:53716) 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9aa00640 1 -- 172.21.15.57:0/3032319812 learned_addr learned my addr 172.21.15.57:0/3032319812 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9a000640 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c72a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb93400640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feb800a3c80 0x7feb800a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9a000640 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c7226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb93400640 1 -- 172.21.15.57:0/2242900978 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7feb8c0f62f0 con 0x7feb9c103bc0 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9a000640 1 -- 172.21.15.57:0/3032319812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 msgr2=0x55c29c727770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9a000640 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c727770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9a000640 1 -- 172.21.15.57:0/3032319812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 msgr2=0x55c29c7226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.179+0000 7feb9a000640 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c7226a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.180+0000 7feb9a000640 1 -- 172.21.15.57:0/3032319812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29c652220 con 0x55c29c727de0 2024-04-13T05:24:12.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.180+0000 7feb9a000640 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c72a270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7feb9c199750 tx=0x7feb84005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f757dc087501c3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.180+0000 7feb9b400640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feb800a3c80 0x7feb800a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.180+0000 7feb8be00640 1 -- 172.21.15.57:0/3032319812 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84011470 con 0x55c29c727de0 2024-04-13T05:24:12.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.181+0000 7feb8be00640 1 -- 172.21.15.57:0/3032319812 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb84011ac0 con 0x55c29c727de0 2024-04-13T05:24:12.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.181+0000 7feb8be00640 1 -- 172.21.15.57:0/3032319812 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84010a50 con 0x55c29c727de0 2024-04-13T05:24:12.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.181+0000 7feba49bb740 1 -- 172.21.15.57:0/3032319812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 msgr2=0x55c29c72a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.181+0000 7feba49bb740 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c72a270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7feb9c199750 tx=0x7feb84005f90 comp rx=0 tx=0).stop 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.181+0000 7feb9b400640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feb800a3c80 0x7feb800a6140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0ca910 tx=0x7feb8c02c620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 -- 172.21.15.57:0/3032319812 shutdown_connections 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c72a270 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c727770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 --2- 172.21.15.57:0/3032319812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c7226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 -- 172.21.15.57:0/3032319812 >> 172.21.15.57:0/3032319812 conn(0x55c29c71d7f0 msgr2=0x55c29c71dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 -- 172.21.15.57:0/3032319812 shutdown_connections 2024-04-13T05:24:12.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.182+0000 7feba49bb740 1 -- 172.21.15.57:0/3032319812 wait complete. 2024-04-13T05:24:12.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.183+0000 7feba49bb740 1 -- start start 2024-04-13T05:24:12.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.183+0000 7feba49bb740 1 Processor -- start 2024-04-13T05:24:12.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.183+0000 7feba49bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722280 0x55c29c72c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.183+0000 7feba49bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 0x55c29c72cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.183+0000 7feba49bb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feba49bb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c29c72ad90 con 0x55c29c727de0 2024-04-13T05:24:12.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feba49bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c29c642a30 con 0x55c29c722c70 2024-04-13T05:24:12.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feba49bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c29c72af10 con 0x55c29c722280 2024-04-13T05:24:12.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feba49bb740 10 client.0 _ll_register_callbacks cb 0x55c29c39ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:12.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 0x55c29c72cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 0x55c29c72cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47748/0 (socket says 172.21.15.57:47748) 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feb9aa00640 1 -- 172.21.15.57:0/596628128 learned_addr learned my addr 172.21.15.57:0/596628128 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feb9a000640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.184+0000 7feb9a000640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722280 0x55c29c72c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb9aa00640 1 -- 172.21.15.57:0/596628128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722280 msgr2=0x55c29c72c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb9aa00640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722280 0x55c29c72c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb9aa00640 1 -- 172.21.15.57:0/596628128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 msgr2=0x55c29c733de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb9aa00640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb9aa00640 1 -- 172.21.15.57:0/596628128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29c4637e0 con 0x55c29c722c70 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb9aa00640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 0x55c29c72cda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb940116c0 tx=0x7feb94011b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde37fcc4e2b6695 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.185+0000 7feb88c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.186+0000 7feb88c00640 1 -- 172.21.15.57:0/596628128 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94010ba0 con 0x55c29c722c70 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.186+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb84011ac0 con 0x55c29c722c70 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.186+0000 7feb88c00640 1 -- 172.21.15.57:0/596628128 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7feb94010d40 con 0x55c29c722c70 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.186+0000 7feb88c00640 1 -- 172.21.15.57:0/596628128 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb94004020 con 0x55c29c722c70 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.186+0000 7feba49bb740 10 client.36209 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:12.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.186+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c29c4c3f70 con 0x55c29c722c70 2024-04-13T05:24:12.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.187+0000 7feb88c00640 1 -- 172.21.15.57:0/596628128 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb84011ac0 con 0x55c29c722c70 2024-04-13T05:24:12.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.187+0000 7feb88c00640 1 -- 172.21.15.57:0/596628128 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feb9401b470 con 0x55c29c722c70 2024-04-13T05:24:12.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feba49bb740 10 client.36209 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:12.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feba49bb740 10 client.36209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:12.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feba49bb740 20 client.36209 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:12.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feba49bb740 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7386a0 0x55c29c73aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feba49bb740 4 client.36209 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:12.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c29c6965e0 con 0x55c29c7386a0 2024-04-13T05:24:12.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.188+0000 7feb9a000640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7386a0 0x55c29c73aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.189+0000 7feb88c00640 10 client.36209 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.189+0000 7feb9a000640 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7386a0 0x55c29c73aab0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4587237e406725e server_cookie=a428774028bcfb5 in_seq=0 out_seq=0 2024-04-13T05:24:12.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feb88c00640 1 -- 172.21.15.57:0/596628128 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c29c6965e0 con 0x55c29c7386a0 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feb88c00640 10 client.36209 handle_command_reply: tid=0 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 client.36209 shutdown 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 msgr2=0x55c29c72cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 0x55c29c72cda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb940116c0 tx=0x7feb94011b80 comp rx=0 tx=0).stop 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 shutdown_connections 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7386a0 0x55c29c73aab0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feb88c00640 0 client.36209 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722c70 0x55c29c72cda0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 --2- 172.21.15.57:0/596628128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722280 0x55c29c72c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.192+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 >> 172.21.15.57:0/596628128 conn(0x55c29c71d7f0 msgr2=0x55c29c71c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.193+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 shutdown_connections 2024-04-13T05:24:12.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.193+0000 7feba49bb740 1 -- 172.21.15.57:0/596628128 wait complete. 2024-04-13T05:24:12.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.193+0000 7feba49bb740 20 client.36209 trim_cache size 0 max 16384 2024-04-13T05:24:12.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.222+0000 7feba49bb740 1 Processor -- start 2024-04-13T05:24:12.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.222+0000 7feba49bb740 1 -- start start 2024-04-13T05:24:12.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feba49bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c71c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feba49bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c733da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feba49bb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c71c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feba49bb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1ab320 con 0x55c29c727de0 2024-04-13T05:24:12.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feba49bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1a8b90 con 0x55c29c722280 2024-04-13T05:24:12.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feba49bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1a69e0 con 0x55c29c722c70 2024-04-13T05:24:12.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feb9a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c71c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feb9a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c71c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47756/0 (socket says 172.21.15.57:47756) 2024-04-13T05:24:12.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feb9a000640 1 -- 172.21.15.57:0/139995195 learned_addr learned my addr 172.21.15.57:0/139995195 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.223+0000 7feb9a000640 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c71c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.224+0000 7feb9aa00640 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c733da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.224+0000 7feb9a000640 1 -- 172.21.15.57:0/139995195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 msgr2=0x55c29c733da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.224+0000 7feb9a000640 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c733da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.224+0000 7feb9a000640 1 -- 172.21.15.57:0/139995195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 msgr2=0x55c29c71c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.224+0000 7feb9a000640 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c71c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.224+0000 7feb9a000640 1 -- 172.21.15.57:0/139995195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29c35b170 con 0x55c29c722280 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.225+0000 7feb9a000640 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c71c160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb8400fc80 tx=0x7feb84015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa92588e31cb0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.225+0000 7feb8b400640 1 -- 172.21.15.57:0/139995195 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8400fdf0 con 0x55c29c722280 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.225+0000 7feb8b400640 1 -- 172.21.15.57:0/139995195 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb84005bb0 con 0x55c29c722280 2024-04-13T05:24:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.225+0000 7feb8b400640 1 -- 172.21.15.57:0/139995195 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84010a10 con 0x55c29c722280 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 -- 172.21.15.57:0/139995195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 msgr2=0x55c29c71c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c71c160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb8400fc80 tx=0x7feb84015040 comp rx=0 tx=0).stop 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 -- 172.21.15.57:0/139995195 shutdown_connections 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c71c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c733da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 --2- 172.21.15.57:0/139995195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c71c160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 -- 172.21.15.57:0/139995195 >> 172.21.15.57:0/139995195 conn(0x55c29c71d7f0 msgr2=0x55c29c724240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.226+0000 7feba49bb740 1 -- 172.21.15.57:0/139995195 shutdown_connections 2024-04-13T05:24:12.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.227+0000 7feba49bb740 1 -- 172.21.15.57:0/139995195 wait complete. 2024-04-13T05:24:12.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.227+0000 7feba49bb740 1 -- start start 2024-04-13T05:24:12.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.227+0000 7feba49bb740 1 Processor -- start 2024-04-13T05:24:12.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c731d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c732240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1a7050 con 0x55c29c727de0 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1a84c0 con 0x55c29c722280 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1a7720 con 0x55c29c722c70 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feba49bb740 10 client.0 _ll_register_callbacks cb 0x7feb9400b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feb9a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c731d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.228+0000 7feb9a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c731d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47762/0 (socket says 172.21.15.57:47762) 2024-04-13T05:24:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 -- 172.21.15.57:0/1290342003 learned_addr learned my addr 172.21.15.57:0/1290342003 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 -- 172.21.15.57:0/1290342003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 msgr2=0x55c29c732240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c732240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 -- 172.21.15.57:0/1290342003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 msgr2=0x55c29c733250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.229+0000 7feb9a000640 1 -- 172.21.15.57:0/1290342003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29c643be0 con 0x55c29c722280 2024-04-13T05:24:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.230+0000 7feb9a000640 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c731d00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb840109b0 tx=0x7feb84000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeee983c28e27c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.230+0000 7feb88c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.230+0000 7feb88c00640 1 -- 172.21.15.57:0/1290342003 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84010530 con 0x55c29c722280 2024-04-13T05:24:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.230+0000 7feb88c00640 1 -- 172.21.15.57:0/1290342003 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7feb84010a10 con 0x55c29c722280 2024-04-13T05:24:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.230+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb84005bb0 con 0x55c29c722280 2024-04-13T05:24:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.231+0000 7feb88c00640 1 -- 172.21.15.57:0/1290342003 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb84010c10 con 0x55c29c722280 2024-04-13T05:24:12.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.231+0000 7feba49bb740 10 client.36215 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:12.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.231+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c29c7a7a80 con 0x55c29c722280 2024-04-13T05:24:12.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.231+0000 7feb88c00640 1 -- 172.21.15.57:0/1290342003 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb840054d0 con 0x55c29c722280 2024-04-13T05:24:12.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feb88c00640 1 -- 172.21.15.57:0/1290342003 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feb8402c070 con 0x55c29c722280 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feba49bb740 10 client.36215 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feba49bb740 10 client.36215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feba49bb740 20 client.36215 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feba49bb740 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7a8200 0x55c29c7aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feba49bb740 4 client.36215 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7feb84005bb0 con 0x55c29c7a8200 2024-04-13T05:24:12.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.232+0000 7feb9aa00640 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7a8200 0x55c29c7aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.233+0000 7feb88c00640 10 client.36215 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.233+0000 7feb9aa00640 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7a8200 0x55c29c7aa620 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d3e0944d2adaea7 server_cookie=316f364f32f0d5d7 in_seq=0 out_seq=0 2024-04-13T05:24:12.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.235+0000 7feb88c00640 1 -- 172.21.15.57:0/1290342003 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7feb84005bb0 con 0x55c29c7a8200 2024-04-13T05:24:12.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.235+0000 7feb88c00640 10 client.36215 handle_command_reply: tid=0 2024-04-13T05:24:12.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.235+0000 7feba49bb740 1 client.36215 shutdown 2024-04-13T05:24:12.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 msgr2=0x55c29c731d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c731d00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb840109b0 tx=0x7feb84000bc0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 shutdown_connections 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c29c7a8200 0x55c29c7aa620 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c29c727de0 0x55c29c733250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c29c722c70 0x55c29c732240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 --2- 172.21.15.57:0/1290342003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c29c722280 0x55c29c731d00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 >> 172.21.15.57:0/1290342003 conn(0x55c29c71d7f0 msgr2=0x55c29c720a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feb88c00640 0 client.36215 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 shutdown_connections 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 1 -- 172.21.15.57:0/1290342003 wait complete. 2024-04-13T05:24:12.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.236+0000 7feba49bb740 20 client.36215 trim_cache size 0 max 16384 2024-04-13T05:24:12.240 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:24:12.240 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:24:12.240 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "57cc08af-0b4b-452f-a310-93ed60acf2f1", 2024-04-13T05:24:12.240 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:24:12.240 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.237+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feb800a3c80 msgr2=0x7feb800a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.237+0000 7feba1a00640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feb800a3c80 0x7feb800a6140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0ca910 tx=0x7feb8c02c620 comp rx=0 tx=0).stop 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.237+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 msgr2=0x7feb9c19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.237+0000 7feba1a00640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c19f7f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feb8c037040 tx=0x7feb8c035ce0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 shutdown_connections 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7feb800a3c80 0x7feb800a6140 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb9c103bc0 0x7feb9c19f7f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9c103200 0x7feb9c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 --2- 172.21.15.57:0/2242900978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9c102010 0x7feb9c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 >> 172.21.15.57:0/2242900978 conn(0x7feb9c0fd5a0 msgr2=0x7feb9c106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.238+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 shutdown_connections 2024-04-13T05:24:12.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.239+0000 7feba1a00640 1 -- 172.21.15.57:0/2242900978 wait complete. 2024-04-13T05:24:12.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:24:12.266 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:24:12.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 -- 172.21.15.57:0/593114234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4108fc0 msgr2=0x7f4ee41093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 --2- 172.21.15.57:0/593114234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4108fc0 0x7f4ee41093e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009cd0 tx=0x7f4edc02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 -- 172.21.15.57:0/593114234 shutdown_connections 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 --2- 172.21.15.57:0/593114234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ee4110840 0x7f4ee4112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 --2- 172.21.15.57:0/593114234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee4109920 0x7f4ee4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 --2- 172.21.15.57:0/593114234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4108fc0 0x7f4ee41093e0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009cd0 tx=0x7f4edc02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.567+0000 7f4eeac00640 1 -- 172.21.15.57:0/593114234 >> 172.21.15.57:0/593114234 conn(0x7f4ee406ee50 msgr2=0x7f4ee4071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.568+0000 7f4eeac00640 1 -- 172.21.15.57:0/593114234 shutdown_connections 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.568+0000 7f4eeac00640 1 -- 172.21.15.57:0/593114234 wait complete. 2024-04-13T05:24:12.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 Processor -- start 2024-04-13T05:24:12.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 -- start start 2024-04-13T05:24:12.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 0x7f4ee41a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee4110840 0x7f4ee41a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ee41ac890 0x7f4ee41aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4116340 con 0x7f4ee4110840 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41161c0 con 0x7f4ee4109920 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.569+0000 7f4eeac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41164c0 con 0x7f4ee41ac890 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.570+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 0x7f4ee41a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.570+0000 7f4ee3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee4110840 0x7f4ee41a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.570+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 0x7f4ee41a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47788/0 (socket says 172.21.15.57:47788) 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.570+0000 7f4ee3e00640 1 -- 172.21.15.57:0/3133766122 learned_addr learned my addr 172.21.15.57:0/3133766122 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.571+0000 7f4ee3e00640 1 -- 172.21.15.57:0/3133766122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ee41ac890 msgr2=0x7f4ee41aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.571+0000 7f4ee3e00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ee41ac890 0x7f4ee41aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.571+0000 7f4ee3e00640 1 -- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee4110840 msgr2=0x7f4ee41a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.571+0000 7f4ee3e00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee4110840 0x7f4ee41a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.571+0000 7f4ee3e00640 1 -- 172.21.15.57:0/3133766122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41af2f0 con 0x7f4ee4109920 2024-04-13T05:24:12.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.571+0000 7f4ee3e00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 0x7f4ee41a72f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4edc006010 tx=0x7f4edc003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.572+0000 7f4ee0c00640 1 -- 172.21.15.57:0/3133766122 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc039050 con 0x7f4ee4109920 2024-04-13T05:24:12.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.572+0000 7f4ee0c00640 1 -- 172.21.15.57:0/3133766122 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4edc03a040 con 0x7f4ee4109920 2024-04-13T05:24:12.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.572+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41af580 con 0x7f4ee4109920 2024-04-13T05:24:12.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.572+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41afa20 con 0x7f4ee4109920 2024-04-13T05:24:12.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.573+0000 7f4ee0c00640 1 -- 172.21.15.57:0/3133766122 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc032b50 con 0x7f4ee4109920 2024-04-13T05:24:12.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.578+0000 7f4eeda55740 1 Processor -- start 2024-04-13T05:24:12.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.579+0000 7f4eeda55740 1 -- start start 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c38d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c3929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c3954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4eeda55740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e99c2f4020 con 0x55e99c38d4d0 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4eeda55740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e99c29a6d0 con 0x55e99c393030 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4eeda55740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e99c2effe0 con 0x55e99c38dec0 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4ee3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c3954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4ee3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c3954c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47804/0 (socket says 172.21.15.57:47804) 2024-04-13T05:24:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.580+0000 7f4ee3400640 1 -- 172.21.15.57:0/3760775391 learned_addr learned my addr 172.21.15.57:0/3760775391 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.581+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c38d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.581+0000 7f4ee0c00640 1 -- 172.21.15.57:0/3133766122 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4edc047070 con 0x7f4ee4109920 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.582+0000 7f4ee8c00640 1 -- 172.21.15.57:0/3760775391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 msgr2=0x55e99c3929c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.582+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c3929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.582+0000 7f4ee8c00640 1 -- 172.21.15.57:0/3760775391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 msgr2=0x55e99c3954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.582+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c3954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.582+0000 7f4ee8c00640 1 -- 172.21.15.57:0/3760775391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99c0d07d0 con 0x55e99c38d4d0 2024-04-13T05:24:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.582+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c38d8f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4ee406bce0 tx=0x7f4ed000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959b5aa8be4f14db server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.583+0000 7f4ed9200640 1 -- 172.21.15.57:0/3760775391 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0016070 con 0x55e99c38d4d0 2024-04-13T05:24:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.583+0000 7f4ed9200640 1 -- 172.21.15.57:0/3760775391 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed000dd70 con 0x55e99c38d4d0 2024-04-13T05:24:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.583+0000 7f4ed9200640 1 -- 172.21.15.57:0/3760775391 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00047a0 con 0x55e99c38d4d0 2024-04-13T05:24:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.583+0000 7f4ee0c00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4ee0c00640 1 -- 172.21.15.57:0/3133766122 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4edc0f2640 con 0x7f4ee4109920 2024-04-13T05:24:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4ee3e00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 -- 172.21.15.57:0/3760775391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 msgr2=0x55e99c38d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c38d8f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4ee406bce0 tx=0x7f4ed000bdb0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 -- 172.21.15.57:0/3760775391 shutdown_connections 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c3954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c3929c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 --2- 172.21.15.57:0/3760775391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c38d8f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.584+0000 7f4eeda55740 1 -- 172.21.15.57:0/3760775391 >> 172.21.15.57:0/3760775391 conn(0x55e99c388a40 msgr2=0x55e99c388e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.585+0000 7f4eeda55740 1 -- 172.21.15.57:0/3760775391 shutdown_connections 2024-04-13T05:24:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.585+0000 7f4eeda55740 1 -- 172.21.15.57:0/3760775391 wait complete. 2024-04-13T05:24:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.585+0000 7f4ee3e00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0be280 tx=0x7f4edc0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.586+0000 7f4eeda55740 1 -- start start 2024-04-13T05:24:12.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.586+0000 7f4eeda55740 1 Processor -- start 2024-04-13T05:24:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38d4d0 0x55e99c3977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 0x55e99c397d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e99c396030 con 0x55e99c38dec0 2024-04-13T05:24:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e99c2ada90 con 0x55e99c393030 2024-04-13T05:24:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e99c3961b0 con 0x55e99c38d4d0 2024-04-13T05:24:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4eeda55740 10 client.0 _ll_register_callbacks cb 0x55e99c2ad480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4ee3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 0x55e99c397d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.587+0000 7f4ee3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 0x55e99c397d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60282/0 (socket says 172.21.15.57:60282) 2024-04-13T05:24:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee3400640 1 -- 172.21.15.57:0/4144690502 learned_addr learned my addr 172.21.15.57:0/4144690502 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee3400640 1 -- 172.21.15.57:0/4144690502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38d4d0 msgr2=0x55e99c3977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee3400640 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38d4d0 0x55e99c3977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee3400640 1 -- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 msgr2=0x55e99c39ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee3400640 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.588+0000 7f4ee3400640 1 -- 172.21.15.57:0/4144690502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99c0063f0 con 0x55e99c38dec0 2024-04-13T05:24:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.589+0000 7f4ee3400640 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 0x55e99c397d00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400ba20 tx=0x7f4ed400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19f2edbed01350b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.589+0000 7f4ecd600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.589+0000 7f4ecd600640 1 -- 172.21.15.57:0/4144690502 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4003e80 con 0x55e99c38dec0 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.590+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed000dd70 con 0x55e99c38dec0 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.590+0000 7f4ecd600640 1 -- 172.21.15.57:0/4144690502 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4ed4011440 con 0x55e99c38dec0 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.590+0000 7f4ecd600640 1 -- 172.21.15.57:0/4144690502 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ed4011640 con 0x55e99c38dec0 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.590+0000 7f4eeda55740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.590+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e99bfc60f0 con 0x55e99c38dec0 2024-04-13T05:24:12.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4ecd600640 1 -- 172.21.15.57:0/4144690502 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed000dd70 con 0x55e99c38dec0 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4ecd600640 1 -- 172.21.15.57:0/4144690502 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ed401ba60 con 0x55e99c38dec0 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4eeda55740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4eeda55740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4eeda55740 20 client.16818 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4eeda55740 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c3a3770 0x55e99c3a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4eeda55740 4 client.16818 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:12.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.591+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e99c293600 con 0x55e99c3a3770 2024-04-13T05:24:12.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.592+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c3a3770 0x55e99c3a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.593+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c3a3770 0x55e99c3a5bd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f6745582c91d2f server_cookie=d84d22467ed96a3d in_seq=0 out_seq=0 2024-04-13T05:24:12.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.593+0000 7f4ecd600640 10 client.16818 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4ecd600640 1 -- 172.21.15.57:0/4144690502 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e99c293600 con 0x55e99c3a3770 2024-04-13T05:24:12.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4ecd600640 10 client.16818 handle_command_reply: tid=0 2024-04-13T05:24:12.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 client.16818 shutdown 2024-04-13T05:24:12.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 msgr2=0x55e99c397d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 0x55e99c397d00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400ba20 tx=0x7f4ed400bee0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 shutdown_connections 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c3a3770 0x55e99c3a5bd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38dec0 0x55e99c397d00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 --2- 172.21.15.57:0/4144690502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38d4d0 0x55e99c3977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 >> 172.21.15.57:0/4144690502 conn(0x55e99c388a40 msgr2=0x55e99c387b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.596+0000 7f4ecd600640 0 client.16818 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.597+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 shutdown_connections 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.597+0000 7f4eeda55740 1 -- 172.21.15.57:0/4144690502 wait complete. 2024-04-13T05:24:12.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.597+0000 7f4eeda55740 20 client.16818 trim_cache size 0 max 16384 2024-04-13T05:24:12.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.638+0000 7f4eeda55740 1 Processor -- start 2024-04-13T05:24:12.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.638+0000 7f4eeda55740 1 -- start start 2024-04-13T05:24:12.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c38d4d0 0x55e99c3965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 0x55e99c38f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4eeda55740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41bc870 con 0x55e99c393030 2024-04-13T05:24:12.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4eeda55740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41ba0e0 con 0x55e99c38d4d0 2024-04-13T05:24:12.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4eeda55740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41b7f30 con 0x55e99c38dec0 2024-04-13T05:24:12.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.639+0000 7f4ee8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 0x55e99c38f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 0x55e99c38f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60286/0 (socket says 172.21.15.57:60286) 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 -- 172.21.15.57:0/156162549 learned_addr learned my addr 172.21.15.57:0/156162549 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c38d4d0 0x55e99c3965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee3400640 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 -- 172.21.15.57:0/156162549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 msgr2=0x55e99c39ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 -- 172.21.15.57:0/156162549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c38d4d0 msgr2=0x55e99c3965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c38d4d0 0x55e99c3965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.640+0000 7f4ee8c00640 1 -- 172.21.15.57:0/156162549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99c398660 con 0x55e99c393030 2024-04-13T05:24:12.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.641+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 0x55e99c38f100 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005940 tx=0x7f4ed0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8818d9078e3fe9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.642+0000 7f4ecfe00640 1 -- 172.21.15.57:0/156162549 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed002d070 con 0x55e99c393030 2024-04-13T05:24:12.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.642+0000 7f4ecfe00640 1 -- 172.21.15.57:0/156162549 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed000de70 con 0x55e99c393030 2024-04-13T05:24:12.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.642+0000 7f4ecfe00640 1 -- 172.21.15.57:0/156162549 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0016040 con 0x55e99c393030 2024-04-13T05:24:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.643+0000 7f4eeda55740 1 -- 172.21.15.57:0/156162549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 msgr2=0x55e99c38f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.643+0000 7f4eeda55740 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 0x55e99c38f100 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005940 tx=0x7f4ed0005e50 comp rx=0 tx=0).stop 2024-04-13T05:24:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 -- 172.21.15.57:0/156162549 shutdown_connections 2024-04-13T05:24:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c393030 0x55e99c38f100 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 --2- 172.21.15.57:0/156162549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c38d4d0 0x55e99c3965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 -- 172.21.15.57:0/156162549 >> 172.21.15.57:0/156162549 conn(0x55e99c388a40 msgr2=0x55e99c3977c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 -- 172.21.15.57:0/156162549 shutdown_connections 2024-04-13T05:24:12.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.644+0000 7f4eeda55740 1 -- 172.21.15.57:0/156162549 wait complete. 2024-04-13T05:24:12.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.645+0000 7f4eeda55740 1 -- start start 2024-04-13T05:24:12.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.645+0000 7f4eeda55740 1 Processor -- start 2024-04-13T05:24:12.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.646+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c39bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4eeda55740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4ee8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c39bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4ee8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c39bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60288/0 (socket says 172.21.15.57:60288) 2024-04-13T05:24:12.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4ee8c00640 1 -- 172.21.15.57:0/2759957311 learned_addr learned my addr 172.21.15.57:0/2759957311 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:12.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41b85a0 con 0x55e99c38d4d0 2024-04-13T05:24:12.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41b9a10 con 0x55e99c393030 2024-04-13T05:24:12.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41b8c70 con 0x55e99c38dec0 2024-04-13T05:24:12.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4eeda55740 10 client.0 _ll_register_callbacks cb 0x7f4ed4004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:12.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.647+0000 7f4ee3400640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.648+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ee8c00640 1 -- 172.21.15.57:0/2759957311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 msgr2=0x55e99c39c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ee8c00640 1 -- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 msgr2=0x55e99c39d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ee8c00640 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99c383030 con 0x55e99c38d4d0 2024-04-13T05:24:12.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ee8c00640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c39bfd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00058f0 tx=0x7f4ed0028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44ba7d6b4c3fd38 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:12.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.649+0000 7f4ecd600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:12.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.650+0000 7f4ecd600640 1 -- 172.21.15.57:0/2759957311 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0020050 con 0x55e99c38d4d0 2024-04-13T05:24:12.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.650+0000 7f4ecd600640 1 -- 172.21.15.57:0/2759957311 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4ed00268d0 con 0x55e99c38d4d0 2024-04-13T05:24:12.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.650+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed000de70 con 0x55e99c38d4d0 2024-04-13T05:24:12.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.651+0000 7f4ecd600640 1 -- 172.21.15.57:0/2759957311 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ed0016070 con 0x55e99c38d4d0 2024-04-13T05:24:12.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.651+0000 7f4eeda55740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-04-13T05:24:12.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.651+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e99c39ebe0 con 0x55e99c38d4d0 2024-04-13T05:24:12.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4ecd600640 1 -- 172.21.15.57:0/2759957311 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed000de70 con 0x55e99c38d4d0 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4ecd600640 1 -- 172.21.15.57:0/2759957311 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ed0004260 con 0x55e99c38d4d0 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4eeda55740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4eeda55740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4eeda55740 20 client.16824 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c409d60 0x55e99c40c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4eeda55740 4 client.16824 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e99c301a40 con 0x55e99c409d60 2024-04-13T05:24:12.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.652+0000 7f4ee3400640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c409d60 0x55e99c40c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:12.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.654+0000 7f4ee3400640 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c409d60 0x55e99c40c150 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3effd0831dcf1a9 server_cookie=a3d58811a05cfecd in_seq=0 out_seq=0 2024-04-13T05:24:12.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.655+0000 7f4ecd600640 10 client.16824 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.655+0000 7f4ecd600640 1 -- 172.21.15.57:0/2759957311 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55e99c301a40 con 0x55e99c409d60 2024-04-13T05:24:12.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.655+0000 7f4ecd600640 10 client.16824 handle_command_reply: tid=0 2024-04-13T05:24:12.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.655+0000 7f4eeda55740 1 client.16824 shutdown 2024-04-13T05:24:12.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 msgr2=0x55e99c39bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c39bfd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00058f0 tx=0x7f4ed0028040 comp rx=0 tx=0).stop 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 shutdown_connections 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e99c409d60 0x55e99c40c150 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e99c393030 0x55e99c39d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e99c38dec0 0x55e99c39c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 --2- 172.21.15.57:0/2759957311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e99c38d4d0 0x55e99c39bfd0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 >> 172.21.15.57:0/2759957311 conn(0x55e99c388a40 msgr2=0x55e99c38f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4ecd600640 0 client.16824 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:12.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.656+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 shutdown_connections 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.657+0000 7f4eeda55740 1 -- 172.21.15.57:0/2759957311 wait complete. 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.657+0000 7f4eeda55740 20 client.16824 trim_cache size 0 max 16384 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1756 inodes in the stack)", 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: "57cc08af-0b4b-452f-a310-93ed60acf2f1": { 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "57cc08af-0b4b-452f-a310-93ed60acf2f1", 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:24:12.660 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:24:12.661 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:24:12.661 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:24:12.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ec80a3bb0 msgr2=0x7f4ec80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0be280 tx=0x7f4edc0095c0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 msgr2=0x7f4ee41a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 0x7f4ee41a72f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4edc006010 tx=0x7f4edc003c20 comp rx=0 tx=0).stop 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 shutdown_connections 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ee41ac890 0x7f4ee41aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ee4110840 0x7f4ee41a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 --2- 172.21.15.57:0/3133766122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ee4109920 0x7f4ee41a72f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.658+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 >> 172.21.15.57:0/3133766122 conn(0x7f4ee406ee50 msgr2=0x7f4ee4071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.659+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 shutdown_connections 2024-04-13T05:24:12.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:12.659+0000 7f4eeac00640 1 -- 172.21.15.57:0/3133766122 wait complete. 2024-04-13T05:24:12.687 INFO:tasks.cephfs.filesystem:scrub status for tag:57cc08af-0b4b-452f-a310-93ed60acf2f1 - {'path': '/', 'tag': '57cc08af-0b4b-452f-a310-93ed60acf2f1', 'options': 'recursive,force'} 2024-04-13T05:24:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: pgmap v654: 129 pgs: 129 active+clean; 401 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-13T05:24:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:24:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:13.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:13.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:12 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: pgmap v654: 129 pgs: 129 active+clean; 401 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-13T05:24:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:24:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:13.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:13.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:13.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:13.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:12 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: pgmap v654: 129 pgs: 129 active+clean; 401 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:13.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:12 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:14.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:13 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:14.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:13 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:14.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:13 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:14.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:13 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:14.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:13 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:14.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:13 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:14 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:14 smithi083 ceph-mon[35133]: pgmap v655: 129 pgs: 129 active+clean; 405 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-04-13T05:24:15.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:14 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:15.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:14 smithi057 ceph-mon[32331]: pgmap v655: 129 pgs: 129 active+clean; 405 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-04-13T05:24:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:14 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:14 smithi189 ceph-mon[35531]: pgmap v655: 129 pgs: 129 active+clean; 405 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-04-13T05:24:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:16 smithi083 ceph-mon[35133]: pgmap v656: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:24:17.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:16 smithi057 ceph-mon[32331]: pgmap v656: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:24:17.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:16 smithi189 ceph-mon[35531]: pgmap v656: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-04-13T05:24:19.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:18 smithi083 ceph-mon[35133]: pgmap v657: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-04-13T05:24:19.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:18 smithi057 ceph-mon[32331]: pgmap v657: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-04-13T05:24:19.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:18 smithi189 ceph-mon[35531]: pgmap v657: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-04-13T05:24:19.638 INFO:tasks.workunit.client.0.smithi057.stdout: 55 7738 78 5251 72 2176 94 2024-04-13T05:24:21.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:20 smithi083 ceph-mon[35133]: pgmap v658: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-04-13T05:24:21.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:20 smithi057 ceph-mon[32331]: pgmap v658: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-04-13T05:24:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:20 smithi189 ceph-mon[35531]: pgmap v658: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-04-13T05:24:23.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:22 smithi083 ceph-mon[35133]: pgmap v659: 129 pgs: 129 active+clean; 418 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-04-13T05:24:23.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:22 smithi057 ceph-mon[32331]: pgmap v659: 129 pgs: 129 active+clean; 418 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-04-13T05:24:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:22 smithi189 ceph-mon[35531]: pgmap v659: 129 pgs: 129 active+clean; 418 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-04-13T05:24:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:24 smithi083 ceph-mon[35133]: pgmap v660: 129 pgs: 129 active+clean; 421 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-04-13T05:24:25.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:24 smithi057 ceph-mon[32331]: pgmap v660: 129 pgs: 129 active+clean; 421 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-04-13T05:24:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:24 smithi189 ceph-mon[35531]: pgmap v660: 129 pgs: 129 active+clean; 421 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-04-13T05:24:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:26 smithi083 ceph-mon[35133]: pgmap v661: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 286 op/s 2024-04-13T05:24:27.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:26 smithi057 ceph-mon[32331]: pgmap v661: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 286 op/s 2024-04-13T05:24:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:26 smithi189 ceph-mon[35531]: pgmap v661: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 286 op/s 2024-04-13T05:24:28.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:27 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:24:28.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:27 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:24:28.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:27 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:24:28.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:27 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:24:28.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:27 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:24:28.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:27 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:24:28.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:27 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:24:28.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:27 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:24:28.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:27 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:24:29.235 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:28 smithi189 ceph-mon[35531]: pgmap v662: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 248 op/s 2024-04-13T05:24:29.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:28 smithi083 ceph-mon[35133]: pgmap v662: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 248 op/s 2024-04-13T05:24:29.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:28 smithi057 ceph-mon[32331]: pgmap v662: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 248 op/s 2024-04-13T05:24:29.638 INFO:tasks.workunit.client.0.smithi057.stdout: 56 8111 114 5542 12 2486 111 2024-04-13T05:24:31.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:30 smithi083 ceph-mon[35133]: pgmap v663: 129 pgs: 129 active+clean; 433 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-04-13T05:24:31.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:30 smithi057 ceph-mon[32331]: pgmap v663: 129 pgs: 129 active+clean; 433 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-04-13T05:24:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:30 smithi189 ceph-mon[35531]: pgmap v663: 129 pgs: 129 active+clean; 433 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-04-13T05:24:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:32 smithi083 ceph-mon[35133]: pgmap v664: 129 pgs: 129 active+clean; 434 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 271 op/s 2024-04-13T05:24:33.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:32 smithi057 ceph-mon[32331]: pgmap v664: 129 pgs: 129 active+clean; 434 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 271 op/s 2024-04-13T05:24:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:32 smithi189 ceph-mon[35531]: pgmap v664: 129 pgs: 129 active+clean; 434 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 271 op/s 2024-04-13T05:24:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:34 smithi083 ceph-mon[35133]: pgmap v665: 129 pgs: 129 active+clean; 438 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 327 op/s 2024-04-13T05:24:35.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:34 smithi057 ceph-mon[32331]: pgmap v665: 129 pgs: 129 active+clean; 438 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 327 op/s 2024-04-13T05:24:35.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:34 smithi189 ceph-mon[35531]: pgmap v665: 129 pgs: 129 active+clean; 438 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 327 op/s 2024-04-13T05:24:37.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:36 smithi083 ceph-mon[35133]: pgmap v666: 129 pgs: 129 active+clean; 443 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 361 op/s 2024-04-13T05:24:37.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:36 smithi057 ceph-mon[32331]: pgmap v666: 129 pgs: 129 active+clean; 443 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 361 op/s 2024-04-13T05:24:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:36 smithi189 ceph-mon[35531]: pgmap v666: 129 pgs: 129 active+clean; 443 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 361 op/s 2024-04-13T05:24:39.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:38 smithi083 ceph-mon[35133]: pgmap v667: 129 pgs: 129 active+clean; 443 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-04-13T05:24:39.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:38 smithi057 ceph-mon[32331]: pgmap v667: 129 pgs: 129 active+clean; 443 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-04-13T05:24:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:38 smithi189 ceph-mon[35531]: pgmap v667: 129 pgs: 129 active+clean; 443 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 301 op/s 2024-04-13T05:24:39.638 INFO:tasks.workunit.client.0.smithi057.stdout: 56 8091 58 5527 35 2201 119 2024-04-13T05:24:41.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:40 smithi083 ceph-mon[35133]: pgmap v668: 129 pgs: 129 active+clean; 422 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-04-13T05:24:41.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:40 smithi057 ceph-mon[32331]: pgmap v668: 129 pgs: 129 active+clean; 422 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-04-13T05:24:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:40 smithi189 ceph-mon[35531]: pgmap v668: 129 pgs: 129 active+clean; 422 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-04-13T05:24:42.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:24:42.688 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:24:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 -- 172.21.15.57:0/2197490075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18105cb0 msgr2=0x7f0b1810c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 --2- 172.21.15.57:0/2197490075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18105cb0 0x7f0b1810c540 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009860 tx=0x7f0b1002bb70 comp rx=0 tx=0).stop 2024-04-13T05:24:42.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 -- 172.21.15.57:0/2197490075 shutdown_connections 2024-04-13T05:24:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 --2- 172.21.15.57:0/2197490075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 0x7f0b18105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 --2- 172.21.15.57:0/2197490075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18105cb0 0x7f0b1810c540 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 --2- 172.21.15.57:0/2197490075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18104100 0x7f0b18104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.946+0000 7f0b17200640 1 -- 172.21.15.57:0/2197490075 >> 172.21.15.57:0/2197490075 conn(0x7f0b180ff670 msgr2=0x7f0b18101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.947+0000 7f0b17200640 1 -- 172.21.15.57:0/2197490075 shutdown_connections 2024-04-13T05:24:42.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.947+0000 7f0b17200640 1 -- 172.21.15.57:0/2197490075 wait complete. 2024-04-13T05:24:42.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.948+0000 7f0b17200640 1 Processor -- start 2024-04-13T05:24:42.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.948+0000 7f0b17200640 1 -- start start 2024-04-13T05:24:42.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.949+0000 7f0b17200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 0x7f0b1819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.949+0000 7f0b17200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 0x7f0b1819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.949+0000 7f0b17200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18105cb0 0x7f0b181a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.949+0000 7f0b17200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1810fc40 con 0x7f0b18104100 2024-04-13T05:24:42.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.949+0000 7f0b17200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1810fac0 con 0x7f0b181052f0 2024-04-13T05:24:42.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.949+0000 7f0b17200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1810fdc0 con 0x7f0b18105cb0 2024-04-13T05:24:42.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.950+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 0x7f0b1819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.950+0000 7f0b16800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18105cb0 0x7f0b181a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.950+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 0x7f0b1819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53566/0 (socket says 172.21.15.57:53566) 2024-04-13T05:24:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.950+0000 7f0b15e00640 1 -- 172.21.15.57:0/3798984238 learned_addr learned my addr 172.21.15.57:0/3798984238 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.950+0000 7f0b16800640 1 -- 172.21.15.57:0/3798984238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 msgr2=0x7f0b1819a820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-13T05:24:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b0d400640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 0x7f0b1819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b16800640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 0x7f0b1819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b16800640 1 -- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 msgr2=0x7f0b1819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b16800640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 0x7f0b1819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b16800640 1 -- 172.21.15.57:0/3798984238 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b181a1f00 con 0x7f0b18105cb0 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b15e00640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 0x7f0b1819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.951+0000 7f0b0d400640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 0x7f0b1819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.952+0000 7f0b16800640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18105cb0 0x7f0b181a1890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009780 tx=0x7f0b10004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.953+0000 7f0b0f400640 1 -- 172.21.15.57:0/3798984238 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10041020 con 0x7f0b18105cb0 2024-04-13T05:24:42.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.953+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181a2190 con 0x7f0b18105cb0 2024-04-13T05:24:42.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.953+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b181a26d0 con 0x7f0b18105cb0 2024-04-13T05:24:42.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.953+0000 7f0b0f400640 1 -- 172.21.15.57:0/3798984238 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b1002c9c0 con 0x7f0b18105cb0 2024-04-13T05:24:42.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.953+0000 7f0b0f400640 1 -- 172.21.15.57:0/3798984238 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10041020 con 0x7f0b18105cb0 2024-04-13T05:24:42.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.958+0000 7f0b1fc27740 1 Processor -- start 2024-04-13T05:24:42.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b0f400640 1 -- 172.21.15.57:0/3798984238 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0b10044020 con 0x7f0b18105cb0 2024-04-13T05:24:42.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 -- start start 2024-04-13T05:24:42.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbeeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddb55020 con 0x55a0ddbee6f0 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddafb6d0 con 0x55a0ddbf4250 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.960+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddb50fe0 con 0x55a0ddbef0e0 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.961+0000 7f0b0d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbeeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.961+0000 7f0b0d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbeeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53580/0 (socket says 172.21.15.57:53580) 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.961+0000 7f0b0d400640 1 -- 172.21.15.57:0/1553442638 learned_addr learned my addr 172.21.15.57:0/1553442638 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.961+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.961+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0d400640 1 -- 172.21.15.57:0/1553442638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 msgr2=0x55a0ddbf3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0d400640 1 -- 172.21.15.57:0/1553442638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 msgr2=0x55a0ddbf66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0d400640 1 -- 172.21.15.57:0/1553442638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0dd9317d0 con 0x55a0ddbee6f0 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0f400640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0afc0a3e70 0x7f0afc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b0f400640 1 -- 172.21.15.57:0/3798984238 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0b100f1b50 con 0x7f0b18105cb0 2024-04-13T05:24:42.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.962+0000 7f0b15e00640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0afc0a3e70 0x7f0afc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.963+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbeeb10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0b000138f0 tx=0x7f0b00009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57110ac6e375e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:42.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.963+0000 7f0b07000640 1 -- 172.21.15.57:0/1553442638 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00019030 con 0x55a0ddbee6f0 2024-04-13T05:24:42.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.963+0000 7f0b07000640 1 -- 172.21.15.57:0/1553442638 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b00011110 con 0x55a0ddbee6f0 2024-04-13T05:24:42.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.963+0000 7f0b07000640 1 -- 172.21.15.57:0/1553442638 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00011dd0 con 0x55a0ddbee6f0 2024-04-13T05:24:42.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1553442638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 msgr2=0x55a0ddbeeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:42.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbeeb10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0b000138f0 tx=0x7f0b00009960 comp rx=0 tx=0).stop 2024-04-13T05:24:42.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b15e00640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0afc0a3e70 0x7f0afc0a6330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009fd0 tx=0x7f0b08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:42.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1553442638 shutdown_connections 2024-04-13T05:24:42.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1553442638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbeeb10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1553442638 >> 172.21.15.57:0/1553442638 conn(0x55a0ddbe9c60 msgr2=0x55a0ddbea060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1553442638 shutdown_connections 2024-04-13T05:24:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.964+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1553442638 wait complete. 2024-04-13T05:24:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.965+0000 7f0b1fc27740 1 -- start start 2024-04-13T05:24:42.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.965+0000 7f0b1fc27740 1 Processor -- start 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbfffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddbf7250 con 0x55a0ddbee6f0 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddb0ea90 con 0x55a0ddbf4250 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddbf73d0 con 0x55a0ddbef0e0 2024-04-13T05:24:42.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.966+0000 7f0b1fc27740 10 client.0 _ll_register_callbacks cb 0x55a0ddb0e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53588/0 (socket says 172.21.15.57:53588) 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 -- 172.21.15.57:0/3095466998 learned_addr learned my addr 172.21.15.57:0/3095466998 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b0d400640 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 -- 172.21.15.57:0/3095466998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 msgr2=0x55a0ddbf8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 -- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 msgr2=0x55a0ddbfffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:42.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbfffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.967+0000 7f0b15e00640 1 -- 172.21.15.57:0/3095466998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0dd8673f0 con 0x55a0ddbee6f0 2024-04-13T05:24:42.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.968+0000 7f0b15e00640 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf8a30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0b080058c0 tx=0x7f0b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f6d080460b75d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.968+0000 7f0afb400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.968+0000 7f0afb400640 1 -- 172.21.15.57:0/3095466998 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800f950 con 0x55a0ddbee6f0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.968+0000 7f0afb400640 1 -- 172.21.15.57:0/3095466998 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0b0800faf0 con 0x55a0ddbee6f0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.969+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b00011110 con 0x55a0ddbee6f0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.969+0000 7f0afb400640 1 -- 172.21.15.57:0/3095466998 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b0800fcf0 con 0x55a0ddbee6f0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.969+0000 7f0b1fc27740 10 client.16836 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.969+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0dd8270f0 con 0x55a0ddbee6f0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.969+0000 7f0afb400640 1 -- 172.21.15.57:0/3095466998 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b00011110 con 0x55a0ddbee6f0 2024-04-13T05:24:42.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0afb400640 1 -- 172.21.15.57:0/3095466998 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0b08027070 con 0x55a0ddbee6f0 2024-04-13T05:24:42.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0b1fc27740 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:42.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0b1fc27740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:42.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0b1fc27740 20 client.16836 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:42.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc0d250 0x55a0ddc0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:42.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0b1fc27740 4 client.16836 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:42.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.970+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0ddaf4600 con 0x55a0ddc0d250 2024-04-13T05:24:42.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.973+0000 7f0b0d400640 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc0d250 0x55a0ddc0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:42.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.975+0000 7f0b0d400640 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc0d250 0x55a0ddc0f710 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5950c3ba297bb1 server_cookie=2a988ace5f207a2f in_seq=0 out_seq=0 2024-04-13T05:24:42.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.975+0000 7f0afb400640 10 client.16836 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:42.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.976+0000 7f0afb400640 1 -- 172.21.15.57:0/3095466998 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0ddaf4600 con 0x55a0ddc0d250 2024-04-13T05:24:42.978 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.976+0000 7f0afb400640 10 client.16836 handle_command_reply: tid=0 2024-04-13T05:24:42.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.976+0000 7f0b1fc27740 1 client.16836 shutdown 2024-04-13T05:24:42.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.976+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 msgr2=0x55a0ddbf8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.976+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf8a30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0b080058c0 tx=0x7f0b08004730 comp rx=0 tx=0).stop 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 shutdown_connections 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc0d250 0x55a0ddc0f710 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbf4250 0x55a0ddbfffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbf8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0afb400640 0 client.16836 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/3095466998 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf8a30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 >> 172.21.15.57:0/3095466998 conn(0x55a0ddbe9c60 msgr2=0x55a0ddbe8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 shutdown_connections 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 1 -- 172.21.15.57:0/3095466998 wait complete. 2024-04-13T05:24:42.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:42.977+0000 7f0b1fc27740 20 client.16836 trim_cache size 0 max 16384 2024-04-13T05:24:43.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.010+0000 7f0b1fc27740 1 Processor -- start 2024-04-13T05:24:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.010+0000 7f0b1fc27740 1 -- start start 2024-04-13T05:24:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40125b0 con 0x55a0ddbf4250 2024-04-13T05:24:43.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400fe20 con 0x55a0ddbee6f0 2024-04-13T05:24:43.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400dc70 con 0x55a0ddbef0e0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b16800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42500/0 (socket says 172.21.15.57:42500) 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.011+0000 7f0b15e00640 1 -- 172.21.15.57:0/1791538972 learned_addr learned my addr 172.21.15.57:0/1791538972 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b15e00640 1 -- 172.21.15.57:0/1791538972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 msgr2=0x55a0ddbfff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b15e00640 1 -- 172.21.15.57:0/1791538972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbf4250 msgr2=0x55a0ddbf8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b15e00640 1 -- 172.21.15.57:0/1791538972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ddbf0c90 con 0x55a0ddbee6f0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b16800640 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.012+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf39a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0b0801f720 tx=0x7f0b08005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=792c3f962b56298b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.013+0000 7f0b06600640 1 -- 172.21.15.57:0/1791538972 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800fb50 con 0x55a0ddbee6f0 2024-04-13T05:24:43.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.013+0000 7f0b06600640 1 -- 172.21.15.57:0/1791538972 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b08005030 con 0x55a0ddbee6f0 2024-04-13T05:24:43.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b06600640 1 -- 172.21.15.57:0/1791538972 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0801fba0 con 0x55a0ddbee6f0 2024-04-13T05:24:43.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1791538972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 msgr2=0x55a0ddbf39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf39a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0b0801f720 tx=0x7f0b08005df0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1791538972 shutdown_connections 2024-04-13T05:24:43.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbf4250 0x55a0ddbf8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1791538972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbf39a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1791538972 >> 172.21.15.57:0/1791538972 conn(0x55a0ddbe9c60 msgr2=0x55a0ddbea040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1791538972 shutdown_connections 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.014+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1791538972 wait complete. 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 -- start start 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 Processor -- start 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbfaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbf4250 0x55a0ddc025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400e2e0 con 0x55a0ddbef0e0 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400f750 con 0x55a0ddbee6f0 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400e9b0 con 0x55a0ddbf4250 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.015+0000 7f0b1fc27740 10 client.0 _ll_register_callbacks cb 0x7f0b08004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b0d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbfaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbfaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42516/0 (socket says 172.21.15.57:42516) 2024-04-13T05:24:43.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 -- 172.21.15.57:0/1876431179 learned_addr learned my addr 172.21.15.57:0/1876431179 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b16800640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbf4250 0x55a0ddc025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 -- 172.21.15.57:0/1876431179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbf4250 msgr2=0x55a0ddc025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbf4250 0x55a0ddc025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 -- 172.21.15.57:0/1876431179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbef0e0 msgr2=0x55a0ddbfb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b15e00640 1 -- 172.21.15.57:0/1876431179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ddc02b20 con 0x55a0ddbee6f0 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.016+0000 7f0b16800640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbf4250 0x55a0ddc025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:43.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.017+0000 7f0b15e00640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbfaef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b0801f650 tx=0x7f0b080123b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4430823c88d3a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.017+0000 7f0afb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.017+0000 7f0afb400640 1 -- 172.21.15.57:0/1876431179 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080053f0 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.017+0000 7f0afb400640 1 -- 172.21.15.57:0/1876431179 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0b08005590 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.018+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b08005030 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.018+0000 7f0afb400640 1 -- 172.21.15.57:0/1876431179 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b0801f8a0 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.018+0000 7f0b1fc27740 10 client.36245 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.018+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0ddc03e10 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.018+0000 7f0afb400640 1 -- 172.21.15.57:0/1876431179 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b08005030 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0afb400640 1 -- 172.21.15.57:0/1876431179 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0b08027040 con 0x55a0ddbee6f0 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b1fc27740 10 client.36245 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b1fc27740 10 client.36245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b1fc27740 20 client.36245 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc045e0 0x55a0ddc06a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b1fc27740 4 client.36245 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:24:43.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0ddb62a40 con 0x55a0ddc045e0 2024-04-13T05:24:43.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.019+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc045e0 0x55a0ddc06a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.020+0000 7f0b0d400640 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc045e0 0x55a0ddc06a00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afcdd04d231e031 server_cookie=c3151b94b13a4f05 in_seq=0 out_seq=0 2024-04-13T05:24:43.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.020+0000 7f0afb400640 10 client.36245 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0afb400640 1 -- 172.21.15.57:0/1876431179 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0ddb62a40 con 0x55a0ddc045e0 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0afb400640 10 client.36245 handle_command_reply: tid=0 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 client.36245 shutdown 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 msgr2=0x55a0ddbfaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbfaef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b0801f650 tx=0x7f0b080123b0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 shutdown_connections 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a0ddc045e0 0x55a0ddc06a00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddbf4250 0x55a0ddc025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a0ddbef0e0 0x55a0ddbfb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 --2- 172.21.15.57:0/1876431179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a0ddbee6f0 0x55a0ddbfaef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 >> 172.21.15.57:0/1876431179 conn(0x55a0ddbe9c60 msgr2=0x55a0ddbf3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.022+0000 7f0afb400640 0 client.36245 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.023+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 shutdown_connections 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.023+0000 7f0b1fc27740 1 -- 172.21.15.57:0/1876431179 wait complete. 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.023+0000 7f0b1fc27740 20 client.36245 trim_cache size 0 max 16384 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:24:43.026 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:24:43.027 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:24:43.027 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:24:43.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0afc0a3e70 msgr2=0x7f0afc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0afc0a3e70 0x7f0afc0a6330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009fd0 tx=0x7f0b08009260 comp rx=0 tx=0).stop 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18105cb0 msgr2=0x7f0b181a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18105cb0 0x7f0b181a1890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009780 tx=0x7f0b10004270 comp rx=0 tx=0).stop 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 shutdown_connections 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0afc0a3e70 0x7f0afc0a6330 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b181052f0 0x7f0b1819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b18105cb0 0x7f0b181a1890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 --2- 172.21.15.57:0/3798984238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b18104100 0x7f0b1819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 >> 172.21.15.57:0/3798984238 conn(0x7f0b180ff670 msgr2=0x7f0b18108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.024+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 shutdown_connections 2024-04-13T05:24:43.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.025+0000 7f0b17200640 1 -- 172.21.15.57:0/3798984238 wait complete. 2024-04-13T05:24:43.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:42 smithi057 ceph-mon[32331]: pgmap v669: 129 pgs: 129 active+clean; 426 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 303 op/s 2024-04-13T05:24:43.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:42 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:24:43.048 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:24:43.048 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:24:43.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:42 smithi083 ceph-mon[35133]: pgmap v669: 129 pgs: 129 active+clean; 426 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 303 op/s 2024-04-13T05:24:43.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:42 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:24:43.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.313+0000 7ff2dda00640 1 -- 172.21.15.57:0/3382199032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8104060 msgr2=0x7ff2d8104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.313+0000 7ff2dda00640 1 --2- 172.21.15.57:0/3382199032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8104060 0x7ff2d8104480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009900 tx=0x7ff2c002bc30 comp rx=0 tx=0).stop 2024-04-13T05:24:43.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 -- 172.21.15.57:0/3382199032 shutdown_connections 2024-04-13T05:24:43.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 --2- 172.21.15.57:0/3382199032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2d8105c10 0x7ff2d810c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 --2- 172.21.15.57:0/3382199032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2d8105250 0x7ff2d81056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 --2- 172.21.15.57:0/3382199032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8104060 0x7ff2d8104480 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 -- 172.21.15.57:0/3382199032 >> 172.21.15.57:0/3382199032 conn(0x7ff2d80ff5d0 msgr2=0x7ff2d8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:43.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 -- 172.21.15.57:0/3382199032 shutdown_connections 2024-04-13T05:24:43.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.314+0000 7ff2dda00640 1 -- 172.21.15.57:0/3382199032 wait complete. 2024-04-13T05:24:43.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.315+0000 7ff2dda00640 1 Processor -- start 2024-04-13T05:24:43.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.316+0000 7ff2dda00640 1 -- start start 2024-04-13T05:24:43.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dda00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2d8105250 0x7ff2d8079cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dda00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 0x7ff2d807a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2d80767e0 0x7ff2d8076c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dda00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d810fbe0 con 0x7ff2d8105c10 2024-04-13T05:24:43.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dda00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d810fa60 con 0x7ff2d80767e0 2024-04-13T05:24:43.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dda00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d810fd60 con 0x7ff2d8105250 2024-04-13T05:24:43.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 0x7ff2d807a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2d80767e0 0x7ff2d8076c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2d7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 0x7ff2d807a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53630/0 (socket says 172.21.15.57:53630) 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2d7400640 1 -- 172.21.15.57:0/1602100701 learned_addr learned my addr 172.21.15.57:0/1602100701 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.317+0000 7ff2d7e00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2d8105250 0x7ff2d8079cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.318+0000 7ff2d7400640 1 -- 172.21.15.57:0/1602100701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2d8105250 msgr2=0x7ff2d8079cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.318+0000 7ff2d7400640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2d8105250 0x7ff2d8079cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.318+0000 7ff2d7400640 1 -- 172.21.15.57:0/1602100701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2d80767e0 msgr2=0x7ff2d8076c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.318+0000 7ff2d7400640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2d80767e0 0x7ff2d8076c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.318+0000 7ff2d7400640 1 -- 172.21.15.57:0/1602100701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d81a6a10 con 0x7ff2d8105c10 2024-04-13T05:24:43.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.319+0000 7ff2d7400640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 0x7ff2d807a210 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b4a0 tx=0x7ff2c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:43.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.319+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8014070 con 0x7ff2d8105c10 2024-04-13T05:24:43.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.319+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2c8010ac0 con 0x7ff2d8105c10 2024-04-13T05:24:43.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.319+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8017c30 con 0x7ff2d8105c10 2024-04-13T05:24:43.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.319+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d81a6c40 con 0x7ff2d8105c10 2024-04-13T05:24:43.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.320+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81a7140 con 0x7ff2d8105c10 2024-04-13T05:24:43.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.321+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d810b590 con 0x7ff2d8105c10 2024-04-13T05:24:43.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.324+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff2c8017070 con 0x7ff2d8105c10 2024-04-13T05:24:43.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.326+0000 7ff2d4c00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2bc0a3cb0 0x7ff2bc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:43.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.326+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff2c80cf790 con 0x7ff2d8105c10 2024-04-13T05:24:43.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.327+0000 7ff2d7e00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2bc0a3cb0 0x7ff2bc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:43.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.328+0000 7ff2d7e00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2bc0a3cb0 0x7ff2bc0a6170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0036040 tx=0x7ff2c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:43.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:43 smithi189 ceph-mon[35531]: pgmap v669: 129 pgs: 129 active+clean; 426 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 303 op/s 2024-04-13T05:24:43.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:43 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:24:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.332+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2c809bd70 con 0x7ff2d8105c10 2024-04-13T05:24:43.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.773+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2d8062c60 con 0x7ff2d8105c10 2024-04-13T05:24:43.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.774+0000 7ff2d4c00640 1 -- 172.21.15.57:0/1602100701 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11859 (secure 0 0 0) 0x7ff2c809b670 con 0x7ff2d8105c10 2024-04-13T05:24:43.777 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:24:43.778 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:24:41.942015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:24:43.778 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 30 2024-04-13T05:24:43.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.779+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2bc0a3cb0 msgr2=0x7ff2bc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.779+0000 7ff2dda00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2bc0a3cb0 0x7ff2bc0a6170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0036040 tx=0x7ff2c0000bc0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 msgr2=0x7ff2d807a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:43.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 0x7ff2d807a210 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b4a0 tx=0x7ff2c800b960 comp rx=0 tx=0).stop 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 shutdown_connections 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2bc0a3cb0 0x7ff2bc0a6170 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2d80767e0 0x7ff2d8076c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2d8105c10 0x7ff2d807a210 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 --2- 172.21.15.57:0/1602100701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2d8105250 0x7ff2d8079cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 >> 172.21.15.57:0/1602100701 conn(0x7ff2d80ff5d0 msgr2=0x7ff2d8108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 shutdown_connections 2024-04-13T05:24:43.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:43.780+0000 7ff2dda00640 1 -- 172.21.15.57:0/1602100701 wait complete. 2024-04-13T05:24:43.804 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:24:44.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 -- 172.21.15.57:0/2734260304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4077bc0 msgr2=0x7fcad4078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 --2- 172.21.15.57:0/2734260304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4077bc0 0x7fcad4078020 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac002bc30 comp rx=0 tx=0).stop 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 -- 172.21.15.57:0/2734260304 shutdown_connections 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 --2- 172.21.15.57:0/2734260304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcad4078560 0x7fcad4078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 --2- 172.21.15.57:0/2734260304 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4077bc0 0x7fcad4078020 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 --2- 172.21.15.57:0/2734260304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4076980 0x7fcad4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.060+0000 7fcadb000640 1 -- 172.21.15.57:0/2734260304 >> 172.21.15.57:0/2734260304 conn(0x7fcad40ff7b0 msgr2=0x7fcad4101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.061+0000 7fcadb000640 1 -- 172.21.15.57:0/2734260304 shutdown_connections 2024-04-13T05:24:44.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.061+0000 7fcadb000640 1 -- 172.21.15.57:0/2734260304 wait complete. 2024-04-13T05:24:44.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.062+0000 7fcadb000640 1 Processor -- start 2024-04-13T05:24:44.065 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.062+0000 7fcadb000640 1 -- start start 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.062+0000 7fcadb000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 0x7fcad419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcadb000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4077bc0 0x7fcad419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcadb000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcad4078560 0x7fcad41a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcadb000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4114180 con 0x7fcad4076980 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcadb000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4114000 con 0x7fcad4077bc0 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcadb000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4114300 con 0x7fcad4078560 2024-04-13T05:24:44.066 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 0x7fcad419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 0x7fcad419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53660/0 (socket says 172.21.15.57:53660) 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcad3e00640 1 -- 172.21.15.57:0/1888334664 learned_addr learned my addr 172.21.15.57:0/1888334664 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcad3400640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4077bc0 0x7fcad419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcad9000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcad4078560 0x7fcad41a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.063+0000 7fcad3e00640 1 -- 172.21.15.57:0/1888334664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcad4078560 msgr2=0x7fcad41a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad3e00640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcad4078560 0x7fcad41a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad3e00640 1 -- 172.21.15.57:0/1888334664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4077bc0 msgr2=0x7fcad419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad3e00640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4077bc0 0x7fcad419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad3e00640 1 -- 172.21.15.57:0/1888334664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41a6470 con 0x7fcad4076980 2024-04-13T05:24:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad3400640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4077bc0 0x7fcad419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad3e00640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 0x7fcad419e850 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcac400d950 tx=0x7fcac400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad0c00640 1 -- 172.21.15.57:0/1888334664 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4017030 con 0x7fcad4076980 2024-04-13T05:24:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad0c00640 1 -- 172.21.15.57:0/1888334664 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac4004020 con 0x7fcad4076980 2024-04-13T05:24:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcad0c00640 1 -- 172.21.15.57:0/1888334664 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40047a0 con 0x7fcad4076980 2024-04-13T05:24:44.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.064+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a6760 con 0x7fcad4076980 2024-04-13T05:24:44.069 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.065+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a6c30 con 0x7fcad4076980 2024-04-13T05:24:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.068+0000 7fcad0c00640 1 -- 172.21.15.57:0/1888334664 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcac4009a30 con 0x7fcad4076980 2024-04-13T05:24:44.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.069+0000 7fcaddf80740 1 Processor -- start 2024-04-13T05:24:44.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 -- start start 2024-04-13T05:24:44.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab40670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab48240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55f36aaa6020 con 0x55f36ab45db0 2024-04-13T05:24:44.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f36aa4c950 con 0x55f36ab40c40 2024-04-13T05:24:44.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcaddf80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f36aaa1fe0 con 0x55f36ab40250 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.070+0000 7fcad3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab48240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab48240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53666/0 (socket says 172.21.15.57:53666) 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3400640 1 -- 172.21.15.57:0/899755451 learned_addr learned my addr 172.21.15.57:0/899755451 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad9000640 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab40670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3e00640 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad0c00640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcab80a3c30 0x7fcab80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad0c00640 1 -- 172.21.15.57:0/1888334664 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcac40d00a0 con 0x7fcad4076980 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3e00640 1 -- 172.21.15.57:0/899755451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 msgr2=0x55f36ab40670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3e00640 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab40670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3e00640 1 -- 172.21.15.57:0/899755451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 msgr2=0x55f36ab48240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3e00640 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab48240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.071+0000 7fcad3e00640 1 -- 172.21.15.57:0/899755451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36a7b83f0 con 0x55f36ab40c40 2024-04-13T05:24:44.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.072+0000 7fcad9000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcab80a3c30 0x7fcab80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.075 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.072+0000 7fcad3400640 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab48240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.072+0000 7fcad3e00640 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab45740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcac40cd740 tx=0x7fcac4005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796011fcbefb6ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.072+0000 7fcac9600640 1 -- 172.21.15.57:0/899755451 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40056a0 con 0x55f36ab40c40 2024-04-13T05:24:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.073+0000 7fcac9600640 1 -- 172.21.15.57:0/899755451 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac40041d0 con 0x55f36ab40c40 2024-04-13T05:24:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.073+0000 7fcac9600640 1 -- 172.21.15.57:0/899755451 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac409a310 con 0x55f36ab40c40 2024-04-13T05:24:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 -- 172.21.15.57:0/899755451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 msgr2=0x55f36ab45740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.076 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab45740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcac40cd740 tx=0x7fcac4005020 comp rx=0 tx=0).stop 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcad9000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcab80a3c30 0x7fcab80a60f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f36ab414a0 tx=0x7fcacc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 -- 172.21.15.57:0/899755451 shutdown_connections 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab45740 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab48240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 --2- 172.21.15.57:0/899755451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab40670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 -- 172.21.15.57:0/899755451 >> 172.21.15.57:0/899755451 conn(0x55f36ab3b7c0 msgr2=0x55f36ab3bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 -- 172.21.15.57:0/899755451 shutdown_connections 2024-04-13T05:24:44.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 -- 172.21.15.57:0/899755451 wait complete. 2024-04-13T05:24:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.074+0000 7fcaddf80740 1 -- start start 2024-04-13T05:24:44.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 Processor -- start 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab4a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab4ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab51cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55f36ab48ca0 con 0x55f36ab45db0 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f36aa5fc80 con 0x55f36ab40c40 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.075+0000 7fcaddf80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f36ab48e20 con 0x55f36ab40250 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcaddf80740 10 client.0 _ll_register_callbacks cb 0x55f36aa5f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab4ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab4a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab4ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42554/0 (socket says 172.21.15.57:42554) 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 -- 172.21.15.57:0/1490325974 learned_addr learned my addr 172.21.15.57:0/1490325974 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad9000640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab51cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 -- 172.21.15.57:0/1490325974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 msgr2=0x55f36ab4a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab4a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 -- 172.21.15.57:0/1490325974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 msgr2=0x55f36ab51cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab51cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.076+0000 7fcad3400640 1 -- 172.21.15.57:0/1490325974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36a8827d0 con 0x55f36ab40c40 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.077+0000 7fcad9000640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab51cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.077+0000 7fcad3400640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab4ac70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcac0036040 tx=0x7fcac0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1e7d2e90e0b110 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.077+0000 7fcabe000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.077+0000 7fcabe000640 1 -- 172.21.15.57:0/1490325974 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0045070 con 0x55f36ab40c40 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.077+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f36aa45750 con 0x55f36ab40c40 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.077+0000 7fcabe000640 1 -- 172.21.15.57:0/1490325974 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcac0005450 con 0x55f36ab40c40 2024-04-13T05:24:44.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.078+0000 7fcabe000640 1 -- 172.21.15.57:0/1490325974 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcac0051050 con 0x55f36ab40c40 2024-04-13T05:24:44.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.078+0000 7fcaddf80740 10 client.36263 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:44.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.078+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f36ab550e0 con 0x55f36ab40c40 2024-04-13T05:24:44.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.078+0000 7fcabe000640 1 -- 172.21.15.57:0/1490325974 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f36aa45750 con 0x55f36ab40c40 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcabe000640 1 -- 172.21.15.57:0/1490325974 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcac0003780 con 0x55f36ab40c40 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcaddf80740 10 client.36263 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcaddf80740 10 client.36263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcaddf80740 20 client.36263 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcaddf80740 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36ab56650 0x55f36ab58ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcaddf80740 4 client.36263 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.079+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f36aab3a40 con 0x55f36ab56650 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.080+0000 7fcad3e00640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36ab56650 0x55f36ab58ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.080+0000 7fcad3e00640 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36ab56650 0x55f36ab58ad0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9af2267d33b3c342 server_cookie=41032dbd6b5438ea in_seq=0 out_seq=0 2024-04-13T05:24:44.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.080+0000 7fcabe000640 10 client.36263 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:44.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.081+0000 7fcabe000640 1 -- 172.21.15.57:0/1490325974 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f36aab3a40 con 0x55f36ab56650 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.081+0000 7fcabe000640 10 client.36263 handle_command_reply: tid=0 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 client.36263 shutdown 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 msgr2=0x55f36ab4ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab4ac70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcac0036040 tx=0x7fcac0004270 comp rx=0 tx=0).stop 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 shutdown_connections 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36ab56650 0x55f36ab58ad0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40c40 0x55f36ab4ac70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab45db0 0x55f36ab51cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 --2- 172.21.15.57:0/1490325974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40250 0x55f36ab4a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 >> 172.21.15.57:0/1490325974 conn(0x55f36ab3b7c0 msgr2=0x55f36ab3aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcabe000640 0 client.36263 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 shutdown_connections 2024-04-13T05:24:44.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.082+0000 7fcaddf80740 1 -- 172.21.15.57:0/1490325974 wait complete. 2024-04-13T05:24:44.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.083+0000 7fcaddf80740 20 client.36263 trim_cache size 0 max 16384 2024-04-13T05:24:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 Processor -- start 2024-04-13T05:24:44.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 -- start start 2024-04-13T05:24:44.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 0x55f36ab491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40c40 0x55f36ab51c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab45db0 0x55f36ab3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b3b20 con 0x55f36ab40250 2024-04-13T05:24:44.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b1390 con 0x55f36ab45db0 2024-04-13T05:24:44.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.107+0000 7fcaddf80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41af1e0 con 0x55f36ab40c40 2024-04-13T05:24:44.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 0x55f36ab491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 0x55f36ab491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53682/0 (socket says 172.21.15.57:53682) 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 -- 172.21.15.57:0/2633383566 learned_addr learned my addr 172.21.15.57:0/2633383566 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3400640 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40c40 0x55f36ab51c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 -- 172.21.15.57:0/2633383566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40c40 msgr2=0x55f36ab51c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40c40 0x55f36ab51c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 -- 172.21.15.57:0/2633383566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab45db0 msgr2=0x55f36ab3a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab45db0 0x55f36ab3a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.108+0000 7fcad3e00640 1 -- 172.21.15.57:0/2633383566 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36ab414d0 con 0x55f36ab40250 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.109+0000 7fcad3e00640 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 0x55f36ab491c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005fd0 tx=0x7fcac40d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b031b0295b84ea6b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.109+0000 7fcac8c00640 1 -- 172.21.15.57:0/2633383566 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4098220 con 0x55f36ab40250 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.109+0000 7fcac8c00640 1 -- 172.21.15.57:0/2633383566 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac409c680 con 0x55f36ab40250 2024-04-13T05:24:44.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.109+0000 7fcac8c00640 1 -- 172.21.15.57:0/2633383566 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40cd980 con 0x55f36ab40250 2024-04-13T05:24:44.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 -- 172.21.15.57:0/2633383566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 msgr2=0x55f36ab491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 0x55f36ab491c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005fd0 tx=0x7fcac40d2040 comp rx=0 tx=0).stop 2024-04-13T05:24:44.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 -- 172.21.15.57:0/2633383566 shutdown_connections 2024-04-13T05:24:44.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab45db0 0x55f36ab3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab40c40 0x55f36ab51c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 --2- 172.21.15.57:0/2633383566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40250 0x55f36ab491c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.110+0000 7fcaddf80740 1 -- 172.21.15.57:0/2633383566 >> 172.21.15.57:0/2633383566 conn(0x55f36ab3b7c0 msgr2=0x55f36ab416e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.111+0000 7fcaddf80740 1 -- 172.21.15.57:0/2633383566 shutdown_connections 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.111+0000 7fcaddf80740 1 -- 172.21.15.57:0/2633383566 wait complete. 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.111+0000 7fcaddf80740 1 -- start start 2024-04-13T05:24:44.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.112+0000 7fcaddf80740 1 Processor -- start 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40250 0x55f36ab4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 0x55f36ab4f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab45db0 0x55f36ab501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41af850 con 0x55f36ab40c40 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b0cc0 con 0x55f36ab40250 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41aff20 con 0x55f36ab45db0 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcaddf80740 10 client.0 _ll_register_callbacks cb 0x7fcac0005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcad3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 0x55f36ab4f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcad3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 0x55f36ab4f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53698/0 (socket says 172.21.15.57:53698) 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcad3400640 1 -- 172.21.15.57:0/680545330 learned_addr learned my addr 172.21.15.57:0/680545330 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.113+0000 7fcad3e00640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40250 0x55f36ab4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad9000640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab45db0 0x55f36ab501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad3400640 1 -- 172.21.15.57:0/680545330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab45db0 msgr2=0x55f36ab501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad3400640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab45db0 0x55f36ab501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad3400640 1 -- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40250 msgr2=0x55f36ab4ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad3400640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40250 0x55f36ab4ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad3400640 1 -- 172.21.15.57:0/680545330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36ab507c0 con 0x55f36ab40c40 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcad3400640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 0x55f36ab4f230 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907f02bac34072d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.114+0000 7fcabe000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.115+0000 7fcabe000640 1 -- 172.21.15.57:0/680545330 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0045040 con 0x55f36ab40c40 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.115+0000 7fcabe000640 1 -- 172.21.15.57:0/680545330 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fcac0005250 con 0x55f36ab40c40 2024-04-13T05:24:44.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.115+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac409c680 con 0x55f36ab40c40 2024-04-13T05:24:44.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.115+0000 7fcabe000640 1 -- 172.21.15.57:0/680545330 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcac0000e60 con 0x55f36ab40c40 2024-04-13T05:24:44.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.115+0000 7fcaddf80740 10 client.16866 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:44.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f36ab51ac0 con 0x55f36ab40c40 2024-04-13T05:24:44.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcabe000640 1 -- 172.21.15.57:0/680545330 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac409c680 con 0x55f36ab40c40 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcabe000640 1 -- 172.21.15.57:0/680545330 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcac0004d10 con 0x55f36ab40c40 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 10 client.16866 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 10 client.16866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 20 client.16866 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36abbcfa0 0x55f36abbf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 4 client.16866 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.116+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f36aaad8b0 con 0x55f36abbcfa0 2024-04-13T05:24:44.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.117+0000 7fcad3e00640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36abbcfa0 0x55f36abbf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcad3e00640 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36abbcfa0 0x55f36abbf390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f87db08aa5affa35 server_cookie=8316c7880cb29752 in_seq=0 out_seq=0 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcabe000640 10 client.16866 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcabe000640 1 -- 172.21.15.57:0/680545330 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f36aaad8b0 con 0x55f36abbcfa0 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcabe000640 10 client.16866 handle_command_reply: tid=0 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcaddf80740 1 client.16866 shutdown 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 msgr2=0x55f36ab4f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.118+0000 7fcaddf80740 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 0x55f36ab4f230 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac00095a0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 shutdown_connections 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55f36abbcfa0 0x55f36abbf390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f36ab45db0 0x55f36ab501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f36ab40c40 0x55f36ab4f230 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 --2- 172.21.15.57:0/680545330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f36ab40250 0x55f36ab4ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcabe000640 0 client.16866 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:24:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 >> 172.21.15.57:0/680545330 conn(0x55f36ab3b7c0 msgr2=0x55f36ab491c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 shutdown_connections 2024-04-13T05:24:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 1 -- 172.21.15.57:0/680545330 wait complete. 2024-04-13T05:24:44.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.119+0000 7fcaddf80740 20 client.16866 trim_cache size 0 max 16384 2024-04-13T05:24:44.123 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.120+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcab80a3c30 msgr2=0x7fcab80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.120+0000 7fcadb000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcab80a3c30 0x7fcab80a60f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f36ab414a0 tx=0x7fcacc004a40 comp rx=0 tx=0).stop 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.120+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 msgr2=0x7fcad419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.120+0000 7fcadb000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 0x7fcad419e850 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcac400d950 tx=0x7fcac400de10 comp rx=0 tx=0).stop 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.120+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 shutdown_connections 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fcab80a3c30 0x7fcab80a60f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcad4077bc0 0x7fcad419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcad4078560 0x7fcad41a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 --2- 172.21.15.57:0/1888334664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcad4076980 0x7fcad419e850 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 >> 172.21.15.57:0/1888334664 conn(0x7fcad40ff7b0 msgr2=0x7fcad4102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 shutdown_connections 2024-04-13T05:24:44.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.121+0000 7fcadb000640 1 -- 172.21.15.57:0/1888334664 wait complete. 2024-04-13T05:24:44.141 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:44 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1602100701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:24:44.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:24:44.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:44 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1602100701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:24:44.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 -- 172.21.15.57:0/1019413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418104240 msgr2=0x7f0418104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 --2- 172.21.15.57:0/1019413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418104240 0x7f0418104660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0408009900 tx=0x7f040802bc40 comp rx=0 tx=0).stop 2024-04-13T05:24:44.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 -- 172.21.15.57:0/1019413655 shutdown_connections 2024-04-13T05:24:44.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 --2- 172.21.15.57:0/1019413655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105df0 0x7f041810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 --2- 172.21.15.57:0/1019413655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0418105430 0x7f04181058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 --2- 172.21.15.57:0/1019413655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418104240 0x7f0418104660 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0408009900 tx=0x7f040802bc40 comp rx=0 tx=0).stop 2024-04-13T05:24:44.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.411+0000 7f041ca00640 1 -- 172.21.15.57:0/1019413655 >> 172.21.15.57:0/1019413655 conn(0x7f04180ff7b0 msgr2=0x7f0418101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.412+0000 7f041ca00640 1 -- 172.21.15.57:0/1019413655 shutdown_connections 2024-04-13T05:24:44.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.412+0000 7f041ca00640 1 -- 172.21.15.57:0/1019413655 wait complete. 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.413+0000 7f041ca00640 1 Processor -- start 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.413+0000 7f041ca00640 1 -- start start 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f041ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105430 0x7f041819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f041ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 0x7f041819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f041ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04181a4070 0x7f04181a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f041ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f041810fda0 con 0x7f04181a4070 2024-04-13T05:24:44.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f041ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f041810fc20 con 0x7f0418105df0 2024-04-13T05:24:44.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f041ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f041810ff20 con 0x7f0418105430 2024-04-13T05:24:44.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f0415400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 0x7f041819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f0415400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 0x7f041819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42592/0 (socket says 172.21.15.57:42592) 2024-04-13T05:24:44.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.414+0000 7f0415400640 1 -- 172.21.15.57:0/4084968333 learned_addr learned my addr 172.21.15.57:0/4084968333 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.415+0000 7f0416800640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04181a4070 0x7f04181a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.415+0000 7f0415e00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105430 0x7f041819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.415+0000 7f0415400640 1 -- 172.21.15.57:0/4084968333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105430 msgr2=0x7f041819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.415+0000 7f0415400640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105430 0x7f041819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.415+0000 7f0415400640 1 -- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04181a4070 msgr2=0x7f04181a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.416+0000 7f0415400640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04181a4070 0x7f04181a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.416+0000 7f0415400640 1 -- 172.21.15.57:0/4084968333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a6ad0 con 0x7f0418105df0 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.416+0000 7f0415e00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105430 0x7f041819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.416+0000 7f0416800640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04181a4070 0x7f04181a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:44.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.416+0000 7f0415400640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 0x7f041819f010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040400e9f0 tx=0x7f040400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.416+0000 7f040ea00640 1 -- 172.21.15.57:0/4084968333 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040400cd80 con 0x7f0418105df0 2024-04-13T05:24:44.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.417+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a6dc0 con 0x7f0418105df0 2024-04-13T05:24:44.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.417+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a7290 con 0x7f0418105df0 2024-04-13T05:24:44.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.417+0000 7f040ea00640 1 -- 172.21.15.57:0/4084968333 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04040108e0 con 0x7f0418105df0 2024-04-13T05:24:44.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.417+0000 7f040ea00640 1 -- 172.21.15.57:0/4084968333 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404018920 con 0x7f0418105df0 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.421+0000 7f041f7ea740 1 Processor -- start 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.421+0000 7f041f7ea740 1 -- start start 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.421+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915539680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 0x55691553e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55691553edc0 0x556915541250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f041f7ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55691548e020 con 0x556915539c50 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f041f7ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556915434950 con 0x55691553edc0 2024-04-13T05:24:44.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f041f7ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556915489fe0 con 0x556915539260 2024-04-13T05:24:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 0x55691553e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 0x55691553e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53726/0 (socket says 172.21.15.57:53726) 2024-04-13T05:24:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 -- 172.21.15.57:0/2253450999 learned_addr learned my addr 172.21.15.57:0/2253450999 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0416800640 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915539680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 -- 172.21.15.57:0/2253450999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 msgr2=0x556915539680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915539680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 -- 172.21.15.57:0/2253450999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55691553edc0 msgr2=0x556915541250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55691553edc0 0x556915541250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.422+0000 7f0415e00640 1 -- 172.21.15.57:0/2253450999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569151a03f0 con 0x556915539c50 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.423+0000 7f0415e00640 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 0x55691553e750 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0408000c00 tx=0x7f0408005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7015785d6304bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.423+0000 7f040ea00640 1 -- 172.21.15.57:0/4084968333 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0404010430 con 0x7f0418105df0 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.423+0000 7f0403000640 1 -- 172.21.15.57:0/2253450999 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408044020 con 0x556915539c50 2024-04-13T05:24:44.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.423+0000 7f0403000640 1 -- 172.21.15.57:0/2253450999 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040802cb70 con 0x556915539c50 2024-04-13T05:24:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.423+0000 7f0403000640 1 -- 172.21.15.57:0/2253450999 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408005650 con 0x556915539c50 2024-04-13T05:24:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.424+0000 7f041f7ea740 1 -- 172.21.15.57:0/2253450999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 msgr2=0x55691553e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.424+0000 7f041f7ea740 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 0x55691553e750 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0408000c00 tx=0x7f0408005d50 comp rx=0 tx=0).stop 2024-04-13T05:24:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f040ea00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03fc0a3c30 0x7f03fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f040ea00640 1 -- 172.21.15.57:0/4084968333 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f04040d09d0 con 0x7f0418105df0 2024-04-13T05:24:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f041f7ea740 1 -- 172.21.15.57:0/2253450999 shutdown_connections 2024-04-13T05:24:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f041f7ea740 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55691553edc0 0x556915541250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f041f7ea740 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556915539c50 0x55691553e750 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f041f7ea740 1 --2- 172.21.15.57:0/2253450999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915539680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.425+0000 7f041f7ea740 1 -- 172.21.15.57:0/2253450999 >> 172.21.15.57:0/2253450999 conn(0x5569155237c0 msgr2=0x556915523bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.426+0000 7f0416800640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03fc0a3c30 0x7f03fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.426+0000 7f041f7ea740 1 -- 172.21.15.57:0/2253450999 shutdown_connections 2024-04-13T05:24:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.426+0000 7f041f7ea740 1 -- 172.21.15.57:0/2253450999 wait complete. 2024-04-13T05:24:44.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.426+0000 7f041f7ea740 1 -- start start 2024-04-13T05:24:44.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.427+0000 7f041f7ea740 1 Processor -- start 2024-04-13T05:24:44.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.427+0000 7f0416800640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03fc0a3c30 0x7f03fc0a60f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55691553a4b0 tx=0x7f0410009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 0x556915543760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539c50 0x556915543ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x55691554ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556915541d10 con 0x55691553edc0 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556915447c80 con 0x556915539260 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556915541e90 con 0x556915539c50 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f041f7ea740 10 client.0 _ll_register_callbacks cb 0x556915447670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f0415e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 0x556915543760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f0415e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 0x556915543760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42608/0 (socket says 172.21.15.57:42608) 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.428+0000 7f0415e00640 1 -- 172.21.15.57:0/1399197798 learned_addr learned my addr 172.21.15.57:0/1399197798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0416800640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x55691554ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0415400640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539c50 0x556915543ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0415e00640 1 -- 172.21.15.57:0/1399197798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539c50 msgr2=0x556915543ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0415e00640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539c50 0x556915543ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0415e00640 1 -- 172.21.15.57:0/1399197798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 msgr2=0x55691554ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0415e00640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x55691554ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0415e00640 1 -- 172.21.15.57:0/1399197798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691526a7d0 con 0x556915539260 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.429+0000 7f0416800640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x55691554ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.430+0000 7f0415e00640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 0x556915543760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0408009850 tx=0x7f0408003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee914c9b50758e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.430+0000 7f03f7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:24:44.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.430+0000 7f03f7400640 1 -- 172.21.15.57:0/1399197798 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040804d070 con 0x556915539260 2024-04-13T05:24:44.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.431+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040802cb70 con 0x556915539260 2024-04-13T05:24:44.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.431+0000 7f03f7400640 1 -- 172.21.15.57:0/1399197798 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f04080055f0 con 0x556915539260 2024-04-13T05:24:44.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.431+0000 7f03f7400640 1 -- 172.21.15.57:0/1399197798 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0408059050 con 0x556915539260 2024-04-13T05:24:44.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.431+0000 7f041f7ea740 10 client.36275 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:44.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.431+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55691554e050 con 0x556915539260 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.431+0000 7f03f7400640 1 -- 172.21.15.57:0/1399197798 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040802cb70 con 0x556915539260 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f03f7400640 1 -- 172.21.15.57:0/1399197798 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f0408040c30 con 0x556915539260 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f041f7ea740 10 client.36275 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f041f7ea740 10 client.36275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f041f7ea740 20 client.36275 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55691554f3a0 0x556915551790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f041f7ea740 4 client.36275 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:44.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.432+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55691542d750 con 0x55691554f3a0 2024-04-13T05:24:44.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.433+0000 7f0415400640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55691554f3a0 0x556915551790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.434+0000 7f0415400640 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55691554f3a0 0x556915551790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd603fc23e76694a server_cookie=8dd6a6fbd78195e1 in_seq=0 out_seq=0 2024-04-13T05:24:44.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.434+0000 7f03f7400640 10 client.36275 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:44.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f03f7400640 1 -- 172.21.15.57:0/1399197798 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55691542d750 con 0x55691554f3a0 2024-04-13T05:24:44.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f03f7400640 10 client.36275 handle_command_reply: tid=0 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 client.36275 shutdown 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 msgr2=0x556915543760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 0x556915543760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0408009850 tx=0x7f0408003820 comp rx=0 tx=0).stop 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 shutdown_connections 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55691554f3a0 0x556915551790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x55691554ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539c50 0x556915543ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1399197798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539260 0x556915543760 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 >> 172.21.15.57:0/1399197798 conn(0x5569155237c0 msgr2=0x556915522a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.436+0000 7f03f7400640 0 client.36275 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:44.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.437+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 shutdown_connections 2024-04-13T05:24:44.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.437+0000 7f041f7ea740 1 -- 172.21.15.57:0/1399197798 wait complete. 2024-04-13T05:24:44.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.437+0000 7f041f7ea740 20 client.36275 trim_cache size 0 max 16384 2024-04-13T05:24:44.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 Processor -- start 2024-04-13T05:24:44.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 -- start start 2024-04-13T05:24:44.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x55691553a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x55691554aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915542a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f03f80124f0 con 0x55691553edc0 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03f800fd60 con 0x556915539c50 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.461+0000 7f041f7ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03f800dbb0 con 0x556915539260 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915542a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915542a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53748/0 (socket says 172.21.15.57:53748) 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 -- 172.21.15.57:0/1230960780 learned_addr learned my addr 172.21.15.57:0/1230960780 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0415400640 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x55691554aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0415e00640 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x55691553a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 -- 172.21.15.57:0/1230960780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 msgr2=0x55691553a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x55691553a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 -- 172.21.15.57:0/1230960780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 msgr2=0x55691554aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x55691554aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.462+0000 7f0416800640 1 -- 172.21.15.57:0/1230960780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556915551dc0 con 0x55691553edc0 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.463+0000 7f0416800640 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915542a70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0410013fd0 tx=0x7f041000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f96c66bc9210e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.464+0000 7f0402600640 1 -- 172.21.15.57:0/1230960780 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041002c070 con 0x55691553edc0 2024-04-13T05:24:44.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.464+0000 7f0402600640 1 -- 172.21.15.57:0/1230960780 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f041001cb10 con 0x55691553edc0 2024-04-13T05:24:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.464+0000 7f0402600640 1 -- 172.21.15.57:0/1230960780 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410027bc0 con 0x55691553edc0 2024-04-13T05:24:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 -- 172.21.15.57:0/1230960780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 msgr2=0x556915542a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915542a70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0410013fd0 tx=0x7f041000b000 comp rx=0 tx=0).stop 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 -- 172.21.15.57:0/1230960780 shutdown_connections 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x55691554aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915542a70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1230960780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x55691553a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 -- 172.21.15.57:0/1230960780 >> 172.21.15.57:0/1230960780 conn(0x5569155237c0 msgr2=0x5569155272b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.465+0000 7f041f7ea740 1 -- 172.21.15.57:0/1230960780 shutdown_connections 2024-04-13T05:24:44.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.466+0000 7f041f7ea740 1 -- 172.21.15.57:0/1230960780 wait complete. 2024-04-13T05:24:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.466+0000 7f041f7ea740 1 -- start start 2024-04-13T05:24:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.466+0000 7f041f7ea740 1 Processor -- start 2024-04-13T05:24:44.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.467+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915548960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.467+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x556915548ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.467+0000 7f041f7ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915549eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0415400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x556915548ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0415400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x556915548ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42634/0 (socket says 172.21.15.57:42634) 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0415e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915548960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0415e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915548960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:60504/0 (socket says 172.21.15.57:60504) 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0416800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915549eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0416800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915549eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53758/0 (socket says 172.21.15.57:53758) 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f041f7ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f03f800e220 con 0x55691553edc0 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f041f7ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03f800f690 con 0x556915539c50 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f041f7ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03f800e8f0 con 0x556915539260 2024-04-13T05:24:44.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f041f7ea740 10 client.0 _ll_register_callbacks cb 0x7f0408004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:44.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.468+0000 7f0415400640 1 -- 172.21.15.57:0/1104952174 learned_addr learned my addr 172.21.15.57:0/1104952174 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0416800640 1 -- 172.21.15.57:0/1104952174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 msgr2=0x556915548960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0416800640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915548960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0416800640 1 -- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 msgr2=0x556915548ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0416800640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x556915548ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0416800640 1 -- 172.21.15.57:0/1104952174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691554a4a0 con 0x55691553edc0 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0415e00640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915548960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.469+0000 7f0415400640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x556915548ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.470+0000 7f0416800640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915549eb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0410013fd0 tx=0x7f0410000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54815b629f874d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.470+0000 7f03f7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.470+0000 7f03f7400640 1 -- 172.21.15.57:0/1104952174 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410034020 con 0x55691553edc0 2024-04-13T05:24:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.471+0000 7f03f7400640 1 -- 172.21.15.57:0/1104952174 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f041001c870 con 0x55691553edc0 2024-04-13T05:24:44.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.471+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f041001cb10 con 0x55691553edc0 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.471+0000 7f03f7400640 1 -- 172.21.15.57:0/1104952174 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f041002c040 con 0x55691553edc0 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.471+0000 7f041f7ea740 10 client.16884 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.471+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569155b5e20 con 0x55691553edc0 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f03f7400640 1 -- 172.21.15.57:0/1104952174 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f041001cb10 con 0x55691553edc0 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f03f7400640 1 -- 172.21.15.57:0/1104952174 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f041001a040 con 0x55691553edc0 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f041f7ea740 10 client.16884 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f041f7ea740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f041f7ea740 20 client.16884 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:44.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5569155b6710 0x5569155b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f041f7ea740 4 client.16884 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:24:44.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55691549ba40 con 0x5569155b6710 2024-04-13T05:24:44.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.472+0000 7f0415e00640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5569155b6710 0x5569155b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.473+0000 7f0415e00640 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5569155b6710 0x5569155b8b30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0f804e8f2381d9a server_cookie=4ef83e3cdb9c8c3c in_seq=0 out_seq=0 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.473+0000 7f03f7400640 10 client.16884 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.473+0000 7f03f7400640 1 -- 172.21.15.57:0/1104952174 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55691549ba40 con 0x5569155b6710 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.473+0000 7f03f7400640 10 client.16884 handle_command_reply: tid=0 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.473+0000 7f041f7ea740 1 client.16884 shutdown 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 msgr2=0x556915549eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915549eb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0410013fd0 tx=0x7f0410000f40 comp rx=0 tx=0).stop 2024-04-13T05:24:44.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 shutdown_connections 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5569155b6710 0x5569155b8b30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556915539c50 0x556915548ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55691553edc0 0x556915549eb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 --2- 172.21.15.57:0/1104952174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556915539260 0x556915548960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f03f7400640 0 client.16884 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 >> 172.21.15.57:0/1104952174 conn(0x5569155237c0 msgr2=0x5569155221f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 shutdown_connections 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.474+0000 7f041f7ea740 1 -- 172.21.15.57:0/1104952174 wait complete. 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.475+0000 7f041f7ea740 20 client.16884 trim_cache size 0 max 16384 2024-04-13T05:24:44.478 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:24:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.475+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03fc0a3c30 msgr2=0x7f03fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.475+0000 7f041ca00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03fc0a3c30 0x7f03fc0a60f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55691553a4b0 tx=0x7f0410009600 comp rx=0 tx=0).stop 2024-04-13T05:24:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.475+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 msgr2=0x7f041819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.475+0000 7f041ca00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 0x7f041819f010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040400e9f0 tx=0x7f040400eeb0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 shutdown_connections 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f03fc0a3c30 0x7f03fc0a60f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418105df0 0x7f041819f010 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04181a4070 0x7f04181a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 --2- 172.21.15.57:0/4084968333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418105430 0x7f041819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 >> 172.21.15.57:0/4084968333 conn(0x7f04180ff7b0 msgr2=0x7f0418078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 shutdown_connections 2024-04-13T05:24:44.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.476+0000 7f041ca00640 1 -- 172.21.15.57:0/4084968333 wait complete. 2024-04-13T05:24:44.499 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:24:44.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:44 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1602100701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:24:44.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 -- 172.21.15.57:0/2972980517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c076980 msgr2=0x7f914c076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 --2- 172.21.15.57:0/2972980517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c076980 0x7f914c076da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9140009be0 tx=0x7f914002bef0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 -- 172.21.15.57:0/2972980517 shutdown_connections 2024-04-13T05:24:44.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 --2- 172.21.15.57:0/2972980517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f914c078560 0x7f914c078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 --2- 172.21.15.57:0/2972980517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c077bc0 0x7f914c078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 --2- 172.21.15.57:0/2972980517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c076980 0x7f914c076da0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 -- 172.21.15.57:0/2972980517 >> 172.21.15.57:0/2972980517 conn(0x7f914c0ff670 msgr2=0x7f914c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 -- 172.21.15.57:0/2972980517 shutdown_connections 2024-04-13T05:24:44.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.774+0000 7f9152400640 1 -- 172.21.15.57:0/2972980517 wait complete. 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 Processor -- start 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 -- start start 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 0x7f914c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f914c077bc0 0x7f914c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c078560 0x7f914c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f914c114080 con 0x7f914c076980 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f914c113f00 con 0x7f914c078560 2024-04-13T05:24:44.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.776+0000 7f9152400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f914c114200 con 0x7f914c077bc0 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 0x7f914c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 0x7f914c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53790/0 (socket says 172.21.15.57:53790) 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 -- 172.21.15.57:0/344725880 learned_addr learned my addr 172.21.15.57:0/344725880 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151a00640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c078560 0x7f914c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f914be00640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f914c077bc0 0x7f914c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 -- 172.21.15.57:0/344725880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f914c077bc0 msgr2=0x7f914c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f914c077bc0 0x7f914c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 -- 172.21.15.57:0/344725880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c078560 msgr2=0x7f914c1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c078560 0x7f914c1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151000640 1 -- 172.21.15.57:0/344725880 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c1a6350 con 0x7f914c076980 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.777+0000 7f9151a00640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c078560 0x7f914c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.778+0000 7f9151000640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 0x7f914c19e730 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9140004af0 tx=0x7f9140033a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.778+0000 7f9149600640 1 -- 172.21.15.57:0/344725880 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140043020 con 0x7f914c076980 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.779+0000 7f9149600640 1 -- 172.21.15.57:0/344725880 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9140003e20 con 0x7f914c076980 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.779+0000 7f9149600640 1 -- 172.21.15.57:0/344725880 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140034430 con 0x7f914c076980 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.779+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c1a65e0 con 0x7f914c076980 2024-04-13T05:24:44.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.779+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1a6b80 con 0x7f914c076980 2024-04-13T05:24:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.783+0000 7f915696d740 1 Processor -- start 2024-04-13T05:24:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.783+0000 7f915696d740 1 -- start start 2024-04-13T05:24:44.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f915696d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d5173860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f915696d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5178930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f915696d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d517b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f915696d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5575d50d9020 con 0x5575d5173e30 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f915696d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575d507f950 con 0x5575d5173440 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f915696d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5575d50d4fe0 con 0x5575d5178fa0 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f9149600640 1 -- 172.21.15.57:0/344725880 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9140046020 con 0x7f914c076980 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f9151a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d5173860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f9151000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5178930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f9151a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d5173860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42654/0 (socket says 172.21.15.57:42654) 2024-04-13T05:24:44.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f9151a00640 1 -- 172.21.15.57:0/2122775608 learned_addr learned my addr 172.21.15.57:0/2122775608 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.784+0000 7f914be00640 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d517b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f9151a00640 1 -- 172.21.15.57:0/2122775608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 msgr2=0x5575d517b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f9151a00640 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d517b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f9151a00640 1 -- 172.21.15.57:0/2122775608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 msgr2=0x5575d5178930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f9151a00640 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5178930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f9151a00640 1 -- 172.21.15.57:0/2122775608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d4deb3f0 con 0x5575d5173440 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f914be00640 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d517b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.785+0000 7f9151a00640 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d5173860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9144009cc0 tx=0x7f91440117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb0b68f6e83a5d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.786+0000 7f913e000640 1 -- 172.21.15.57:0/2122775608 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144012450 con 0x5575d5173440 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.786+0000 7f913e000640 1 -- 172.21.15.57:0/2122775608 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9144012aa0 con 0x5575d5173440 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.786+0000 7f913e000640 1 -- 172.21.15.57:0/2122775608 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914400fbb0 con 0x5575d5173440 2024-04-13T05:24:44.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.786+0000 7f9149600640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91340a3cb0 0x7f91340a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.786+0000 7f9151000640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91340a3cb0 0x7f91340a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.786+0000 7f9149600640 1 -- 172.21.15.57:0/344725880 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f91400f30b0 con 0x7f914c076980 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 -- 172.21.15.57:0/2122775608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 msgr2=0x5575d5173860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d5173860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9144009cc0 tx=0x7f91440117b0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f9151000640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91340a3cb0 0x7f91340a6170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9140006010 tx=0x7f91400f6a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 -- 172.21.15.57:0/2122775608 shutdown_connections 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d517b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5178930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 --2- 172.21.15.57:0/2122775608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d5173860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.787+0000 7f915696d740 1 -- 172.21.15.57:0/2122775608 >> 172.21.15.57:0/2122775608 conn(0x5575d516e9b0 msgr2=0x5575d516edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.788+0000 7f915696d740 1 -- 172.21.15.57:0/2122775608 shutdown_connections 2024-04-13T05:24:44.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.788+0000 7f915696d740 1 -- 172.21.15.57:0/2122775608 wait complete. 2024-04-13T05:24:44.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 -- start start 2024-04-13T05:24:44.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 Processor -- start 2024-04-13T05:24:44.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5173440 0x5575d517d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d517df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5178fa0 0x5575d5184f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5575d517bf50 con 0x5575d5173e30 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575d5092c80 con 0x5575d5178fa0 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.789+0000 7f915696d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5575d517c0d0 con 0x5575d5173440 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.790+0000 7f915696d740 10 client.0 _ll_register_callbacks cb 0x5575d5092670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.790+0000 7f914be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d517df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.790+0000 7f914be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d517df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53804/0 (socket says 172.21.15.57:53804) 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.790+0000 7f914be00640 1 -- 172.21.15.57:0/3644205935 learned_addr learned my addr 172.21.15.57:0/3644205935 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.790+0000 7f9151a00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5178fa0 0x5575d5184f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.790+0000 7f9151a00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5173440 0x5575d517d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f914be00640 1 -- 172.21.15.57:0/3644205935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5173440 msgr2=0x5575d517d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f914be00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5173440 0x5575d517d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f914be00640 1 -- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5178fa0 msgr2=0x5575d5184f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f914be00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5178fa0 0x5575d5184f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f914be00640 1 -- 172.21.15.57:0/3644205935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d4eb57d0 con 0x5575d5173e30 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f914be00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d517df10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9138009bf0 tx=0x7f91380039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279db709a841712e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f9132a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.791+0000 7f9132a00640 1 -- 172.21.15.57:0/3644205935 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138012860 con 0x5575d5173e30 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.792+0000 7f9132a00640 1 -- 172.21.15.57:0/3644205935 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9138012a00 con 0x5575d5173e30 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.792+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9144012aa0 con 0x5575d5173e30 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.792+0000 7f9132a00640 1 -- 172.21.15.57:0/3644205935 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9138012c00 con 0x5575d5173e30 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.792+0000 7f915696d740 10 client.16896 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:44.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.792+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575d5199360 con 0x5575d5173e30 2024-04-13T05:24:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f9132a00640 1 -- 172.21.15.57:0/3644205935 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9144012aa0 con 0x5575d5173e30 2024-04-13T05:24:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f9132a00640 1 -- 172.21.15.57:0/3644205935 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9138025420 con 0x5575d5173e30 2024-04-13T05:24:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f915696d740 10 client.16896 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f915696d740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f915696d740 20 client.16896 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:44.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f915696d740 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d519aae0 0x5575d519cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f915696d740 4 client.16896 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.793+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575d5078750 con 0x5575d519aae0 2024-04-13T05:24:44.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.794+0000 7f9151a00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d519aae0 0x5575d519cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.795+0000 7f9151a00640 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d519aae0 0x5575d519cf00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9dd21bb2252b41 server_cookie=e991cb671f3e427e in_seq=0 out_seq=0 2024-04-13T05:24:44.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.795+0000 7f9132a00640 10 client.16896 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:44.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f9132a00640 1 -- 172.21.15.57:0/3644205935 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575d5078750 con 0x5575d519aae0 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f9132a00640 10 client.16896 handle_command_reply: tid=0 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 client.16896 shutdown 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 msgr2=0x5575d517df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d517df10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9138009bf0 tx=0x7f91380039c0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 shutdown_connections 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d519aae0 0x5575d519cf00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5178fa0 0x5575d5184f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d517df10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 --2- 172.21.15.57:0/3644205935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5173440 0x5575d517d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 >> 172.21.15.57:0/3644205935 conn(0x5575d516e9b0 msgr2=0x5575d516dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.797+0000 7f9132a00640 0 client.16896 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:44.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.798+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 shutdown_connections 2024-04-13T05:24:44.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.798+0000 7f915696d740 1 -- 172.21.15.57:0/3644205935 wait complete. 2024-04-13T05:24:44.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.798+0000 7f915696d740 20 client.16896 trim_cache size 0 max 16384 2024-04-13T05:24:44.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 Processor -- start 2024-04-13T05:24:44.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 -- start start 2024-04-13T05:24:44.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d51860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5184f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d516d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1ab0f0 con 0x5575d5173e30 2024-04-13T05:24:44.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1a8960 con 0x5575d5173440 2024-04-13T05:24:44.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.821+0000 7f915696d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f914c10bc50 con 0x5575d5178fa0 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f914be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5184f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d51860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d51860b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42674/0 (socket says 172.21.15.57:42674) 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 -- 172.21.15.57:0/3373787355 learned_addr learned my addr 172.21.15.57:0/3373787355 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d516d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 -- 172.21.15.57:0/3373787355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 msgr2=0x5575d516d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d516d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 -- 172.21.15.57:0/3373787355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 msgr2=0x5575d5184f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5184f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f9151a00640 1 -- 172.21.15.57:0/3373787355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d517c250 con 0x5575d5173440 2024-04-13T05:24:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.822+0000 7f914be00640 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5184f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.823+0000 7f9151a00640 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d51860b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9144009fd0 tx=0x7f91440117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee0d635729b80277 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.823+0000 7f913d600640 1 -- 172.21.15.57:0/3373787355 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144009680 con 0x5575d5173440 2024-04-13T05:24:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.823+0000 7f913d600640 1 -- 172.21.15.57:0/3373787355 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91440054f0 con 0x5575d5173440 2024-04-13T05:24:44.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.823+0000 7f913d600640 1 -- 172.21.15.57:0/3373787355 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144012a80 con 0x5575d5173440 2024-04-13T05:24:44.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 -- 172.21.15.57:0/3373787355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 msgr2=0x5575d51860b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d51860b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9144009fd0 tx=0x7f91440117b0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 -- 172.21.15.57:0/3373787355 shutdown_connections 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d516d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173e30 0x5575d5184f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 --2- 172.21.15.57:0/3373787355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173440 0x5575d51860b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 -- 172.21.15.57:0/3373787355 >> 172.21.15.57:0/3373787355 conn(0x5575d516e9b0 msgr2=0x5575d51859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.824+0000 7f915696d740 1 -- 172.21.15.57:0/3373787355 shutdown_connections 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.825+0000 7f915696d740 1 -- 172.21.15.57:0/3373787355 wait complete. 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.825+0000 7f915696d740 1 -- start start 2024-04-13T05:24:44.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.825+0000 7f915696d740 1 Processor -- start 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 0x5575d5182010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173e30 0x5575d5182550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d5183560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1a6ea0 con 0x5575d5173440 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1a8290 con 0x5575d5173e30 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f914c1a74f0 con 0x5575d5178fa0 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f915696d740 10 client.0 _ll_register_callbacks cb 0x7f9138009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f9151a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 0x5575d5182010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f914be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173e30 0x5575d5182550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f9151a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 0x5575d5182010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53810/0 (socket says 172.21.15.57:53810) 2024-04-13T05:24:44.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f914be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173e30 0x5575d5182550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:42682/0 (socket says 172.21.15.57:42682) 2024-04-13T05:24:44.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.826+0000 7f9151a00640 1 -- 172.21.15.57:0/3943318409 learned_addr learned my addr 172.21.15.57:0/3943318409 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d5183560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 -- 172.21.15.57:0/3943318409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 msgr2=0x5575d5183560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d5183560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 -- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173e30 msgr2=0x5575d5182550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173e30 0x5575d5182550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 -- 172.21.15.57:0/3943318409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d5183b50 con 0x5575d5173440 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9151a00640 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 0x5575d5182010 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9144009570 tx=0x7f9144009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d1283c160edb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.827+0000 7f9132a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.828+0000 7f9132a00640 1 -- 172.21.15.57:0/3943318409 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144012450 con 0x5575d5173440 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.828+0000 7f9132a00640 1 -- 172.21.15.57:0/3943318409 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f91440125f0 con 0x5575d5173440 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.828+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91440054f0 con 0x5575d5173440 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.828+0000 7f9132a00640 1 -- 172.21.15.57:0/3943318409 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9144005040 con 0x5575d5173440 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.828+0000 7f915696d740 10 client.16902 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:44.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.828+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575d5184e50 con 0x5575d5173440 2024-04-13T05:24:44.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f9132a00640 1 -- 172.21.15.57:0/3943318409 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91440054f0 con 0x5575d5173440 2024-04-13T05:24:44.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f9132a00640 1 -- 172.21.15.57:0/3943318409 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9144026040 con 0x5575d5173440 2024-04-13T05:24:44.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f915696d740 10 client.16902 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:44.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f915696d740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:44.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f915696d740 20 client.16902 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:44.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f915696d740 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d52012c0 0x5575d52036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:44.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f915696d740 4 client.16902 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:24:44.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.829+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575d50e6a40 con 0x5575d52012c0 2024-04-13T05:24:44.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.830+0000 7f914be00640 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d52012c0 0x5575d52036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:44.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.831+0000 7f914be00640 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d52012c0 0x5575d52036e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f596ebaf386b5d server_cookie=bc4e575e855d6b29 in_seq=0 out_seq=0 2024-04-13T05:24:44.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.831+0000 7f9132a00640 10 client.16902 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:44.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f9132a00640 1 -- 172.21.15.57:0/3943318409 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575d50e6a40 con 0x5575d52012c0 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f9132a00640 10 client.16902 handle_command_reply: tid=0 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 client.16902 shutdown 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 msgr2=0x5575d5182010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 0x5575d5182010 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9144009570 tx=0x7f9144009e20 comp rx=0 tx=0).stop 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 shutdown_connections 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5575d52012c0 0x5575d52036e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575d5173e30 0x5575d5182550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d5178fa0 0x5575d5183560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 --2- 172.21.15.57:0/3943318409 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5575d5173440 0x5575d5182010 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 >> 172.21.15.57:0/3943318409 conn(0x5575d516e9b0 msgr2=0x5575d5171c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f9132a00640 0 client.16902 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 shutdown_connections 2024-04-13T05:24:44.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 1 -- 172.21.15.57:0/3943318409 wait complete. 2024-04-13T05:24:44.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.832+0000 7f915696d740 20 client.16902 trim_cache size 0 max 16384 2024-04-13T05:24:44.836 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91340a3cb0 msgr2=0x7f91340a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91340a3cb0 0x7f91340a6170 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9140006010 tx=0x7f91400f6a70 comp rx=0 tx=0).stop 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 msgr2=0x7f914c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 0x7f914c19e730 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9140004af0 tx=0x7f9140033a90 comp rx=0 tx=0).stop 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 shutdown_connections 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f91340a3cb0 0x7f91340a6170 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f914c078560 0x7f914c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f914c077bc0 0x7f914c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 --2- 172.21.15.57:0/344725880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f914c076980 0x7f914c19e730 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.833+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 >> 172.21.15.57:0/344725880 conn(0x7f914c0ff670 msgr2=0x7f914c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.834+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 shutdown_connections 2024-04-13T05:24:44.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:44.834+0000 7f9152400640 1 -- 172.21.15.57:0/344725880 wait complete. 2024-04-13T05:24:45.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:45 smithi057 ceph-mon[32331]: pgmap v670: 129 pgs: 129 active+clean; 428 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-04-13T05:24:45.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:45 smithi189 ceph-mon[35531]: pgmap v670: 129 pgs: 129 active+clean; 428 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-04-13T05:24:45.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:45 smithi083 ceph-mon[35133]: pgmap v670: 129 pgs: 129 active+clean; 428 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-04-13T05:24:45.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:24:45.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:24:45.859 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:24:46.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 -- 172.21.15.57:0/859416656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd01030b0 msgr2=0x7f6fd0105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/859416656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd01030b0 0x7f6fd0105520 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009900 tx=0x7f6fbc02bc40 comp rx=0 tx=0).stop 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 -- 172.21.15.57:0/859416656 shutdown_connections 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/859416656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fd0105a60 0x7f6fd0105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/859416656 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd01030b0 0x7f6fd0105520 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/859416656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd0100740 0x7f6fd0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 -- 172.21.15.57:0/859416656 >> 172.21.15.57:0/859416656 conn(0x7f6fd00fc360 msgr2=0x7f6fd00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 -- 172.21.15.57:0/859416656 shutdown_connections 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.120+0000 7f6fd6a00640 1 -- 172.21.15.57:0/859416656 wait complete. 2024-04-13T05:24:46.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.121+0000 7f6fd6a00640 1 Processor -- start 2024-04-13T05:24:46.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.121+0000 7f6fd6a00640 1 -- start start 2024-04-13T05:24:46.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd0100740 0x7f6fd019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 0x7f6fd019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fd01a3ea0 0x7f6fd01a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd6a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0114040 con 0x7f6fd0105a60 2024-04-13T05:24:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd0113ec0 con 0x7f6fd0100740 2024-04-13T05:24:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd6a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01141c0 con 0x7f6fd01a3ea0 2024-04-13T05:24:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 0x7f6fd019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 0x7f6fd019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56928/0 (socket says 172.21.15.57:56928) 2024-04-13T05:24:46.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd4c00640 1 -- 172.21.15.57:0/3700170389 learned_addr learned my addr 172.21.15.57:0/3700170389 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.122+0000 7f6fd5600640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd0100740 0x7f6fd019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd6000640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fd01a3ea0 0x7f6fd01a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd4c00640 1 -- 172.21.15.57:0/3700170389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fd01a3ea0 msgr2=0x7f6fd01a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd4c00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fd01a3ea0 0x7f6fd01a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd4c00640 1 -- 172.21.15.57:0/3700170389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd0100740 msgr2=0x7f6fd019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd4c00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd0100740 0x7f6fd019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd4c00640 1 -- 172.21.15.57:0/3700170389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd01a6900 con 0x7f6fd0105a60 2024-04-13T05:24:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.123+0000 7f6fd4c00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 0x7f6fd019ee40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c00 tx=0x7f6fbc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.124+0000 7f6fce000640 1 -- 172.21.15.57:0/3700170389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc043020 con 0x7f6fd0105a60 2024-04-13T05:24:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.124+0000 7f6fce000640 1 -- 172.21.15.57:0/3700170389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fbc0044c0 con 0x7f6fd0105a60 2024-04-13T05:24:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.124+0000 7f6fce000640 1 -- 172.21.15.57:0/3700170389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc003bb0 con 0x7f6fd0105a60 2024-04-13T05:24:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.124+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd01a6b90 con 0x7f6fd0105a60 2024-04-13T05:24:46.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.124+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd01a7090 con 0x7f6fd0105a60 2024-04-13T05:24:46.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.128+0000 7f6fdae65740 1 Processor -- start 2024-04-13T05:24:46.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.128+0000 7f6fdae65740 1 -- start start 2024-04-13T05:24:46.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc77910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc7a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fdae65740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a06dbd8bc0 con 0x55a06dc72420 2024-04-13T05:24:46.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fdae65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a06db7f660 con 0x55a06dc72e10 2024-04-13T05:24:46.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fdae65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a06dbd4b80 con 0x55a06dc77f80 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc72840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56932/0 (socket says 172.21.15.57:56932) 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fd6000640 1 -- 172.21.15.57:0/2397633586 learned_addr learned my addr 172.21.15.57:0/2397633586 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.129+0000 7f6fd5600640 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc77910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fd6000640 1 -- 172.21.15.57:0/2397633586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 msgr2=0x55a06dc7a410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fd6000640 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc7a410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fd6000640 1 -- 172.21.15.57:0/2397633586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 msgr2=0x55a06dc77910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fd6000640 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc77910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fce000640 1 -- 172.21.15.57:0/3700170389 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6fbc046020 con 0x7f6fd0105a60 2024-04-13T05:24:46.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fd6000640 1 -- 172.21.15.57:0/2397633586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06dba2220 con 0x55a06dc72420 2024-04-13T05:24:46.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.130+0000 7f6fd6000640 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc72840 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009ee0 tx=0x7f6fc8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fb210e268773f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.131+0000 7f6fc2600640 1 -- 172.21.15.57:0/2397633586 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8016070 con 0x55a06dc72420 2024-04-13T05:24:46.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.131+0000 7f6fc2600640 1 -- 172.21.15.57:0/2397633586 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc8004d20 con 0x55a06dc72420 2024-04-13T05:24:46.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.131+0000 7f6fc2600640 1 -- 172.21.15.57:0/2397633586 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8012740 con 0x55a06dc72420 2024-04-13T05:24:46.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fce000640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6fb80a3cb0 0x7f6fb80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 -- 172.21.15.57:0/2397633586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 msgr2=0x55a06dc72840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc72840 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009ee0 tx=0x7f6fc8010740 comp rx=0 tx=0).stop 2024-04-13T05:24:46.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fce000640 1 -- 172.21.15.57:0/3700170389 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6fbc0f2980 con 0x7f6fd0105a60 2024-04-13T05:24:46.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 -- 172.21.15.57:0/2397633586 shutdown_connections 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc77910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc7a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 --2- 172.21.15.57:0/2397633586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc72840 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.132+0000 7f6fdae65740 1 -- 172.21.15.57:0/2397633586 >> 172.21.15.57:0/2397633586 conn(0x55a06dc6d990 msgr2=0x55a06dc6dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.133+0000 7f6fdae65740 1 -- 172.21.15.57:0/2397633586 shutdown_connections 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.133+0000 7f6fd5600640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6fb80a3cb0 0x7f6fb80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.133+0000 7f6fdae65740 1 -- 172.21.15.57:0/2397633586 wait complete. 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.133+0000 7f6fdae65740 1 -- start start 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 Processor -- start 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fd5600640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6fb80a3cb0 0x7f6fb80a6170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55a06dc6e900 tx=0x7f6fc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc7c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc7cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc83f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a06dc7af30 con 0x55a06dc77f80 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a06db92a30 con 0x55a06dc72e10 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a06dc7b0b0 con 0x55a06dc72420 2024-04-13T05:24:46.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.134+0000 7f6fdae65740 10 client.0 _ll_register_callbacks cb 0x55a06d8eac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:46.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc83f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc7cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc83f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56940/0 (socket says 172.21.15.57:56940) 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 -- 172.21.15.57:0/4125494503 learned_addr learned my addr 172.21.15.57:0/4125494503 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc7cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:33698/0 (socket says 172.21.15.57:33698) 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd5600640 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc7c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 -- 172.21.15.57:0/4125494503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 msgr2=0x55a06dc7c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc7c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 -- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 msgr2=0x55a06dc7cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc7cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 -- 172.21.15.57:0/4125494503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06d9b37e0 con 0x55a06dc77f80 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.135+0000 7f6fd6000640 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc83f50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006010 tx=0x7f6fc80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955ff4f8481a5e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fb6a00640 1 -- 172.21.15.57:0/4125494503 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8016040 con 0x55a06dc77f80 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fb6a00640 1 -- 172.21.15.57:0/4125494503 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6fc80052e0 con 0x55a06dc77f80 2024-04-13T05:24:46.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc8004d20 con 0x55a06dc77f80 2024-04-13T05:24:46.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fb6a00640 1 -- 172.21.15.57:0/4125494503 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fc8030050 con 0x55a06dc77f80 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fdae65740 10 client.16914 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.136+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a06da13f70 con 0x55a06dc77f80 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fb6a00640 1 -- 172.21.15.57:0/4125494503 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc80054e0 con 0x55a06dc77f80 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fb6a00640 1 -- 172.21.15.57:0/4125494503 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6fc802b030 con 0x55a06dc77f80 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fdae65740 10 client.16914 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fdae65740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fdae65740 20 client.16914 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fdae65740 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dc884f0 0x55a06dc8a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fdae65740 4 client.16914 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.137+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6fc8004d20 con 0x55a06dc884f0 2024-04-13T05:24:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.138+0000 7f6fd5600640 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dc884f0 0x55a06dc8a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.139+0000 7f6fd5600640 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dc884f0 0x55a06dc8a8e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69ad36102ef8c21 server_cookie=76d0b84f26a7b79a in_seq=0 out_seq=0 2024-04-13T05:24:46.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.139+0000 7f6fb6a00640 10 client.16914 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.140+0000 7f6fb6a00640 1 -- 172.21.15.57:0/4125494503 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6fc8004d20 con 0x55a06dc884f0 2024-04-13T05:24:46.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.140+0000 7f6fb6a00640 10 client.16914 handle_command_reply: tid=0 2024-04-13T05:24:46.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.140+0000 7f6fdae65740 1 client.16914 shutdown 2024-04-13T05:24:46.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 msgr2=0x55a06dc83f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc83f50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006010 tx=0x7f6fc80058d0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 shutdown_connections 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dc884f0 0x55a06dc8a8e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc7cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc83f50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 --2- 172.21.15.57:0/4125494503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc7c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fb6a00640 0 client.16914 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 >> 172.21.15.57:0/4125494503 conn(0x55a06dc6d990 msgr2=0x55a06dc6cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 shutdown_connections 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.141+0000 7f6fdae65740 1 -- 172.21.15.57:0/4125494503 wait complete. 2024-04-13T05:24:46.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.142+0000 7f6fdae65740 20 client.16914 trim_cache size 0 max 16384 2024-04-13T05:24:46.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.176+0000 7f6fdae65740 1 Processor -- start 2024-04-13T05:24:46.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 -- start start 2024-04-13T05:24:46.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc6d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc71300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b3e90 con 0x55a06dc77f80 2024-04-13T05:24:46.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b1700 con 0x55a06dc72e10 2024-04-13T05:24:46.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fdae65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01af550 con 0x55a06dc72420 2024-04-13T05:24:46.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc71300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fd6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc71300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56956/0 (socket says 172.21.15.57:56956) 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.177+0000 7f6fd6000640 1 -- 172.21.15.57:0/3906430909 learned_addr learned my addr 172.21.15.57:0/3906430909 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd4c00640 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd5600640 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc6d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd6000640 1 -- 172.21.15.57:0/3906430909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 msgr2=0x55a06dc6d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd6000640 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc6d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd6000640 1 -- 172.21.15.57:0/3906430909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 msgr2=0x55a06dc83f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd6000640 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd6000640 1 -- 172.21.15.57:0/3906430909 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06dc7bb30 con 0x55a06dc77f80 2024-04-13T05:24:46.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.178+0000 7f6fd6000640 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc71300 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000f80 tx=0x7f6fc8009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebf3d839fc369e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.179+0000 7f6fc1c00640 1 -- 172.21.15.57:0/3906430909 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8016040 con 0x55a06dc77f80 2024-04-13T05:24:46.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.179+0000 7f6fc1c00640 1 -- 172.21.15.57:0/3906430909 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc801a730 con 0x55a06dc77f80 2024-04-13T05:24:46.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.179+0000 7f6fc1c00640 1 -- 172.21.15.57:0/3906430909 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8005590 con 0x55a06dc77f80 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 -- 172.21.15.57:0/3906430909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 msgr2=0x55a06dc71300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc71300 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000f80 tx=0x7f6fc8009de0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 -- 172.21.15.57:0/3906430909 shutdown_connections 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc77f80 0x55a06dc71300 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 --2- 172.21.15.57:0/3906430909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc72420 0x55a06dc6d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 -- 172.21.15.57:0/3906430909 >> 172.21.15.57:0/3906430909 conn(0x55a06dc6d990 msgr2=0x55a06dc70bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 -- 172.21.15.57:0/3906430909 shutdown_connections 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 -- 172.21.15.57:0/3906430909 wait complete. 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.180+0000 7f6fdae65740 1 -- start start 2024-04-13T05:24:46.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 Processor -- start 2024-04-13T05:24:46.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc81e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01afbc0 con 0x55a06dc72420 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b1030 con 0x55a06dc72e10 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01b0290 con 0x55a06dc77f80 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.181+0000 7f6fdae65740 10 client.0 _ll_register_callbacks cb 0x7f6fc40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd5600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:33718/0 (socket says 172.21.15.57:33718) 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd4c00640 1 -- 172.21.15.57:0/630290091 learned_addr learned my addr 172.21.15.57:0/630290091 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd6000640 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc81e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd5600640 1 -- 172.21.15.57:0/630290091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 msgr2=0x55a06dc81e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd5600640 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc81e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd5600640 1 -- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 msgr2=0x55a06dc83ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd5600640 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.182+0000 7f6fd5600640 1 -- 172.21.15.57:0/630290091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06d8ab170 con 0x55a06dc72420 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.183+0000 7f6fd5600640 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc837a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x55a06dc8b030 tx=0x7f6fc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1026247b2fb1cec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.183+0000 7f6fb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.183+0000 7f6fb6a00640 1 -- 172.21.15.57:0/630290091 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402b530 con 0x55a06dc72420 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.183+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc801a730 con 0x55a06dc72420 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.183+0000 7f6fb6a00640 1 -- 172.21.15.57:0/630290091 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6fc402b6d0 con 0x55a06dc72420 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fb6a00640 1 -- 172.21.15.57:0/630290091 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fc403b050 con 0x55a06dc72420 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 10 client.16920 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a06db93be0 con 0x55a06dc72420 2024-04-13T05:24:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fb6a00640 1 -- 172.21.15.57:0/630290091 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fc801a730 con 0x55a06dc72420 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fb6a00640 1 -- 172.21.15.57:0/630290091 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6fc401f560 con 0x55a06dc72420 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 10 client.16920 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 20 client.16920 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dcefa50 0x55a06dcf1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 4 client.16920 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:24:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.184+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a06dbe65e0 con 0x55a06dcefa50 2024-04-13T05:24:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.185+0000 7f6fd4c00640 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dcefa50 0x55a06dcf1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.186+0000 7f6fd4c00640 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dcefa50 0x55a06dcf1e70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff609d00426a556 server_cookie=cf285e930a13f479 in_seq=0 out_seq=0 2024-04-13T05:24:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.186+0000 7f6fb6a00640 10 client.16920 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.188+0000 7f6fb6a00640 1 -- 172.21.15.57:0/630290091 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a06dbe65e0 con 0x55a06dcefa50 2024-04-13T05:24:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.188+0000 7f6fb6a00640 10 client.16920 handle_command_reply: tid=0 2024-04-13T05:24:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.188+0000 7f6fdae65740 1 client.16920 shutdown 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.188+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 msgr2=0x55a06dc837a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.188+0000 7f6fdae65740 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc837a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x55a06dc8b030 tx=0x7f6fc4004730 comp rx=0 tx=0).stop 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.188+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 shutdown_connections 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a06dcefa50 0x55a06dcf1e70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a06dc72e10 0x55a06dc83ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a06dc77f80 0x55a06dc81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 --2- 172.21.15.57:0/630290091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a06dc72420 0x55a06dc837a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 >> 172.21.15.57:0/630290091 conn(0x55a06dc6d990 msgr2=0x55a06dc71480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fb6a00640 0 client.16920 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 shutdown_connections 2024-04-13T05:24:46.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 1 -- 172.21.15.57:0/630290091 wait complete. 2024-04-13T05:24:46.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.189+0000 7f6fdae65740 20 client.16920 trim_cache size 0 max 16384 2024-04-13T05:24:46.192 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:24:46.193 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:24:46.193 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "265687f9-feaf-4f52-80b6-de763cd81ba0", 2024-04-13T05:24:46.193 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:24:46.193 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:24:46.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6fb80a3cb0 msgr2=0x7f6fb80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6fb80a3cb0 0x7f6fb80a6170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55a06dc6e900 tx=0x7f6fc4005f90 comp rx=0 tx=0).stop 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 msgr2=0x7f6fd019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 0x7f6fd019ee40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c00 tx=0x7f6fbc0040c0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 shutdown_connections 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f6fb80a3cb0 0x7f6fb80a6170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fd01a3ea0 0x7f6fd01a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fd0105a60 0x7f6fd019ee40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 --2- 172.21.15.57:0/3700170389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fd0100740 0x7f6fd019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.190+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 >> 172.21.15.57:0/3700170389 conn(0x7f6fd00fc360 msgr2=0x7f6fd0101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.191+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 shutdown_connections 2024-04-13T05:24:46.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.191+0000 7f6fd6a00640 1 -- 172.21.15.57:0/3700170389 wait complete. 2024-04-13T05:24:46.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:24:46.213 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:24:46.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 -- 172.21.15.57:0/2933942170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a8107c60 msgr2=0x7f49a810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 --2- 172.21.15.57:0/2933942170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a8107c60 0x7f49a810a120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009be0 tx=0x7f49a002bef0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 -- 172.21.15.57:0/2933942170 shutdown_connections 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 --2- 172.21.15.57:0/2933942170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a81006d0 0x7f49a81076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 --2- 172.21.15.57:0/2933942170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a8107c60 0x7f49a810a120 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 --2- 172.21.15.57:0/2933942170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 0x7f49a8100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.487+0000 7f49ad000640 1 -- 172.21.15.57:0/2933942170 >> 172.21.15.57:0/2933942170 conn(0x7f49a80fb7c0 msgr2=0x7f49a80fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.488+0000 7f49ad000640 1 -- 172.21.15.57:0/2933942170 shutdown_connections 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.488+0000 7f49ad000640 1 -- 172.21.15.57:0/2933942170 wait complete. 2024-04-13T05:24:46.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.489+0000 7f49ad000640 1 Processor -- start 2024-04-13T05:24:46.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.489+0000 7f49ad000640 1 -- start start 2024-04-13T05:24:46.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.489+0000 7f49ad000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 0x7f49a819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.489+0000 7f49ad000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 0x7f49a819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49ad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a8107c60 0x7f49a81a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49ad000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810db30 con 0x7f49a81006d0 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49ad000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810d9b0 con 0x7f49a8107c60 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49ad000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810dcb0 con 0x7f49a80ffd70 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 0x7f49a819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 0x7f49a819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56982/0 (socket says 172.21.15.57:56982) 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5400640 1 -- 172.21.15.57:0/2332103765 learned_addr learned my addr 172.21.15.57:0/2332103765 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5e00640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 0x7f49a819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a6800640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a8107c60 0x7f49a81a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5400640 1 -- 172.21.15.57:0/2332103765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 msgr2=0x7f49a819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5400640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 0x7f49a819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.490+0000 7f49a5400640 1 -- 172.21.15.57:0/2332103765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a8107c60 msgr2=0x7f49a81a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f49a5400640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a8107c60 0x7f49a81a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f49a5400640 1 -- 172.21.15.57:0/2332103765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a81a1ff0 con 0x7f49a81006d0 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f49a5e00640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 0x7f49a819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f49a6800640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a8107c60 0x7f49a81a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f49a5400640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 0x7f49a819a960 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f499400e9f0 tx=0x7f499400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f499ea00640 1 -- 172.21.15.57:0/2332103765 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400ccc0 con 0x7f49a81006d0 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f499ea00640 1 -- 172.21.15.57:0/2332103765 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49940035f0 con 0x7f49a81006d0 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f499ea00640 1 -- 172.21.15.57:0/2332103765 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400ce60 con 0x7f49a81006d0 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.491+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a8103710 con 0x7f49a81006d0 2024-04-13T05:24:46.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.492+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a8103c10 con 0x7f49a81006d0 2024-04-13T05:24:46.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 Processor -- start 2024-04-13T05:24:46.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 -- start start 2024-04-13T05:24:46.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa6b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 0x55bfefa709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bfef9d2020 con 0x55bfefa6bec0 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bfef9786d0 con 0x55bfefa71030 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.496+0000 7f49aff6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bfef9cdfe0 con 0x55bfefa6b4d0 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.497+0000 7f49a5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 0x55bfefa709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.497+0000 7f49a5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 0x55bfefa709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56986/0 (socket says 172.21.15.57:56986) 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.497+0000 7f49a5e00640 1 -- 172.21.15.57:0/1713141360 learned_addr learned my addr 172.21.15.57:0/1713141360 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.497+0000 7f49a6800640 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa6b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.498+0000 7f499ea00640 1 -- 172.21.15.57:0/2332103765 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49940037a0 con 0x7f49a81006d0 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.498+0000 7f49a5e00640 1 -- 172.21.15.57:0/1713141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 msgr2=0x55bfefa6b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.498+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa6b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.498+0000 7f49a5e00640 1 -- 172.21.15.57:0/1713141360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 msgr2=0x55bfefa734c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.498+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa734c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.498+0000 7f49a5e00640 1 -- 172.21.15.57:0/1713141360 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfef7ae7d0 con 0x55bfefa6bec0 2024-04-13T05:24:46.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.499+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 0x55bfefa709c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4998009d50 tx=0x7f49980117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da94db77db594963 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.499+0000 7f4993400640 1 -- 172.21.15.57:0/1713141360 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998012650 con 0x55bfefa6bec0 2024-04-13T05:24:46.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.499+0000 7f4993400640 1 -- 172.21.15.57:0/1713141360 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4998009e40 con 0x55bfefa6bec0 2024-04-13T05:24:46.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.499+0000 7f4993400640 1 -- 172.21.15.57:0/1713141360 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499800fbf0 con 0x55bfefa6bec0 2024-04-13T05:24:46.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.500+0000 7f499ea00640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f498c0a3c00 0x7f498c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.500+0000 7f499ea00640 1 -- 172.21.15.57:0/2332103765 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49940d0b00 con 0x7f49a81006d0 2024-04-13T05:24:46.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.500+0000 7f49aff6f740 1 -- 172.21.15.57:0/1713141360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 msgr2=0x55bfefa709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.500+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 0x55bfefa709c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4998009d50 tx=0x7f49980117b0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.501+0000 7f49a6800640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f498c0a3c00 0x7f498c0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.501+0000 7f49aff6f740 1 -- 172.21.15.57:0/1713141360 shutdown_connections 2024-04-13T05:24:46.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.501+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.501+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6bec0 0x55bfefa709c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.501+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1713141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa6b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.501+0000 7f49aff6f740 1 -- 172.21.15.57:0/1713141360 >> 172.21.15.57:0/1713141360 conn(0x55bfefa66a40 msgr2=0x55bfefa66e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.502+0000 7f49aff6f740 1 -- 172.21.15.57:0/1713141360 shutdown_connections 2024-04-13T05:24:46.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.502+0000 7f49aff6f740 1 -- 172.21.15.57:0/1713141360 wait complete. 2024-04-13T05:24:46.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.502+0000 7f49a6800640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f498c0a3c00 0x7f498c0a60c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f49a0006010 tx=0x7f49a0005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.502+0000 7f49aff6f740 1 -- start start 2024-04-13T05:24:46.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.503+0000 7f49aff6f740 1 Processor -- start 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.503+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa86810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.503+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa86d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.503+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa71030 0x55bfefa8dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49aff6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bfefa84ff0 con 0x55bfefa71030 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49aff6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bfef98ba90 con 0x55bfefa6bec0 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49aff6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bfefa85170 con 0x55bfefa6b4d0 2024-04-13T05:24:46.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49aff6f740 10 client.0 _ll_register_callbacks cb 0x55bfef98b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:46.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa86d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa86d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:33742/0 (socket says 172.21.15.57:33742) 2024-04-13T05:24:46.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49a5400640 1 -- 172.21.15.57:0/1096098846 learned_addr learned my addr 172.21.15.57:0/1096098846 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.504+0000 7f49a6800640 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa71030 0x55bfefa8dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f49a6800640 1 -- 172.21.15.57:0/1096098846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 msgr2=0x55bfefa86810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f49a6800640 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa86810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f49a6800640 1 -- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 msgr2=0x55bfefa86d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f49a6800640 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa86d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f49a6800640 1 -- 172.21.15.57:0/1096098846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfef6e43f0 con 0x55bfefa71030 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f49a6800640 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa71030 0x55bfefa8dd90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f49a0036040 tx=0x7f49a002bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2cd027497bd33a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.505+0000 7f4987e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:46.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.506+0000 7f4987e00640 1 -- 172.21.15.57:0/1096098846 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0046070 con 0x55bfefa71030 2024-04-13T05:24:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.506+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bfef971600 con 0x55bfefa71030 2024-04-13T05:24:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.506+0000 7f4987e00640 1 -- 172.21.15.57:0/1096098846 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49a0034430 con 0x55bfefa71030 2024-04-13T05:24:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.506+0000 7f4987e00640 1 -- 172.21.15.57:0/1096098846 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49a0033790 con 0x55bfefa71030 2024-04-13T05:24:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.506+0000 7f49aff6f740 10 client.16932 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:46.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.506+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfef6a40f0 con 0x55bfefa71030 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f4987e00640 1 -- 172.21.15.57:0/1096098846 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bfef971600 con 0x55bfefa71030 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f4987e00640 1 -- 172.21.15.57:0/1096098846 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f49a004d630 con 0x55bfefa71030 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f49aff6f740 10 client.16932 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f49aff6f740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f49aff6f740 20 client.16932 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefa926d0 0x55bfefa94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f49aff6f740 4 client.16932 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.507+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfef9dfa40 con 0x55bfefa926d0 2024-04-13T05:24:46.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.508+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefa926d0 0x55bfefa94b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.509+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefa926d0 0x55bfefa94b60 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72a55891c05a444 server_cookie=73f5d93c533c802e in_seq=0 out_seq=0 2024-04-13T05:24:46.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.509+0000 7f4987e00640 10 client.16932 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.512+0000 7f4987e00640 1 -- 172.21.15.57:0/1096098846 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfef9dfa40 con 0x55bfefa926d0 2024-04-13T05:24:46.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.512+0000 7f4987e00640 10 client.16932 handle_command_reply: tid=0 2024-04-13T05:24:46.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.512+0000 7f49aff6f740 1 client.16932 shutdown 2024-04-13T05:24:46.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa71030 msgr2=0x55bfefa8dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa71030 0x55bfefa8dd90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f49a0036040 tx=0x7f49a002bef0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 shutdown_connections 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefa926d0 0x55bfefa94b60 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa86d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa71030 0x55bfefa8dd90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1096098846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6b4d0 0x55bfefa86810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 >> 172.21.15.57:0/1096098846 conn(0x55bfefa66a40 msgr2=0x55bfefa65b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.513+0000 7f4987e00640 0 client.16932 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.514+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 shutdown_connections 2024-04-13T05:24:46.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.514+0000 7f49aff6f740 1 -- 172.21.15.57:0/1096098846 wait complete. 2024-04-13T05:24:46.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.514+0000 7f49aff6f740 20 client.16932 trim_cache size 0 max 16384 2024-04-13T05:24:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.549+0000 7f49aff6f740 1 Processor -- start 2024-04-13T05:24:46.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.550+0000 7f49aff6f740 1 -- start start 2024-04-13T05:24:46.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.551+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa85e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.551+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa8dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa71030 0x55bfefa652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49aff6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81ab2e0 con 0x55bfefa6b4d0 2024-04-13T05:24:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49aff6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a8b50 con 0x55bfefa6bec0 2024-04-13T05:24:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49aff6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a69a0 con 0x55bfefa71030 2024-04-13T05:24:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa8dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa8dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:33748/0 (socket says 172.21.15.57:33748) 2024-04-13T05:24:46.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.552+0000 7f49a5400640 1 -- 172.21.15.57:0/334582622 learned_addr learned my addr 172.21.15.57:0/334582622 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.553+0000 7f49a6800640 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa71030 0x55bfefa652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.553+0000 7f49a5e00640 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa85e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.554+0000 7f49a5400640 1 -- 172.21.15.57:0/334582622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa71030 msgr2=0x55bfefa652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.554+0000 7f49a5400640 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa71030 0x55bfefa652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.554+0000 7f49a5400640 1 -- 172.21.15.57:0/334582622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 msgr2=0x55bfefa85e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.554+0000 7f49a5400640 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa85e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.554+0000 7f49a5400640 1 -- 172.21.15.57:0/334582622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfefa677b0 con 0x55bfefa6bec0 2024-04-13T05:24:46.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.556+0000 7f49a5400640 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa8dd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f499409c740 tx=0x7f4994087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fadc88655d8f314c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.556+0000 7f4992a00640 1 -- 172.21.15.57:0/334582622 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49940b5030 con 0x55bfefa6bec0 2024-04-13T05:24:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.556+0000 7f4992a00640 1 -- 172.21.15.57:0/334582622 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4994003b10 con 0x55bfefa6bec0 2024-04-13T05:24:46.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.556+0000 7f4992a00640 1 -- 172.21.15.57:0/334582622 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499409ad40 con 0x55bfefa6bec0 2024-04-13T05:24:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.557+0000 7f49aff6f740 1 -- 172.21.15.57:0/334582622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 msgr2=0x55bfefa8dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.557+0000 7f49aff6f740 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa8dd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f499409c740 tx=0x7f4994087040 comp rx=0 tx=0).stop 2024-04-13T05:24:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.558+0000 7f49aff6f740 1 -- 172.21.15.57:0/334582622 shutdown_connections 2024-04-13T05:24:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.558+0000 7f49aff6f740 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa6bec0 0x55bfefa8dd50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.558+0000 7f49aff6f740 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa71030 0x55bfefa652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.558+0000 7f49aff6f740 1 --2- 172.21.15.57:0/334582622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa85e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.558+0000 7f49aff6f740 1 -- 172.21.15.57:0/334582622 >> 172.21.15.57:0/334582622 conn(0x55bfefa66a40 msgr2=0x55bfefa6d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.559+0000 7f49aff6f740 1 -- 172.21.15.57:0/334582622 shutdown_connections 2024-04-13T05:24:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.559+0000 7f49aff6f740 1 -- 172.21.15.57:0/334582622 wait complete. 2024-04-13T05:24:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.559+0000 7f49aff6f740 1 -- start start 2024-04-13T05:24:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.559+0000 7f49aff6f740 1 Processor -- start 2024-04-13T05:24:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.560+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa8af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6bec0 0x55bfefa8b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49aff6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa8c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49aff6f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a7010 con 0x55bfefa6b4d0 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49aff6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a8480 con 0x55bfefa71030 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49aff6f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81a76e0 con 0x55bfefa6bec0 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49aff6f740 10 client.0 _ll_register_callbacks cb 0x7f49a0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49a5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa8af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49a5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa8af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57008/0 (socket says 172.21.15.57:57008) 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.561+0000 7f49a5e00640 1 -- 172.21.15.57:0/1452327818 learned_addr learned my addr 172.21.15.57:0/1452327818 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.562+0000 7f49a5e00640 1 -- 172.21.15.57:0/1452327818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6bec0 msgr2=0x55bfefa8b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.562+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6bec0 0x55bfefa8b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.562+0000 7f49a5e00640 1 -- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 msgr2=0x55bfefa8c480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:24:46.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.562+0000 7f49a6800640 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa8c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.562+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa8c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.562+0000 7f49a5e00640 1 -- 172.21.15.57:0/1452327818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfefa8ca70 con 0x55bfefa6b4d0 2024-04-13T05:24:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.563+0000 7f49a5e00640 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa8af30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f49980117f0 tx=0x7f4998009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3363af7c5d9e2780 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:24:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.563+0000 7f4987e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:24:46.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.564+0000 7f4987e00640 1 -- 172.21.15.57:0/1452327818 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998005450 con 0x55bfefa6b4d0 2024-04-13T05:24:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.564+0000 7f4987e00640 1 -- 172.21.15.57:0/1452327818 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49980270e0 con 0x55bfefa6b4d0 2024-04-13T05:24:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.564+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bfefa86600 con 0x55bfefa6b4d0 2024-04-13T05:24:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.564+0000 7f4987e00640 1 -- 172.21.15.57:0/1452327818 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49980055f0 con 0x55bfefa6b4d0 2024-04-13T05:24:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.565+0000 7f49aff6f740 10 client.16938 fetch_fsmap learned FSMap version 30 2024-04-13T05:24:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.565+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfefa8de20 con 0x55bfefa6b4d0 2024-04-13T05:24:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f4987e00640 1 -- 172.21.15.57:0/1452327818 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bfefa86600 con 0x55bfefa6b4d0 2024-04-13T05:24:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f4987e00640 1 -- 172.21.15.57:0/1452327818 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f4998003b80 con 0x55bfefa6b4d0 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f49aff6f740 10 client.16938 fetch_fsmap finished waiting for FSMap version 30 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f49aff6f740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f49aff6f740 20 client.16938 populate_metadata read hostname 'smithi057' 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefaf8ec0 0x55bfefafb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f49aff6f740 4 client.16938 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.566+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bfef9d98b0 con 0x55bfefaf8ec0 2024-04-13T05:24:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.567+0000 7f49a5400640 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefaf8ec0 0x55bfefafb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:24:46.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.569+0000 7f49a5400640 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefaf8ec0 0x55bfefafb2e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a35bdd6d3d4d9d server_cookie=affb00becff2b5ea in_seq=0 out_seq=0 2024-04-13T05:24:46.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.569+0000 7f4987e00640 10 client.16938 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f4987e00640 1 -- 172.21.15.57:0/1452327818 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55bfef9d98b0 con 0x55bfefaf8ec0 2024-04-13T05:24:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f4987e00640 10 client.16938 handle_command_reply: tid=0 2024-04-13T05:24:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 client.16938 shutdown 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 msgr2=0x55bfefa8af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa8af30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f49980117f0 tx=0x7f4998009530 comp rx=0 tx=0).stop 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 shutdown_connections 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55bfefaf8ec0 0x55bfefafb2e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfefa71030 0x55bfefa8c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f4987e00640 0 client.16938 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:24:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfefa6bec0 0x55bfefa8b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 --2- 172.21.15.57:0/1452327818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bfefa6b4d0 0x55bfefa8af30 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.573+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 >> 172.21.15.57:0/1452327818 conn(0x55bfefa66a40 msgr2=0x55bfefa69ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.574+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 shutdown_connections 2024-04-13T05:24:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.574+0000 7f49aff6f740 1 -- 172.21.15.57:0/1452327818 wait complete. 2024-04-13T05:24:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.574+0000 7f49aff6f740 20 client.16938 trim_cache size 0 max 16384 2024-04-13T05:24:46.578 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:24:46.578 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (2253 inodes in the stack)", 2024-04-13T05:24:46.578 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout: "265687f9-feaf-4f52-80b6-de763cd81ba0": { 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "265687f9-feaf-4f52-80b6-de763cd81ba0", 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:24:46.579 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.575+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f498c0a3c00 msgr2=0x7f498c0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.575+0000 7f49ad000640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f498c0a3c00 0x7f498c0a60c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f49a0006010 tx=0x7f49a0005ec0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.575+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 msgr2=0x7f49a819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.575+0000 7f49ad000640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 0x7f49a819a960 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f499400e9f0 tx=0x7f499400eeb0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.576+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 shutdown_connections 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.576+0000 7f49ad000640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f498c0a3c00 0x7f498c0a60c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.576+0000 7f49ad000640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a8107c60 0x7f49a81a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.576+0000 7f49ad000640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49a81006d0 0x7f49a819a960 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.576+0000 7f49ad000640 1 --2- 172.21.15.57:0/2332103765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49a80ffd70 0x7f49a819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.576+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 >> 172.21.15.57:0/2332103765 conn(0x7f49a80fb7c0 msgr2=0x7f49a80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:24:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.577+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 shutdown_connections 2024-04-13T05:24:46.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:24:46.577+0000 7f49ad000640 1 -- 172.21.15.57:0/2332103765 wait complete. 2024-04-13T05:24:46.634 INFO:tasks.cephfs.filesystem:scrub status for tag:265687f9-feaf-4f52-80b6-de763cd81ba0 - {'path': '/', 'tag': '265687f9-feaf-4f52-80b6-de763cd81ba0', 'options': 'recursive,force'} 2024-04-13T05:24:46.752 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:24:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:24:46.284+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001a4d [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44/ auth{2=1} v81253 ap=2 DIRTYPARENT f(v12 m2024-04-13T05:23:31.482085+0000 85=85+0) n(v5 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@357} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06149180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000001a4d:[<0x2000000021a/blog-44 v81253>,<0x10000000208/blogtest_in v14057>,<0x10000000201/src v2829>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:23:31.370751+0000 86=86+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1)","memory_value.dirstat":"f(v12 m2024-04-13T05:23:31.482085+0000 85=85+0)","memory_value.rstat":"n(v5 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-13T05:24:46.753 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:24:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:24:46.289+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000021b [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0/ auth{0=1,2=1} fragtree_t(*^3) v47842 ap=2 f(v342 m2024-04-13T05:21:28.256974+0000 273=273+0) n(v10 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1) (inest sync->mix g=0,2) caps={15447=pAsLsXs/-@1608} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559b052eb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x2000000021b:[<0x2000000021a/blog-0 v47842>,<0x10000000208/blogtest_in v11726>,<0x10000000201/src v2821>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x2000000021b:[<0x2000000021a/blog-0 v47842>,<0x10000000208/blogtest_in v14055>,<0x10000000201/src v2829>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:21:28.196434+0000 274=274+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1)","memory_value.dirstat":"f(v342 m2024-04-13T05:21:28.256974+0000 273=273+0)","memory_value.rstat":"n(v10 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:24:46.753 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:24:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:24:46.326+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001100 [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/ auth{2=1} v73177 ap=2 f(v16 m2024-04-13T05:22:48.044991+0000 79=79+0) n(v5 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@198} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06366580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000001100:[<0x2000000021a/blog-25 v73177>,<0x10000000208/blogtest_in v13864>,<0x10000000201/src v2827>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x20000001100:[<0x2000000021a/blog-25 v73177>,<0x10000000208/blogtest_in v14061>,<0x10000000201/src v2829>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:22:47.912987+0000 80=80+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1)","memory_value.dirstat":"f(v16 m2024-04-13T05:22:48.044991+0000 79=79+0)","memory_value.rstat":"n(v5 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:24:46.753 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:24:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:24:46.405+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000124e [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28/ auth{2=1} fragtree_t(*^3) v89249 ap=2 DIRTYPARENT f(v327 m2024-04-13T05:24:09.361281+0000 176=176+0) n(v16 rc2024-04-13T05:24:09.361281+0000 b2739044 177=176+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@1192} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06337080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x2000000124e:[<0x2000000021a/blog-28 v89249>,<0x10000000208/blogtest_in v14069>,<0x10000000201/src v2829>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:24:09.226284+0000 177=177+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:24:09.361281+0000 b2739044 177=176+1)","memory_value.dirstat":"f(v327 m2024-04-13T05:24:09.361281+0000 176=176+0)","memory_value.rstat":"n(v16 rc2024-04-13T05:24:09.361281+0000 b2739044 177=176+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-04-13T05:24:46.753 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:24:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:24:46.503+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000a25 [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-12/ auth{2=1} fragtree_t(*^3) v88559 ap=2 f(v166 m2024-04-13T05:24:02.817990+0000 162=162+0) n(v13 rc2024-04-13T05:24:02.817990+0000 b3165556 163=162+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@783} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559b05636100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000000a25:[<0x2000000021a/blog-12 v88559>,<0x10000000208/blogtest_in v12408>,<0x10000000201/src v2823>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x20000000a25:[<0x2000000021a/blog-12 v88559>,<0x10000000208/blogtest_in v14085>,<0x10000000201/src v2829>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:24:02.565198+0000 163=163+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:24:02.817990+0000 b3165556 163=162+1)","memory_value.dirstat":"f(v166 m2024-04-13T05:24:02.817990+0000 162=162+0)","memory_value.rstat":"n(v13 rc2024-04-13T05:24:02.817990+0000 b3165556 163=162+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: pgmap v671: 129 pgs: 129 active+clean; 433 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 292 op/s 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: Scrub error on inode 0x2000000124e (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:47 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000000a25 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-12) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: pgmap v671: 129 pgs: 129 active+clean; 433 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 292 op/s 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: Scrub error on inode 0x2000000124e (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:47 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000000a25 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-12) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: pgmap v671: 129 pgs: 129 active+clean; 433 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 292 op/s 2024-04-13T05:24:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: Scrub error on inode 0x2000000124e (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28) see mds.h log and `damage ls` output for details 2024-04-13T05:24:47.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:47 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000000a25 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-12) see mds.h log and `damage ls` output for details 2024-04-13T05:24:48.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:24:48.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:48.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:24:48.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:48 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:24:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:48.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:24:48.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:48 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:48.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:48 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:49 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:49 smithi189 ceph-mon[35531]: pgmap v672: 129 pgs: 129 active+clean; 433 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-04-13T05:24:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:49 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:49 smithi083 ceph-mon[35133]: pgmap v672: 129 pgs: 129 active+clean; 433 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-04-13T05:24:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:49 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:24:49.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:49 smithi057 ceph-mon[32331]: pgmap v672: 129 pgs: 129 active+clean; 433 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-04-13T05:24:49.638 INFO:tasks.workunit.client.0.smithi057.stdout: 56 7342 20 4378 64 2749 119 2024-04-13T05:24:51.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:51 smithi189 ceph-mon[35531]: pgmap v673: 129 pgs: 129 active+clean; 439 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-04-13T05:24:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:51 smithi083 ceph-mon[35133]: pgmap v673: 129 pgs: 129 active+clean; 439 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-04-13T05:24:51.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:51 smithi057 ceph-mon[32331]: pgmap v673: 129 pgs: 129 active+clean; 439 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 371 op/s 2024-04-13T05:24:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:53 smithi189 ceph-mon[35531]: pgmap v674: 129 pgs: 129 active+clean; 441 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 348 op/s 2024-04-13T05:24:53.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:53 smithi057 ceph-mon[32331]: pgmap v674: 129 pgs: 129 active+clean; 441 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 348 op/s 2024-04-13T05:24:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:53 smithi083 ceph-mon[35133]: pgmap v674: 129 pgs: 129 active+clean; 441 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 348 op/s 2024-04-13T05:24:55.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:55 smithi189 ceph-mon[35531]: pgmap v675: 129 pgs: 129 active+clean; 447 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 374 op/s 2024-04-13T05:24:55.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:55 smithi083 ceph-mon[35133]: pgmap v675: 129 pgs: 129 active+clean; 447 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 374 op/s 2024-04-13T05:24:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:55 smithi057 ceph-mon[32331]: pgmap v675: 129 pgs: 129 active+clean; 447 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 374 op/s 2024-04-13T05:24:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:57 smithi189 ceph-mon[35531]: pgmap v676: 129 pgs: 129 active+clean; 449 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 360 op/s 2024-04-13T05:24:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:57 smithi083 ceph-mon[35133]: pgmap v676: 129 pgs: 129 active+clean; 449 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 360 op/s 2024-04-13T05:24:57.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:57 smithi057 ceph-mon[32331]: pgmap v676: 129 pgs: 129 active+clean; 449 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 360 op/s 2024-04-13T05:24:58.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:58 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:24:58.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:58 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:24:58.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:58 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:24:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:58 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:24:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:58 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:24:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:58 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:24:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:58 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:24:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:58 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:24:58.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:58 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:24:59.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:24:59 smithi189 ceph-mon[35531]: pgmap v677: 129 pgs: 129 active+clean; 449 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 312 op/s 2024-04-13T05:24:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:24:59 smithi083 ceph-mon[35133]: pgmap v677: 129 pgs: 129 active+clean; 449 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 312 op/s 2024-04-13T05:24:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:24:59 smithi057 ceph-mon[32331]: pgmap v677: 129 pgs: 129 active+clean; 449 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 312 op/s 2024-04-13T05:24:59.639 INFO:tasks.workunit.client.0.smithi057.stdout: 57 8033 31 4797 65 2510 126 2024-04-13T05:25:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:01 smithi083 ceph-mon[35133]: pgmap v678: 129 pgs: 129 active+clean; 457 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 368 op/s 2024-04-13T05:25:01.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:01 smithi057 ceph-mon[32331]: pgmap v678: 129 pgs: 129 active+clean; 457 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 368 op/s 2024-04-13T05:25:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:01 smithi189 ceph-mon[35531]: pgmap v678: 129 pgs: 129 active+clean; 457 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 368 op/s 2024-04-13T05:25:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:03 smithi083 ceph-mon[35133]: pgmap v679: 129 pgs: 129 active+clean; 457 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-04-13T05:25:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:03 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:25:03.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:03 smithi057 ceph-mon[32331]: pgmap v679: 129 pgs: 129 active+clean; 457 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-04-13T05:25:03.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:03 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:25:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:03 smithi189 ceph-mon[35531]: pgmap v679: 129 pgs: 129 active+clean; 457 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2024-04-13T05:25:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:03 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:25:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:05 smithi083 ceph-mon[35133]: pgmap v680: 129 pgs: 129 active+clean; 459 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-04-13T05:25:05.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:25:05.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:25:05.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:25:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:05 smithi057 ceph-mon[32331]: pgmap v680: 129 pgs: 129 active+clean; 459 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-04-13T05:25:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:25:05.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:25:05.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:25:05.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:05 smithi189 ceph-mon[35531]: pgmap v680: 129 pgs: 129 active+clean; 459 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-04-13T05:25:05.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:25:05.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:25:05.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:25:07.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:07 smithi083 ceph-mon[35133]: pgmap v681: 129 pgs: 129 active+clean; 464 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-04-13T05:25:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:07 smithi057 ceph-mon[32331]: pgmap v681: 129 pgs: 129 active+clean; 464 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-04-13T05:25:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:07 smithi189 ceph-mon[35531]: pgmap v681: 129 pgs: 129 active+clean; 464 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 140 op/s 2024-04-13T05:25:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:09 smithi083 ceph-mon[35133]: pgmap v682: 129 pgs: 129 active+clean; 464 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-04-13T05:25:09.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:09 smithi057 ceph-mon[32331]: pgmap v682: 129 pgs: 129 active+clean; 464 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-04-13T05:25:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:09 smithi189 ceph-mon[35531]: pgmap v682: 129 pgs: 129 active+clean; 464 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-04-13T05:25:09.639 INFO:tasks.workunit.client.0.smithi057.stdout: 59 7956 54 5491 75 2704 122 2024-04-13T05:25:11.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:11 smithi083 ceph-mon[35133]: pgmap v683: 129 pgs: 129 active+clean; 452 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-04-13T05:25:11.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:11 smithi057 ceph-mon[32331]: pgmap v683: 129 pgs: 129 active+clean; 452 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-04-13T05:25:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:11 smithi189 ceph-mon[35531]: pgmap v683: 129 pgs: 129 active+clean; 452 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-04-13T05:25:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:13 smithi083 ceph-mon[35133]: pgmap v684: 129 pgs: 129 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:25:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:13 smithi057 ceph-mon[32331]: pgmap v684: 129 pgs: 129 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:25:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:13 smithi189 ceph-mon[35531]: pgmap v684: 129 pgs: 129 active+clean; 453 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-04-13T05:25:15.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:15 smithi083 ceph-mon[35133]: pgmap v685: 129 pgs: 129 active+clean; 460 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-04-13T05:25:15.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:15 smithi057 ceph-mon[32331]: pgmap v685: 129 pgs: 129 active+clean; 460 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-04-13T05:25:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:15 smithi189 ceph-mon[35531]: pgmap v685: 129 pgs: 129 active+clean; 460 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-04-13T05:25:16.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:25:16.635 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:25:16.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 -- 172.21.15.57:0/2917773949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 msgr2=0x7ff924110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 --2- 172.21.15.57:0/2917773949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff924110940 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009ca0 tx=0x7ff91c02c100 comp rx=0 tx=0).stop 2024-04-13T05:25:16.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 -- 172.21.15.57:0/2917773949 shutdown_connections 2024-04-13T05:25:16.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 --2- 172.21.15.57:0/2917773949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff924110940 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 --2- 172.21.15.57:0/2917773949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9241096f0 0x7ff924109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 --2- 172.21.15.57:0/2917773949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff924108500 0x7ff924108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 -- 172.21.15.57:0/2917773949 >> 172.21.15.57:0/2917773949 conn(0x7ff924077490 msgr2=0x7ff9240798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:16.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.874+0000 7ff92a400640 1 -- 172.21.15.57:0/2917773949 shutdown_connections 2024-04-13T05:25:16.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.875+0000 7ff92a400640 1 -- 172.21.15.57:0/2917773949 wait complete. 2024-04-13T05:25:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 Processor -- start 2024-04-13T05:25:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 -- start start 2024-04-13T05:25:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924108500 0x7ff92419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9241096f0 0x7ff92419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff9241a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff924114040 con 0x7ff92410a0b0 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff924113ec0 con 0x7ff9241096f0 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.876+0000 7ff92a400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241141c0 con 0x7ff924108500 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff9241a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff9241a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48516/0 (socket says 172.21.15.57:48516) 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 -- 172.21.15.57:0/2679941348 learned_addr learned my addr 172.21.15.57:0/2679941348 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929000640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924108500 0x7ff92419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 -- 172.21.15.57:0/2679941348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924108500 msgr2=0x7ff92419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924108500 0x7ff92419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 -- 172.21.15.57:0/2679941348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9241096f0 msgr2=0x7ff92419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9241096f0 0x7ff92419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.877+0000 7ff929a00640 1 -- 172.21.15.57:0/2679941348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9241a6280 con 0x7ff92410a0b0 2024-04-13T05:25:16.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.878+0000 7ff929a00640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff9241a5c10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff91c02c710 tx=0x7ff91c035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:16.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.878+0000 7ff921600640 1 -- 172.21.15.57:0/2679941348 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c039040 con 0x7ff92410a0b0 2024-04-13T05:25:16.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.878+0000 7ff921600640 1 -- 172.21.15.57:0/2679941348 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff91c044070 con 0x7ff92410a0b0 2024-04-13T05:25:16.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.878+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9241a6510 con 0x7ff92410a0b0 2024-04-13T05:25:16.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.878+0000 7ff921600640 1 -- 172.21.15.57:0/2679941348 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0044a0 con 0x7ff92410a0b0 2024-04-13T05:25:16.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.878+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241a6a00 con 0x7ff92410a0b0 2024-04-13T05:25:16.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 Processor -- start 2024-04-13T05:25:16.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff921600640 1 -- 172.21.15.57:0/2679941348 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff91c0327c0 con 0x7ff92410a0b0 2024-04-13T05:25:16.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 -- start start 2024-04-13T05:25:16.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x56315047bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 0x563150480be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5631503e2020 con 0x56315047c0e0 2024-04-13T05:25:16.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631503886d0 con 0x56315047b6f0 2024-04-13T05:25:16.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.882+0000 7ff92e882740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631503ddfe0 con 0x563150481250 2024-04-13T05:25:16.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 0x563150480be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 0x563150480be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48524/0 (socket says 172.21.15.57:48524) 2024-04-13T05:25:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 -- 172.21.15.57:0/266997768 learned_addr learned my addr 172.21.15.57:0/266997768 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff923e00640 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff923e00640 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x56315047bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 -- 172.21.15.57:0/266997768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 msgr2=0x5631504836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504836e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 -- 172.21.15.57:0/266997768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 msgr2=0x56315047bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x56315047bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.883+0000 7ff929000640 1 -- 172.21.15.57:0/266997768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631501be7d0 con 0x56315047c0e0 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff921600640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff90c0a3bb0 0x7ff90c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff921600640 1 -- 172.21.15.57:0/2679941348 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff91c0f2e60 con 0x7ff92410a0b0 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff929000640 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 0x563150480be0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b7d0 tx=0x7ff918004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f9f46b1a55c935 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff923e00640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff90c0a3bb0 0x7ff90c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff916000640 1 -- 172.21.15.57:0/266997768 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918003dd0 con 0x56315047c0e0 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff916000640 1 -- 172.21.15.57:0/266997768 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff918015070 con 0x56315047c0e0 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.884+0000 7ff916000640 1 -- 172.21.15.57:0/266997768 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918011730 con 0x56315047c0e0 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 -- 172.21.15.57:0/266997768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 msgr2=0x563150480be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 0x563150480be0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b7d0 tx=0x7ff918004610 comp rx=0 tx=0).stop 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff923e00640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff90c0a3bb0 0x7ff90c0a6070 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x56315047c940 tx=0x7ff910005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 -- 172.21.15.57:0/266997768 shutdown_connections 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047c0e0 0x563150480be0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 --2- 172.21.15.57:0/266997768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x56315047bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.885+0000 7ff92e882740 1 -- 172.21.15.57:0/266997768 >> 172.21.15.57:0/266997768 conn(0x563150476c60 msgr2=0x563150477060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.886+0000 7ff92e882740 1 -- 172.21.15.57:0/266997768 shutdown_connections 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.886+0000 7ff92e882740 1 -- 172.21.15.57:0/266997768 wait complete. 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.886+0000 7ff92e882740 1 -- start start 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.886+0000 7ff92e882740 1 Processor -- start 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x563150485a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x563150485f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 0x56315048cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563150484200 con 0x563150481250 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56315039ba90 con 0x56315047b6f0 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563150484380 con 0x56315047c0e0 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff92e882740 10 client.0 _ll_register_callbacks cb 0x56315039b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff929a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 0x56315048cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff929a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 0x56315048cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48536/0 (socket says 172.21.15.57:48536) 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff929a00640 1 -- 172.21.15.57:0/3778259991 learned_addr learned my addr 172.21.15.57:0/3778259991 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff923e00640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x563150485f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.887+0000 7ff929000640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x563150485a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff929a00640 1 -- 172.21.15.57:0/3778259991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 msgr2=0x563150485f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff929a00640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x563150485f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff929a00640 1 -- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 msgr2=0x563150485a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff929a00640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x563150485a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff929a00640 1 -- 172.21.15.57:0/3778259991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631500f43f0 con 0x563150481250 2024-04-13T05:25:16.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff929a00640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 0x56315048cfa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0bed30 tx=0x7ff91c0bec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc23875510c4af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:16.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff90aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff90aa00640 1 -- 172.21.15.57:0/3778259991 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c004840 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff90aa00640 1 -- 172.21.15.57:0/3778259991 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff91c0bf000 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.888+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff918015070 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff90aa00640 1 -- 172.21.15.57:0/3778259991 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff91c0beda0 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 10 client.16950 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631500b40f0 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff90aa00640 1 -- 172.21.15.57:0/3778259991 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff918015070 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff90aa00640 1 -- 172.21.15.57:0/3778259991 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff91c03c030 con 0x563150481250 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 10 client.16950 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:16.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 20 client.16950 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563150491910 0x563150493dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 4 client.16950 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.889+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563150381600 con 0x563150491910 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.890+0000 7ff929000640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563150491910 0x563150493dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.891+0000 7ff929000640 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563150491910 0x563150493dd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8664df7477838a93 server_cookie=da167ac98b580af0 in_seq=0 out_seq=0 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.891+0000 7ff90aa00640 10 client.16950 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff90aa00640 1 -- 172.21.15.57:0/3778259991 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563150381600 con 0x563150491910 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff90aa00640 10 client.16950 handle_command_reply: tid=0 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 client.16950 shutdown 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 msgr2=0x56315048cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 0x56315048cfa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0bed30 tx=0x7ff91c0bec30 comp rx=0 tx=0).stop 2024-04-13T05:25:16.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 shutdown_connections 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x563150491910 0x563150493dd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563150481250 0x56315048cfa0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x563150485f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 --2- 172.21.15.57:0/3778259991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047b6f0 0x563150485a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff90aa00640 0 client.16950 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.893+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 >> 172.21.15.57:0/3778259991 conn(0x563150476c60 msgr2=0x563150475e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.894+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 shutdown_connections 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.894+0000 7ff92e882740 1 -- 172.21.15.57:0/3778259991 wait complete. 2024-04-13T05:25:16.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.894+0000 7ff92e882740 20 client.16950 trim_cache size 0 max 16384 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 Processor -- start 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 -- start start 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150483640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x56315048cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563150481250 0x563150477040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b3780 con 0x56315047b6f0 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b0ff0 con 0x563150481250 2024-04-13T05:25:16.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.913+0000 7ff92e882740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241aee40 con 0x56315047c0e0 2024-04-13T05:25:16.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150483640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150483640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48546/0 (socket says 172.21.15.57:48546) 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 -- 172.21.15.57:0/1071440568 learned_addr learned my addr 172.21.15.57:0/1071440568 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff923e00640 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x56315048cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 -- 172.21.15.57:0/1071440568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 msgr2=0x56315048cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x56315048cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929a00640 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563150481250 0x563150477040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 -- 172.21.15.57:0/1071440568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563150481250 msgr2=0x563150477040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563150481250 0x563150477040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 -- 172.21.15.57:0/1071440568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315047c970 con 0x56315047b6f0 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.914+0000 7ff929000640 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150483640 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b7d0 tx=0x7ff918013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ccc71feb18b261 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.915+0000 7ff915600640 1 -- 172.21.15.57:0/1071440568 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918003d10 con 0x56315047b6f0 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.915+0000 7ff915600640 1 -- 172.21.15.57:0/1071440568 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff918015070 con 0x56315047b6f0 2024-04-13T05:25:16.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.915+0000 7ff915600640 1 -- 172.21.15.57:0/1071440568 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918004c40 con 0x56315047b6f0 2024-04-13T05:25:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.915+0000 7ff92e882740 1 -- 172.21.15.57:0/1071440568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 msgr2=0x563150483640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.915+0000 7ff92e882740 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150483640 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b7d0 tx=0x7ff918013040 comp rx=0 tx=0).stop 2024-04-13T05:25:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 -- 172.21.15.57:0/1071440568 shutdown_connections 2024-04-13T05:25:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563150481250 0x563150477040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56315047c0e0 0x56315048cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 --2- 172.21.15.57:0/1071440568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150483640 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 -- 172.21.15.57:0/1071440568 >> 172.21.15.57:0/1071440568 conn(0x563150476c60 msgr2=0x563150480a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 -- 172.21.15.57:0/1071440568 shutdown_connections 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 -- 172.21.15.57:0/1071440568 wait complete. 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 -- start start 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.916+0000 7ff92e882740 1 Processor -- start 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150489e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047c0e0 0x56315048a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241af4b0 con 0x56315047b6f0 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b0920 con 0x56315047c0e0 2024-04-13T05:25:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241afb80 con 0x563150481250 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff92e882740 10 client.0 _ll_register_callbacks cb 0x7ff91c0ba920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150489e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150489e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48562/0 (socket says 172.21.15.57:48562) 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 -- 172.21.15.57:0/1860544902 learned_addr learned my addr 172.21.15.57:0/1860544902 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff923e00640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047c0e0 0x56315048a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 -- 172.21.15.57:0/1860544902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 msgr2=0x5631504f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929a00640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 -- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047c0e0 msgr2=0x56315048a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047c0e0 0x56315048a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff929000640 1 -- 172.21.15.57:0/1860544902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631504f6480 con 0x56315047b6f0 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.917+0000 7ff923e00640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047c0e0 0x56315048a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:16.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.918+0000 7ff929000640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150489e90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff918003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5e3e3859174fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.918+0000 7ff90aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.918+0000 7ff90aa00640 1 -- 172.21.15.57:0/1860544902 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918036020 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.918+0000 7ff90aa00640 1 -- 172.21.15.57:0/1860544902 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff9180292e0 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.918+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff918015070 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.918+0000 7ff90aa00640 1 -- 172.21.15.57:0/1860544902 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff918041050 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 10 client.16956 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5631504f7740 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff90aa00640 1 -- 172.21.15.57:0/1860544902 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff918015070 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff90aa00640 1 -- 172.21.15.57:0/1860544902 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff91801aaf0 con 0x56315047b6f0 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 10 client.16956 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 20 client.16956 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5631504f7f50 0x5631504fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 4 client.16956 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:25:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.919+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631503efa40 con 0x5631504f7f50 2024-04-13T05:25:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.920+0000 7ff923e00640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5631504f7f50 0x5631504fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.921+0000 7ff923e00640 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5631504f7f50 0x5631504fa370 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0c14f296377826 server_cookie=55f825c7551160f3 in_seq=0 out_seq=0 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.921+0000 7ff90aa00640 10 client.16956 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.921+0000 7ff90aa00640 1 -- 172.21.15.57:0/1860544902 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5631503efa40 con 0x5631504f7f50 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.921+0000 7ff90aa00640 10 client.16956 handle_command_reply: tid=0 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.921+0000 7ff92e882740 1 client.16956 shutdown 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 msgr2=0x563150489e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150489e90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff918003d10 comp rx=0 tx=0).stop 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 shutdown_connections 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5631504f7f50 0x5631504fa370 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56315047c0e0 0x56315048a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563150481250 0x5631504f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 --2- 172.21.15.57:0/1860544902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56315047b6f0 0x563150489e90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff90aa00640 0 client.16956 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:16.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 >> 172.21.15.57:0/1860544902 conn(0x563150476c60 msgr2=0x56315047d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:16.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 shutdown_connections 2024-04-13T05:25:16.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 1 -- 172.21.15.57:0/1860544902 wait complete. 2024-04-13T05:25:16.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.922+0000 7ff92e882740 20 client.16956 trim_cache size 0 max 16384 2024-04-13T05:25:16.929 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:25:16.930 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:25:16.930 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:25:16.930 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:25:16.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff90c0a3bb0 msgr2=0x7ff90c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff90c0a3bb0 0x7ff90c0a6070 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x56315047c940 tx=0x7ff910005f90 comp rx=0 tx=0).stop 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 msgr2=0x7ff9241a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff9241a5c10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff91c02c710 tx=0x7ff91c035d60 comp rx=0 tx=0).stop 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 shutdown_connections 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff90c0a3bb0 0x7ff90c0a6070 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9241096f0 0x7ff92419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff92410a0b0 0x7ff9241a5c10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 --2- 172.21.15.57:0/2679941348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924108500 0x7ff92419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:16.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 >> 172.21.15.57:0/2679941348 conn(0x7ff924077490 msgr2=0x7ff92410c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:16.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 shutdown_connections 2024-04-13T05:25:16.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:16.923+0000 7ff92a400640 1 -- 172.21.15.57:0/2679941348 wait complete. 2024-04-13T05:25:16.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:25:16.943 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:25:17.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.155+0000 7f54cac00640 1 -- 172.21.15.57:0/563574273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41051f0 msgr2=0x7f54c41075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.155+0000 7f54cac00640 1 --2- 172.21.15.57:0/563574273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41051f0 0x7f54c41075e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009900 tx=0x7f54b002bc30 comp rx=0 tx=0).stop 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 -- 172.21.15.57:0/563574273 shutdown_connections 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 --2- 172.21.15.57:0/563574273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4107b20 0x7f54c4109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 --2- 172.21.15.57:0/563574273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41051f0 0x7f54c41075e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 --2- 172.21.15.57:0/563574273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c41028c0 0x7f54c4104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 -- 172.21.15.57:0/563574273 >> 172.21.15.57:0/563574273 conn(0x7f54c40fc2d0 msgr2=0x7f54c40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 -- 172.21.15.57:0/563574273 shutdown_connections 2024-04-13T05:25:17.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.156+0000 7f54cac00640 1 -- 172.21.15.57:0/563574273 wait complete. 2024-04-13T05:25:17.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 Processor -- start 2024-04-13T05:25:17.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 -- start start 2024-04-13T05:25:17.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 0x7f54c419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c41051f0 0x7f54c419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4107b20 0x7f54c41a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f54c410d9d0 con 0x7f54c41028c0 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54c410d850 con 0x7f54c4107b20 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.157+0000 7f54cac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c410db50 con 0x7f54c41051f0 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 0x7f54c419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 0x7f54c419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48590/0 (socket says 172.21.15.57:48590) 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 -- 172.21.15.57:0/757997317 learned_addr learned my addr 172.21.15.57:0/757997317 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c8e00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c41051f0 0x7f54c419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54ca200640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4107b20 0x7f54c41a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 -- 172.21.15.57:0/757997317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c41051f0 msgr2=0x7f54c419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c41051f0 0x7f54c419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 -- 172.21.15.57:0/757997317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4107b20 msgr2=0x7f54c41a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4107b20 0x7f54c41a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.158+0000 7f54c9800640 1 -- 172.21.15.57:0/757997317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c41a1f40 con 0x7f54c41028c0 2024-04-13T05:25:17.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.159+0000 7f54c9800640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 0x7f54c419a320 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f54b800d770 tx=0x7f54b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.159+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x7f54c41028c0 2024-04-13T05:25:17.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.159+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54b80054d0 con 0x7f54c41028c0 2024-04-13T05:25:17.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.159+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8004cf0 con 0x7f54c41028c0 2024-04-13T05:25:17.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.159+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c41a2230 con 0x7f54c41028c0 2024-04-13T05:25:17.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.159+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c41a2700 con 0x7f54c41028c0 2024-04-13T05:25:17.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.160+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c4109670 con 0x7f54c41028c0 2024-04-13T05:25:17.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.163+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f54b8004e90 con 0x7f54c41028c0 2024-04-13T05:25:17.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.164+0000 7f54c2000640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54ac0a3bb0 0x7f54ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.164+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f54b80d0270 con 0x7f54c41028c0 2024-04-13T05:25:17.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.164+0000 7f54c8e00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54ac0a3bb0 0x7f54ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.165+0000 7f54c8e00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54ac0a3bb0 0x7f54ac0a6070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f54b002c240 tx=0x7f54b002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.169+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54b809c850 con 0x7f54c41028c0 2024-04-13T05:25:17.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:17 smithi057 ceph-mon[32331]: pgmap v686: 129 pgs: 129 active+clean; 462 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-04-13T05:25:17.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.415+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54c402d5e0 con 0x7f54c41028c0 2024-04-13T05:25:17.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.417+0000 7f54c2000640 1 -- 172.21.15.57:0/757997317 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11861 (secure 0 0 0) 0x7f54b809c150 con 0x7f54c41028c0 2024-04-13T05:25:17.420 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:25:17.420 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:24:46.942474+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:25:17.420 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-04-13T05:25:17.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.419+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54ac0a3bb0 msgr2=0x7f54ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.419+0000 7f54cac00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54ac0a3bb0 0x7f54ac0a6070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f54b002c240 tx=0x7f54b002c7e0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.419+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 msgr2=0x7f54c419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.419+0000 7f54cac00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 0x7f54c419a320 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f54b800d770 tx=0x7f54b800dc30 comp rx=0 tx=0).stop 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 shutdown_connections 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f54ac0a3bb0 0x7f54ac0a6070 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4107b20 0x7f54c41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c41051f0 0x7f54c419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 --2- 172.21.15.57:0/757997317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f54c41028c0 0x7f54c419a320 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 >> 172.21.15.57:0/757997317 conn(0x7f54c40fc2d0 msgr2=0x7f54c4101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 shutdown_connections 2024-04-13T05:25:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.420+0000 7f54cac00640 1 -- 172.21.15.57:0/757997317 wait complete. 2024-04-13T05:25:17.441 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:25:17.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:17 smithi083 ceph-mon[35133]: pgmap v686: 129 pgs: 129 active+clean; 462 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-04-13T05:25:17.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:17 smithi189 ceph-mon[35531]: pgmap v686: 129 pgs: 129 active+clean; 462 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2024-04-13T05:25:17.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.672+0000 7f553e000640 1 -- 172.21.15.57:0/2166578776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 msgr2=0x7f553810a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.672+0000 7f553e000640 1 --2- 172.21.15.57:0/2166578776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f553810a340 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5530009ca0 tx=0x7f553002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 -- 172.21.15.57:0/2166578776 shutdown_connections 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 --2- 172.21.15.57:0/2166578776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5538103060 0x7f55381034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 --2- 172.21.15.57:0/2166578776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f553810a340 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 --2- 172.21.15.57:0/2166578776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 0x7f55381022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 -- 172.21.15.57:0/2166578776 >> 172.21.15.57:0/2166578776 conn(0x7f55380fd460 msgr2=0x7f55380ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 -- 172.21.15.57:0/2166578776 shutdown_connections 2024-04-13T05:25:17.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.673+0000 7f553e000640 1 -- 172.21.15.57:0/2166578776 wait complete. 2024-04-13T05:25:17.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 Processor -- start 2024-04-13T05:25:17.677 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 -- start start 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 0x7f553819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5538103060 0x7f553819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f55381a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f553810da20 con 0x7f5538103a20 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f553810d8a0 con 0x7f5538103060 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.674+0000 7f553e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f553810dba0 con 0x7f5538101e80 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f55381a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f55381a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48596/0 (socket says 172.21.15.57:48596) 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 -- 172.21.15.57:0/3793145236 learned_addr learned my addr 172.21.15.57:0/3793145236 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553cc00640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 0x7f553819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f5537e00640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5538103060 0x7f553819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 -- 172.21.15.57:0/3793145236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 msgr2=0x7f553819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 0x7f553819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 -- 172.21.15.57:0/3793145236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5538103060 msgr2=0x7f553819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5538103060 0x7f553819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 -- 172.21.15.57:0/3793145236 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55381a1e40 con 0x7f5538103a20 2024-04-13T05:25:17.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553cc00640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 0x7f553819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:17.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.675+0000 7f553d600640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f55381a17d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5530032380 tx=0x7f5530032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.676+0000 7f5535600640 1 -- 172.21.15.57:0/3793145236 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530041020 con 0x7f5538103a20 2024-04-13T05:25:17.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.676+0000 7f5535600640 1 -- 172.21.15.57:0/3793145236 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5530003e20 con 0x7f5538103a20 2024-04-13T05:25:17.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.676+0000 7f5535600640 1 -- 172.21.15.57:0/3793145236 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530039040 con 0x7f5538103a20 2024-04-13T05:25:17.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.676+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55381a20d0 con 0x7f5538103a20 2024-04-13T05:25:17.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.676+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381a25c0 con 0x7f5538103a20 2024-04-13T05:25:17.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.679+0000 7f55425c5740 1 Processor -- start 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f5535600640 1 -- 172.21.15.57:0/3793145236 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5530044020 con 0x7f5538103a20 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 -- start start 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfcc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 0x5628ddfcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5628ddf2d020 con 0x5628ddfccfa0 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628dded3950 con 0x5628ddfc7440 2024-04-13T05:25:17.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.680+0000 7f55425c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5628ddf28fe0 con 0x5628ddfc7e30 2024-04-13T05:25:17.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.681+0000 7f5537e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 0x5628ddfcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.681+0000 7f553cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfcc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.681+0000 7f5537e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 0x5628ddfcf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48608/0 (socket says 172.21.15.57:48608) 2024-04-13T05:25:17.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.681+0000 7f5537e00640 1 -- 172.21.15.57:0/415792426 learned_addr learned my addr 172.21.15.57:0/415792426 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.681+0000 7f5537e00640 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.681+0000 7f5535600640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f55200a3bb0 0x7f55200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5537e00640 1 -- 172.21.15.57:0/415792426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 msgr2=0x5628ddfcc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5537e00640 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfcc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5537e00640 1 -- 172.21.15.57:0/415792426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 msgr2=0x5628ddfc7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5537e00640 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5537e00640 1 -- 172.21.15.57:0/415792426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ddc3f3f0 con 0x5628ddfccfa0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5535600640 1 -- 172.21.15.57:0/3793145236 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f55300f2af0 con 0x7f5538103a20 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f553cc00640 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfcc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f553d600640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f55200a3bb0 0x7f55200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5537e00640 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 0x5628ddfcf430 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f552400bfd0 tx=0x7f552400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb9188f031507c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.682+0000 7f5529c00640 1 -- 172.21.15.57:0/415792426 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400e950 con 0x5628ddfccfa0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f553d600640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f55200a3bb0 0x7f55200a6070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f55300be960 tx=0x7f553002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f5529c00640 1 -- 172.21.15.57:0/415792426 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5524010430 con 0x5628ddfccfa0 2024-04-13T05:25:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f5529c00640 1 -- 172.21.15.57:0/415792426 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400f620 con 0x5628ddfccfa0 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 -- 172.21.15.57:0/415792426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 msgr2=0x5628ddfcf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 0x5628ddfcf430 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f552400bfd0 tx=0x7f552400b460 comp rx=0 tx=0).stop 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 -- 172.21.15.57:0/415792426 shutdown_connections 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfccfa0 0x5628ddfcf430 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfcc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 --2- 172.21.15.57:0/415792426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.683+0000 7f55425c5740 1 -- 172.21.15.57:0/415792426 >> 172.21.15.57:0/415792426 conn(0x5628ddfc29b0 msgr2=0x5628ddfc2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.684+0000 7f55425c5740 1 -- 172.21.15.57:0/415792426 shutdown_connections 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.684+0000 7f55425c5740 1 -- 172.21.15.57:0/415792426 wait complete. 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.684+0000 7f55425c5740 1 -- start start 2024-04-13T05:25:17.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.684+0000 7f55425c5740 1 Processor -- start 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7440 0x5628ddfd19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 0x5628ddfd1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfccfa0 0x5628ddfd8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5628ddfcff50 con 0x5628ddfc7e30 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628ddee6c80 con 0x5628ddfccfa0 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5628ddfd00d0 con 0x5628ddfc7440 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f55425c5740 10 client.0 _ll_register_callbacks cb 0x5628ddee6670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 0x5628ddfd1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 0x5628ddfd1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48610/0 (socket says 172.21.15.57:48610) 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 -- 172.21.15.57:0/2019322649 learned_addr learned my addr 172.21.15.57:0/2019322649 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f5537e00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7440 0x5628ddfd19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f5537e00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfccfa0 0x5628ddfd8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 -- 172.21.15.57:0/2019322649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7440 msgr2=0x5628ddfd19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7440 0x5628ddfd19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 -- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfccfa0 msgr2=0x5628ddfd8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.685+0000 7f553cc00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfccfa0 0x5628ddfd8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f553cc00640 1 -- 172.21.15.57:0/2019322649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ddd097d0 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f5537e00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfccfa0 0x5628ddfd8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f553cc00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 0x5628ddfd1ef0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f552c00bf80 tx=0x7f552c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc13eafa8f4c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f551e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f551e000640 1 -- 172.21.15.57:0/2019322649 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c01f060 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f551e000640 1 -- 172.21.15.57:0/2019322649 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f552c010960 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5524010430 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f551e000640 1 -- 172.21.15.57:0/2019322649 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f552c026050 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.686+0000 7f55425c5740 10 client.16974 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628ddfdc370 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f551e000640 1 -- 172.21.15.57:0/2019322649 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5524010430 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f551e000640 1 -- 172.21.15.57:0/2019322649 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f552c019030 con 0x5628ddfc7e30 2024-04-13T05:25:17.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 10 client.16974 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:17.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:25:17.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 20 client.16974 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:17.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfddae0 0x5628ddfdff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 4 client.16974 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:17.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.687+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628ddecc750 con 0x5628ddfddae0 2024-04-13T05:25:17.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.688+0000 7f5537e00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfddae0 0x5628ddfdff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.689+0000 7f5537e00640 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfddae0 0x5628ddfdff40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c64492ef67066398 server_cookie=c4269078ef16c605 in_seq=0 out_seq=0 2024-04-13T05:25:17.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.689+0000 7f551e000640 10 client.16974 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:17.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f551e000640 1 -- 172.21.15.57:0/2019322649 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628ddecc750 con 0x5628ddfddae0 2024-04-13T05:25:17.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f551e000640 10 client.16974 handle_command_reply: tid=0 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 client.16974 shutdown 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 msgr2=0x5628ddfd1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 0x5628ddfd1ef0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f552c00bf80 tx=0x7f552c0128e0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 shutdown_connections 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfddae0 0x5628ddfdff40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfccfa0 0x5628ddfd8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7e30 0x5628ddfd1ef0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f551e000640 0 client.16974 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 --2- 172.21.15.57:0/2019322649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7440 0x5628ddfd19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.691+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 >> 172.21.15.57:0/2019322649 conn(0x5628ddfc29b0 msgr2=0x5628ddfc1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.692+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 shutdown_connections 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.692+0000 7f55425c5740 1 -- 172.21.15.57:0/2019322649 wait complete. 2024-04-13T05:25:17.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.692+0000 7f55425c5740 20 client.16974 trim_cache size 0 max 16384 2024-04-13T05:25:17.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 Processor -- start 2024-04-13T05:25:17.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 -- start start 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 0x5628ddfc9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfd8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f55381af410 con 0x5628ddfc29b0 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55381acc80 con 0x5628ddfc7440 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f55425c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55381aaad0 con 0x5628ddfc7e30 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f5537e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 0x5628ddfc9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f5537e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 0x5628ddfc9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48626/0 (socket says 172.21.15.57:48626) 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.707+0000 7f5537e00640 1 -- 172.21.15.57:0/1029274230 learned_addr learned my addr 172.21.15.57:0/1029274230 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f553cc00640 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5537e00640 1 -- 172.21.15.57:0/1029274230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 msgr2=0x5628ddfd8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5537e00640 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfd8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5537e00640 1 -- 172.21.15.57:0/1029274230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 msgr2=0x5628ddfc9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5537e00640 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5537e00640 1 -- 172.21.15.57:0/1029274230 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ddfc8210 con 0x5628ddfc29b0 2024-04-13T05:25:17.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5537e00640 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 0x5628ddfc9330 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5524001eb0 tx=0x7f5524005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5dba23c55855741 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.708+0000 7f5529200640 1 -- 172.21.15.57:0/1029274230 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524005110 con 0x5628ddfc29b0 2024-04-13T05:25:17.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f5529200640 1 -- 172.21.15.57:0/1029274230 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5524029770 con 0x5628ddfc29b0 2024-04-13T05:25:17.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f5529200640 1 -- 172.21.15.57:0/1029274230 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400e980 con 0x5628ddfc29b0 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 -- 172.21.15.57:0/1029274230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 msgr2=0x5628ddfc9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 0x5628ddfc9330 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5524001eb0 tx=0x7f5524005f10 comp rx=0 tx=0).stop 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 -- 172.21.15.57:0/1029274230 shutdown_connections 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7440 0x5628ddfc9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc7e30 0x5628ddfd8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 --2- 172.21.15.57:0/1029274230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc29b0 0x5628ddfc9330 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 -- 172.21.15.57:0/1029274230 >> 172.21.15.57:0/1029274230 conn(0x5628ddfc8c10 msgr2=0x5628ddfd04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 -- 172.21.15.57:0/1029274230 shutdown_connections 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.709+0000 7f55425c5740 1 -- 172.21.15.57:0/1029274230 wait complete. 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 -- start start 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 Processor -- start 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc29b0 0x5628ddfd6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 0x5628ddfd6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7e30 0x5628ddfd7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f55381ab140 con 0x5628ddfc7440 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55381ac5b0 con 0x5628ddfc7e30 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55381ab810 con 0x5628ddfc29b0 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.710+0000 7f55425c5740 10 client.0 _ll_register_callbacks cb 0x7f552c00bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 0x5628ddfd6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 0x5628ddfd6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48638/0 (socket says 172.21.15.57:48638) 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 -- 172.21.15.57:0/1878052178 learned_addr learned my addr 172.21.15.57:0/1878052178 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f5537e00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc29b0 0x5628ddfd6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 -- 172.21.15.57:0/1878052178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc29b0 msgr2=0x5628ddfd6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc29b0 0x5628ddfd6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 -- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7e30 msgr2=0x5628ddfd7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7e30 0x5628ddfd7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 -- 172.21.15.57:0/1878052178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ddfd7f50 con 0x5628ddfc7440 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f5537e00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc29b0 0x5628ddfd6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f553cc00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 0x5628ddfd6950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f552c000c00 tx=0x7f552c00bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1ca95f48f587bb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.711+0000 7f551e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f551e000640 1 -- 172.21.15.57:0/1878052178 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c02e070 con 0x5628ddfc7440 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f551e000640 1 -- 172.21.15.57:0/1878052178 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f552c010760 con 0x5628ddfc7440 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5524029770 con 0x5628ddfc7440 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f551e000640 1 -- 172.21.15.57:0/1878052178 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f552c03a050 con 0x5628ddfc7440 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f55425c5740 10 client.16980 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:17.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628ddfd9280 con 0x5628ddfc7440 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f551e000640 1 -- 172.21.15.57:0/1878052178 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5524029770 con 0x5628ddfc7440 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.712+0000 7f551e000640 1 -- 172.21.15.57:0/1878052178 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f552c019040 con 0x5628ddfc7440 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f55425c5740 10 client.16980 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f55425c5740 10 client.16980 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f55425c5740 20 client.16980 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f55425c5740 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfccfa0 0x5628de0467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f55425c5740 4 client.16980 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628ddf3aa40 con 0x5628ddfccfa0 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.713+0000 7f5537e00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfccfa0 0x5628de0467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.714+0000 7f5537e00640 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfccfa0 0x5628de0467e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46902f682df44ee server_cookie=9bb1b0beeed13f75 in_seq=0 out_seq=0 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.714+0000 7f551e000640 10 client.16980 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:17.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f551e000640 1 -- 172.21.15.57:0/1878052178 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628ddf3aa40 con 0x5628ddfccfa0 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f551e000640 10 client.16980 handle_command_reply: tid=0 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 client.16980 shutdown 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 msgr2=0x5628ddfd6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 0x5628ddfd6950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f552c000c00 tx=0x7f552c00bb00 comp rx=0 tx=0).stop 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 shutdown_connections 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5628ddfccfa0 0x5628de0467e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628ddfc7e30 0x5628ddfd7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628ddfc7440 0x5628ddfd6950 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 --2- 172.21.15.57:0/1878052178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628ddfc29b0 0x5628ddfd6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f551e000640 0 client.16980 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 >> 172.21.15.57:0/1878052178 conn(0x5628ddfc8c10 msgr2=0x5628ddfd04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 shutdown_connections 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 1 -- 172.21.15.57:0/1878052178 wait complete. 2024-04-13T05:25:17.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.715+0000 7f55425c5740 20 client.16980 trim_cache size 0 max 16384 2024-04-13T05:25:17.721 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:25:17.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f55200a3bb0 msgr2=0x7f55200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f55200a3bb0 0x7f55200a6070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f55300be960 tx=0x7f553002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 msgr2=0x7f55381a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f55381a17d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5530032380 tx=0x7f5530032820 comp rx=0 tx=0).stop 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 shutdown_connections 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f55200a3bb0 0x7f55200a6070 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5538103060 0x7f553819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5538103a20 0x7f55381a17d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 --2- 172.21.15.57:0/3793145236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5538101e80 0x7f553819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 >> 172.21.15.57:0/3793145236 conn(0x7f55380fd460 msgr2=0x7f55380741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 shutdown_connections 2024-04-13T05:25:17.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.716+0000 7f553e000640 1 -- 172.21.15.57:0/3793145236 wait complete. 2024-04-13T05:25:17.736 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 -- 172.21.15.57:0/2326236750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333810a0b0 msgr2=0x7f3338110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 --2- 172.21.15.57:0/2326236750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333810a0b0 0x7f3338110940 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3330009ca0 tx=0x7f333002c100 comp rx=0 tx=0).stop 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 -- 172.21.15.57:0/2326236750 shutdown_connections 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 --2- 172.21.15.57:0/2326236750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33381096f0 0x7f3338109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 --2- 172.21.15.57:0/2326236750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333810a0b0 0x7f3338110940 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 --2- 172.21.15.57:0/2326236750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3338108500 0x7f3338108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 -- 172.21.15.57:0/2326236750 >> 172.21.15.57:0/2326236750 conn(0x7f3338077490 msgr2=0x7f33380798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 -- 172.21.15.57:0/2326236750 shutdown_connections 2024-04-13T05:25:17.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.942+0000 7f333f000640 1 -- 172.21.15.57:0/2326236750 wait complete. 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.943+0000 7f333f000640 1 Processor -- start 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.943+0000 7f333f000640 1 -- start start 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3338108500 0x7f333819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333f000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 0x7f333819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333810a0b0 0x7f33381a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333f000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3338114010 con 0x7f33381096f0 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3338113e90 con 0x7f3338108500 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3338114190 con 0x7f333810a0b0 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 0x7f333819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 0x7f333819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48652/0 (socket says 172.21.15.57:48652) 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333d200640 1 -- 172.21.15.57:0/3166392313 learned_addr learned my addr 172.21.15.57:0/3166392313 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.944+0000 7f333dc00640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3338108500 0x7f333819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333e600640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333810a0b0 0x7f33381a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333d200640 1 -- 172.21.15.57:0/3166392313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333810a0b0 msgr2=0x7f33381a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333d200640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333810a0b0 0x7f33381a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333d200640 1 -- 172.21.15.57:0/3166392313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3338108500 msgr2=0x7f333819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333d200640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3338108500 0x7f333819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333d200640 1 -- 172.21.15.57:0/3166392313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a62a0 con 0x7f33381096f0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333e600640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333810a0b0 0x7f33381a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333d200640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 0x7f333819ebc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f332800d950 tx=0x7f332800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f3336000640 1 -- 172.21.15.57:0/3166392313 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328017070 con 0x7f33381096f0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a6590 con 0x7f33381096f0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.945+0000 7f3336000640 1 -- 172.21.15.57:0/3166392313 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3328009c30 con 0x7f33381096f0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.946+0000 7f3336000640 1 -- 172.21.15.57:0/3166392313 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332800f620 con 0x7f33381096f0 2024-04-13T05:25:17.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.946+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a6ae0 con 0x7f33381096f0 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 Processor -- start 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f3336000640 1 -- 172.21.15.57:0/3166392313 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f33280109a0 con 0x7f33381096f0 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 -- start start 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff791860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff796930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff799430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5620ff6f7020 con 0x5620ff791e30 2024-04-13T05:25:17.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5620ff69d950 con 0x5620ff791440 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.949+0000 7f334346f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5620ff6f2fe0 con 0x5620ff796fa0 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff796930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff796930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48662/0 (socket says 172.21.15.57:48662) 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 -- 172.21.15.57:0/3278606548 learned_addr learned my addr 172.21.15.57:0/3278606548 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333d200640 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff799430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333e600640 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff791860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 -- 172.21.15.57:0/3278606548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 msgr2=0x5620ff799430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff799430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 -- 172.21.15.57:0/3278606548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 msgr2=0x5620ff791860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff791860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333dc00640 1 -- 172.21.15.57:0/3278606548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ff4093f0 con 0x5620ff791e30 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.950+0000 7f333e600640 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff791860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f333dc00640 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff796930 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b7d0 tx=0x7f332c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80326a0d5ad99dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f3336000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f33200a3bb0 0x7f33200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f3326a00640 1 -- 172.21.15.57:0/3278606548 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c003dd0 con 0x5620ff791e30 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f3326a00640 1 -- 172.21.15.57:0/3278606548 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f332c015070 con 0x5620ff791e30 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f3326a00640 1 -- 172.21.15.57:0/3278606548 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0116a0 con 0x5620ff791e30 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f3336000640 1 -- 172.21.15.57:0/3166392313 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f33280d03c0 con 0x7f33381096f0 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.951+0000 7f333e600640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f33200a3bb0 0x7f33200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 -- 172.21.15.57:0/3278606548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 msgr2=0x5620ff796930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff796930 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b7d0 tx=0x7f332c004610 comp rx=0 tx=0).stop 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 -- 172.21.15.57:0/3278606548 shutdown_connections 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff799430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff796930 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 --2- 172.21.15.57:0/3278606548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff791860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f334346f740 1 -- 172.21.15.57:0/3278606548 >> 172.21.15.57:0/3278606548 conn(0x5620ff78c9b0 msgr2=0x5620ff78cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.952+0000 7f333e600640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f33200a3bb0 0x7f33200a6070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3330037040 tx=0x7f3330005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.953+0000 7f334346f740 1 -- 172.21.15.57:0/3278606548 shutdown_connections 2024-04-13T05:25:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.953+0000 7f334346f740 1 -- 172.21.15.57:0/3278606548 wait complete. 2024-04-13T05:25:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.953+0000 7f334346f740 1 -- start start 2024-04-13T05:25:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.953+0000 7f334346f740 1 Processor -- start 2024-04-13T05:25:17.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 0x5620ff7a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5620ff799f50 con 0x5620ff796fa0 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5620ff6b0c80 con 0x5620ff791440 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5620ff79a0d0 con 0x5620ff791e30 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f334346f740 10 client.0 _ll_register_callbacks cb 0x5620ff6b0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f333e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 0x5620ff7a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f333e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 0x5620ff7a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48672/0 (socket says 172.21.15.57:48672) 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.954+0000 7f333e600640 1 -- 172.21.15.57:0/1590484659 learned_addr learned my addr 172.21.15.57:0/1590484659 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333dc00640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333d200640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333e600640 1 -- 172.21.15.57:0/1590484659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791e30 msgr2=0x5620ff79bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333e600640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333e600640 1 -- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 msgr2=0x5620ff79b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333e600640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333e600640 1 -- 172.21.15.57:0/1590484659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ff4d37d0 con 0x5620ff796fa0 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333d200640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333dc00640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f333e600640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 0x5620ff7a2f30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3330036000 tx=0x7f33300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1944a8e1707f092 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.955+0000 7f331b400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f331b400640 1 -- 172.21.15.57:0/1590484659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330046050 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f331b400640 1 -- 172.21.15.57:0/1590484659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f33300045c0 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f332c015070 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f331b400640 1 -- 172.21.15.57:0/1590484659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3330039040 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f334346f740 10 client.16992 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5620ff7a6280 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.956+0000 7f331b400640 1 -- 172.21.15.57:0/1590484659 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f332c015070 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f331b400640 1 -- 172.21.15.57:0/1590484659 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f333004b440 con 0x5620ff796fa0 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f334346f740 10 client.16992 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f334346f740 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f334346f740 20 client.16992 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f334346f740 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff7a7a60 0x5620ff7a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f334346f740 4 client.16992 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620ff696750 con 0x5620ff7a7a60 2024-04-13T05:25:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.957+0000 7f333dc00640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff7a7a60 0x5620ff7a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.959+0000 7f333dc00640 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff7a7a60 0x5620ff7a9ea0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aca842b502c8a59d server_cookie=54f397a0350253a2 in_seq=0 out_seq=0 2024-04-13T05:25:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.959+0000 7f331b400640 10 client.16992 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f331b400640 1 -- 172.21.15.57:0/1590484659 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620ff696750 con 0x5620ff7a7a60 2024-04-13T05:25:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f331b400640 10 client.16992 handle_command_reply: tid=0 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 client.16992 shutdown 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 msgr2=0x5620ff7a2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 0x5620ff7a2f30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3330036000 tx=0x7f33300095c0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 shutdown_connections 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff7a7a60 0x5620ff7a9ea0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff796fa0 0x5620ff7a2f30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 --2- 172.21.15.57:0/1590484659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 >> 172.21.15.57:0/1590484659 conn(0x5620ff78c9b0 msgr2=0x5620ff78bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f331b400640 0 client.16992 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 shutdown_connections 2024-04-13T05:25:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 1 -- 172.21.15.57:0/1590484659 wait complete. 2024-04-13T05:25:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.961+0000 7f334346f740 20 client.16992 trim_cache size 0 max 16384 2024-04-13T05:25:17.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 Processor -- start 2024-04-13T05:25:17.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 -- start start 2024-04-13T05:25:17.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff7a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f33381b38a0 con 0x5620ff791440 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33381b1110 con 0x5620ff791e30 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f334346f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33381aef60 con 0x5620ff796fa0 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f333dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f333dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48680/0 (socket says 172.21.15.57:48680) 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.977+0000 7f333dc00640 1 -- 172.21.15.57:0/1356088778 learned_addr learned my addr 172.21.15.57:0/1356088778 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f333dc00640 1 -- 172.21.15.57:0/1356088778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 msgr2=0x5620ff7a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f333dc00640 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff7a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f333dc00640 1 -- 172.21.15.57:0/1356088778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791e30 msgr2=0x5620ff79bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f333dc00640 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f333dc00640 1 -- 172.21.15.57:0/1356088778 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ff796600 con 0x5620ff791440 2024-04-13T05:25:17.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f333dc00640 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x5620ff7928a0 tx=0x7f332c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fdb3de7a643358 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f3326000640 1 -- 172.21.15.57:0/1356088778 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c003e50 con 0x5620ff791440 2024-04-13T05:25:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f3326000640 1 -- 172.21.15.57:0/1356088778 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f332c015050 con 0x5620ff791440 2024-04-13T05:25:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.978+0000 7f3326000640 1 -- 172.21.15.57:0/1356088778 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c01a890 con 0x5620ff791440 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 -- 172.21.15.57:0/1356088778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 msgr2=0x5620ff79b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x5620ff7928a0 tx=0x7f332c00b410 comp rx=0 tx=0).stop 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 -- 172.21.15.57:0/1356088778 shutdown_connections 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff791e30 0x5620ff79bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff796fa0 0x5620ff7a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 --2- 172.21.15.57:0/1356088778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791440 0x5620ff79b9b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 -- 172.21.15.57:0/1356088778 >> 172.21.15.57:0/1356088778 conn(0x5620ff78c9b0 msgr2=0x5620ff787b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 -- 172.21.15.57:0/1356088778 shutdown_connections 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.979+0000 7f334346f740 1 -- 172.21.15.57:0/1356088778 wait complete. 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 -- start start 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 Processor -- start 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791440 0x5620ff79fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff7a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff796fa0 0x5620ff80c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f33381af5d0 con 0x5620ff791e30 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33381b0a40 con 0x5620ff796fa0 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33381afca0 con 0x5620ff791440 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.980+0000 7f334346f740 10 client.0 _ll_register_callbacks cb 0x7f3330032980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff7a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff7a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48686/0 (socket says 172.21.15.57:48686) 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 -- 172.21.15.57:0/3729151813 learned_addr learned my addr 172.21.15.57:0/3729151813 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333dc00640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791440 0x5620ff79fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 -- 172.21.15.57:0/3729151813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791440 msgr2=0x5620ff79fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791440 0x5620ff79fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 -- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff796fa0 msgr2=0x5620ff80c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333e600640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff796fa0 0x5620ff80c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff796fa0 0x5620ff80c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 -- 172.21.15.57:0/3729151813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ff80c550 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333e600640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff796fa0 0x5620ff80c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f333d200640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff7a03d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f33280a49e0 tx=0x7f332809bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978a8897b1f3aaed server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.981+0000 7f331b400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f331b400640 1 -- 172.21.15.57:0/3729151813 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328020040 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f331b400640 1 -- 172.21.15.57:0/3729151813 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f33280d3c50 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f332c015050 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f331b400640 1 -- 172.21.15.57:0/3729151813 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f33280d3e50 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 10 client.16998 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5620ff80d800 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f331b400640 1 -- 172.21.15.57:0/3729151813 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f332c015050 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f331b400640 1 -- 172.21.15.57:0/3729151813 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f33280a0140 con 0x5620ff791e30 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 10 client.16998 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 10 client.16998 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:25:17.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 20 client.16998 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff80e240 0x5620ff810660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 4 client.16998 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.982+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620ff704a40 con 0x5620ff80e240 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.983+0000 7f333dc00640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff80e240 0x5620ff810660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.984+0000 7f333dc00640 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff80e240 0x5620ff810660 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2a25a1cf70e4bee server_cookie=81b6cf6e8affd20a in_seq=0 out_seq=0 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.984+0000 7f331b400640 10 client.16998 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.984+0000 7f331b400640 1 -- 172.21.15.57:0/3729151813 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620ff704a40 con 0x5620ff80e240 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.984+0000 7f331b400640 10 client.16998 handle_command_reply: tid=0 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.984+0000 7f334346f740 1 client.16998 shutdown 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 msgr2=0x5620ff7a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff7a03d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f33280a49e0 tx=0x7f332809bde0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 shutdown_connections 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x5620ff80e240 0x5620ff810660 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620ff796fa0 0x5620ff80c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ff791e30 0x5620ff7a03d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 --2- 172.21.15.57:0/3729151813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620ff791440 0x5620ff79fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f331b400640 0 client.16998 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 >> 172.21.15.57:0/3729151813 conn(0x5620ff78c9b0 msgr2=0x5620ff79b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 shutdown_connections 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 1 -- 172.21.15.57:0/3729151813 wait complete. 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.985+0000 7f334346f740 20 client.16998 trim_cache size 0 max 16384 2024-04-13T05:25:17.995 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f33200a3bb0 msgr2=0x7f33200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f33200a3bb0 0x7f33200a6070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3330037040 tx=0x7f3330005da0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 msgr2=0x7f333819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 0x7f333819ebc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f332800d950 tx=0x7f332800de10 comp rx=0 tx=0).stop 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 shutdown_connections 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f33200a3bb0 0x7f33200a6070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333810a0b0 0x7f33381a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33381096f0 0x7f333819ebc0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 --2- 172.21.15.57:0/3166392313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3338108500 0x7f333819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:17.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 >> 172.21.15.57:0/3166392313 conn(0x7f3338077490 msgr2=0x7f333810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:17.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 shutdown_connections 2024-04-13T05:25:17.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:17.986+0000 7f333f000640 1 -- 172.21.15.57:0/3166392313 wait complete. 2024-04-13T05:25:18.007 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:25:18.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 -- 172.21.15.57:0/2058704905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 msgr2=0x7f11e410c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 --2- 172.21.15.57:0/2058704905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e410c540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009ca0 tx=0x7f11dc02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 -- 172.21.15.57:0/2058704905 shutdown_connections 2024-04-13T05:25:18.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 --2- 172.21.15.57:0/2058704905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e41052f0 0x7f11e4105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 --2- 172.21.15.57:0/2058704905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e410c540 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 --2- 172.21.15.57:0/2058704905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e4104100 0x7f11e4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 -- 172.21.15.57:0/2058704905 >> 172.21.15.57:0/2058704905 conn(0x7f11e40ff670 msgr2=0x7f11e4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:18.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 -- 172.21.15.57:0/2058704905 shutdown_connections 2024-04-13T05:25:18.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.209+0000 7f11ea000640 1 -- 172.21.15.57:0/2058704905 wait complete. 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.210+0000 7f11ea000640 1 Processor -- start 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.210+0000 7f11ea000640 1 -- start start 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11ea000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e4104100 0x7f11e419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11ea000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e41052f0 0x7f11e419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11ea000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e41a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11ea000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e410fc10 con 0x7f11e4105cb0 2024-04-13T05:25:18.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11ea000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e410fa90 con 0x7f11e41052f0 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11ea000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11e410fd90 con 0x7f11e4104100 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11e9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e41a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11e9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e41a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48702/0 (socket says 172.21.15.57:48702) 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11e9600640 1 -- 172.21.15.57:0/2517803571 learned_addr learned my addr 172.21.15.57:0/2517803571 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11e3e00640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e41052f0 0x7f11e419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.211+0000 7f11e8c00640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e4104100 0x7f11e419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e9600640 1 -- 172.21.15.57:0/2517803571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e4104100 msgr2=0x7f11e419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e9600640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e4104100 0x7f11e419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e9600640 1 -- 172.21.15.57:0/2517803571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e41052f0 msgr2=0x7f11e419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e9600640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e41052f0 0x7f11e419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e9600640 1 -- 172.21.15.57:0/2517803571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e41a1ef0 con 0x7f11e4105cb0 2024-04-13T05:25:18.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e9600640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e41a1880 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f11dc032380 tx=0x7f11dc032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:18.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e1600640 1 -- 172.21.15.57:0/2517803571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc041020 con 0x7f11e4105cb0 2024-04-13T05:25:18.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e1600640 1 -- 172.21.15.57:0/2517803571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11dc003e20 con 0x7f11e4105cb0 2024-04-13T05:25:18.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11e1600640 1 -- 172.21.15.57:0/2517803571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc039040 con 0x7f11e4105cb0 2024-04-13T05:25:18.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.212+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e41a2180 con 0x7f11e4105cb0 2024-04-13T05:25:18.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.213+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e41a2620 con 0x7f11e4105cb0 2024-04-13T05:25:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.216+0000 7f11ee62a740 1 Processor -- start 2024-04-13T05:25:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.216+0000 7f11e1600640 1 -- 172.21.15.57:0/2517803571 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f11dc044020 con 0x7f11e4105cb0 2024-04-13T05:25:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.216+0000 7f11ee62a740 1 -- start start 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031bf6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031bfb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031bfe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11ee62a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x565031b5c020 con 0x565031bf6e30 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11ee62a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565031b02950 con 0x565031bf6440 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11ee62a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565031b57fe0 con 0x565031bfbfa0 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031bfb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031bfb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48712/0 (socket says 172.21.15.57:48712) 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e8c00640 1 -- 172.21.15.57:0/2562090461 learned_addr learned my addr 172.21.15.57:0/2562090461 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:18.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e3e00640 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031bf6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e3e00640 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031bfe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e8c00640 1 -- 172.21.15.57:0/2562090461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 msgr2=0x565031bfe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e8c00640 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031bfe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.217+0000 7f11e8c00640 1 -- 172.21.15.57:0/2562090461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 msgr2=0x565031bf6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11e8c00640 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031bf6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11e8c00640 1 -- 172.21.15.57:0/2562090461 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503186e3f0 con 0x565031bf6e30 2024-04-13T05:25:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11e3e00640 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031bfe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:18.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11e8c00640 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031bfb930 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f11d800bc90 tx=0x7f11d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32599d3de95ee4de server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:18.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11d5c00640 1 -- 172.21.15.57:0/2562090461 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8015070 con 0x565031bf6e30 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11d5c00640 1 -- 172.21.15.57:0/2562090461 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d80035f0 con 0x565031bf6e30 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.218+0000 7f11d5c00640 1 -- 172.21.15.57:0/2562090461 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80116a0 con 0x565031bf6e30 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11e1600640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11cc0a3c30 0x7f11cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11e1600640 1 -- 172.21.15.57:0/2517803571 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f11dc0f29f0 con 0x7f11e4105cb0 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 -- 172.21.15.57:0/2562090461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 msgr2=0x565031bfb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031bfb930 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f11d800bc90 tx=0x7f11d800f730 comp rx=0 tx=0).stop 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11e3e00640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11cc0a3c30 0x7f11cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 -- 172.21.15.57:0/2562090461 shutdown_connections 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031bfe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031bfb930 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 --2- 172.21.15.57:0/2562090461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031bf6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 -- 172.21.15.57:0/2562090461 >> 172.21.15.57:0/2562090461 conn(0x565031bf19b0 msgr2=0x565031bf1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 -- 172.21.15.57:0/2562090461 shutdown_connections 2024-04-13T05:25:18.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.219+0000 7f11ee62a740 1 -- 172.21.15.57:0/2562090461 wait complete. 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.220+0000 7f11ee62a740 1 -- start start 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.220+0000 7f11e3e00640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11cc0a3c30 0x7f11cc0a60f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x565031bf7690 tx=0x7f11d0014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.220+0000 7f11ee62a740 1 Processor -- start 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031c00990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031c00ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c07f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x565031bfefa0 con 0x565031bf6e30 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565031b15c80 con 0x565031bf6440 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565031bff120 con 0x565031bfbfa0 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11ee62a740 10 client.0 _ll_register_callbacks cb 0x565031b15670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031c00ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031c00ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48726/0 (socket says 172.21.15.57:48726) 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 -- 172.21.15.57:0/850667015 learned_addr learned my addr 172.21.15.57:0/850667015 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e9600640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c07f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e8c00640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031c00990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 -- 172.21.15.57:0/850667015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 msgr2=0x565031c07f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c07f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 -- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 msgr2=0x565031c00990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031c00990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.221+0000 7f11e3e00640 1 -- 172.21.15.57:0/850667015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650319387d0 con 0x565031bf6e30 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11e3e00640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031c00ed0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f11d0003870 tx=0x7f11d0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3c4532b6d88b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ca000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ca000640 1 -- 172.21.15.57:0/850667015 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0009d30 con 0x565031bf6e30 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ca000640 1 -- 172.21.15.57:0/850667015 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f11d0019050 con 0x565031bf6e30 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d80035f0 con 0x565031bf6e30 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ca000640 1 -- 172.21.15.57:0/850667015 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11d0031050 con 0x565031bf6e30 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ee62a740 10 client.17010 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.222+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565031c0b260 con 0x565031bf6e30 2024-04-13T05:25:18.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ca000640 1 -- 172.21.15.57:0/850667015 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d80035f0 con 0x565031bf6e30 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ca000640 1 -- 172.21.15.57:0/850667015 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f11d001b070 con 0x565031bf6e30 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ee62a740 10 client.17010 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ee62a740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ee62a740 20 client.17010 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ee62a740 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c0c9f0 0x565031c0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ee62a740 4 client.17010 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.223+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565031afb750 con 0x565031c0c9f0 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.224+0000 7f11e8c00640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c0c9f0 0x565031c0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.225+0000 7f11e8c00640 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c0c9f0 0x565031c0ee50 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b1146e3aa6d46e server_cookie=acbb62fcd1d4d281 in_seq=0 out_seq=0 2024-04-13T05:25:18.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.225+0000 7f11ca000640 10 client.17010 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:18.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ca000640 1 -- 172.21.15.57:0/850667015 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565031afb750 con 0x565031c0c9f0 2024-04-13T05:25:18.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ca000640 10 client.17010 handle_command_reply: tid=0 2024-04-13T05:25:18.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 client.17010 shutdown 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 msgr2=0x565031c00ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031c00ed0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f11d0003870 tx=0x7f11d0009640 comp rx=0 tx=0).stop 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 shutdown_connections 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c0c9f0 0x565031c0ee50 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c07f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6e30 0x565031c00ed0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 --2- 172.21.15.57:0/850667015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6440 0x565031c00990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ca000640 0 client.17010 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 >> 172.21.15.57:0/850667015 conn(0x565031bf19b0 msgr2=0x565031bf0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 shutdown_connections 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 1 -- 172.21.15.57:0/850667015 wait complete. 2024-04-13T05:25:18.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.227+0000 7f11ee62a740 20 client.17010 trim_cache size 0 max 16384 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.242+0000 7f11ee62a740 1 Processor -- start 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.242+0000 7f11ee62a740 1 -- start start 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031bf8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bf6e30 0x565031c07ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bfbfa0 0x565031bf8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11ee62a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41af540 con 0x565031bf6440 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11ee62a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41acdb0 con 0x565031bfbfa0 2024-04-13T05:25:18.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11ee62a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41aac00 con 0x565031bf6e30 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031bf8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031bf8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48742/0 (socket says 172.21.15.57:48742) 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e8c00640 1 -- 172.21.15.57:0/797334173 learned_addr learned my addr 172.21.15.57:0/797334173 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e3e00640 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bf6e30 0x565031c07ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e9600640 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bfbfa0 0x565031bf8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e8c00640 1 -- 172.21.15.57:0/797334173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bf6e30 msgr2=0x565031c07ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e8c00640 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bf6e30 0x565031c07ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.243+0000 7f11e8c00640 1 -- 172.21.15.57:0/797334173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bfbfa0 msgr2=0x565031bf8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11e8c00640 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bfbfa0 0x565031bf8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11e8c00640 1 -- 172.21.15.57:0/797334173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565031bffdb0 con 0x565031bf6440 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11e9600640 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bfbfa0 0x565031bf8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11e8c00640 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031bf8330 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f11d800bc90 tx=0x7f11d800b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705396d83e5bdde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:18.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11d5200640 1 -- 172.21.15.57:0/797334173 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8015050 con 0x565031bf6440 2024-04-13T05:25:18.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11d5200640 1 -- 172.21.15.57:0/797334173 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d8005580 con 0x565031bf6440 2024-04-13T05:25:18.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.244+0000 7f11d5200640 1 -- 172.21.15.57:0/797334173 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80262a0 con 0x565031bf6440 2024-04-13T05:25:18.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 -- 172.21.15.57:0/797334173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 msgr2=0x565031bf8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031bf8330 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f11d800bc90 tx=0x7f11d800b580 comp rx=0 tx=0).stop 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 -- 172.21.15.57:0/797334173 shutdown_connections 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bfbfa0 0x565031bf8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bf6e30 0x565031c07ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 --2- 172.21.15.57:0/797334173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031bf8330 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 -- 172.21.15.57:0/797334173 >> 172.21.15.57:0/797334173 conn(0x565031bf19b0 msgr2=0x565031c00990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 -- 172.21.15.57:0/797334173 shutdown_connections 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 -- 172.21.15.57:0/797334173 wait complete. 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.245+0000 7f11ee62a740 1 -- start start 2024-04-13T05:25:18.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 Processor -- start 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031c04fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6e30 0x565031c05520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41ab270 con 0x565031bf6440 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41ac6e0 con 0x565031bf6e30 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41ab940 con 0x565031bfbfa0 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11ee62a740 10 client.0 _ll_register_callbacks cb 0x7f11d00044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11e8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031c04fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11e8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031c04fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48758/0 (socket says 172.21.15.57:48758) 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.246+0000 7f11e8c00640 1 -- 172.21.15.57:0/805557243 learned_addr learned my addr 172.21.15.57:0/805557243 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e3e00640 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6e30 0x565031c05520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e8c00640 1 -- 172.21.15.57:0/805557243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 msgr2=0x565031c064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e8c00640 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e8c00640 1 -- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6e30 msgr2=0x565031c05520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e8c00640 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6e30 0x565031c05520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e8c00640 1 -- 172.21.15.57:0/805557243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565031c06ad0 con 0x565031bf6440 2024-04-13T05:25:18.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11e8c00640 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031c04fe0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f11d8000c00 tx=0x7f11d8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77b38959796abb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:18.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11ca000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.247+0000 7f11ca000640 1 -- 172.21.15.57:0/805557243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8033060 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d8005580 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ca000640 1 -- 172.21.15.57:0/805557243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f11d8004a90 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ca000640 1 -- 172.21.15.57:0/805557243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11d8015020 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ee62a740 10 client.17016 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565031c07d40 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ca000640 1 -- 172.21.15.57:0/805557243 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d8005580 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ca000640 1 -- 172.21.15.57:0/805557243 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f11d80257f0 con 0x565031bf6440 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ee62a740 10 client.17016 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:18.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.248+0000 7f11ee62a740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.249+0000 7f11ee62a740 20 client.17016 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.249+0000 7f11ee62a740 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c73170 0x565031c75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.249+0000 7f11ee62a740 4 client.17016 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.249+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565031b69a40 con 0x565031c73170 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.249+0000 7f11e3e00640 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c73170 0x565031c75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.250+0000 7f11e3e00640 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c73170 0x565031c75590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354818994b6e62d3 server_cookie=fe25a172b8825b12 in_seq=0 out_seq=0 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.250+0000 7f11ca000640 10 client.17016 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.250+0000 7f11ca000640 1 -- 172.21.15.57:0/805557243 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565031b69a40 con 0x565031c73170 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.250+0000 7f11ca000640 10 client.17016 handle_command_reply: tid=0 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.250+0000 7f11ee62a740 1 client.17016 shutdown 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 msgr2=0x565031c04fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031c04fe0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f11d8000c00 tx=0x7f11d8005040 comp rx=0 tx=0).stop 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 shutdown_connections 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x565031c73170 0x565031c75590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565031bf6e30 0x565031c05520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565031bfbfa0 0x565031c064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 --2- 172.21.15.57:0/805557243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565031bf6440 0x565031c04fe0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ca000640 0 client.17016 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 >> 172.21.15.57:0/805557243 conn(0x565031bf19b0 msgr2=0x565031c00990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 shutdown_connections 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 1 -- 172.21.15.57:0/805557243 wait complete. 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.251+0000 7f11ee62a740 20 client.17016 trim_cache size 0 max 16384 2024-04-13T05:25:18.256 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11cc0a3c30 msgr2=0x7f11cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11cc0a3c30 0x7f11cc0a60f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x565031bf7690 tx=0x7f11d0014040 comp rx=0 tx=0).stop 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 msgr2=0x7f11e41a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e41a1880 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f11dc032380 tx=0x7f11dc032820 comp rx=0 tx=0).stop 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 shutdown_connections 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f11cc0a3c30 0x7f11cc0a60f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e41052f0 0x7f11e419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e4105cb0 0x7f11e41a1880 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 --2- 172.21.15.57:0/2517803571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e4104100 0x7f11e419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:18.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 >> 172.21.15.57:0/2517803571 conn(0x7f11e40ff670 msgr2=0x7f11e41086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:18.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 shutdown_connections 2024-04-13T05:25:18.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:18.252+0000 7f11ea000640 1 -- 172.21.15.57:0/2517803571 wait complete. 2024-04-13T05:25:18.269 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:18 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/757997317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:25:18.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:18 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/757997317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:25:18.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:18 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/757997317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:25:19.272 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:25:19.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:25:19.272 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:25:19.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:19 smithi083 ceph-mon[35133]: pgmap v687: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 -- 172.21.15.57:0/1205090401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801096f0 msgr2=0x7ff280109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 --2- 172.21.15.57:0/1205090401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801096f0 0x7ff280109b70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009900 tx=0x7ff26c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 -- 172.21.15.57:0/1205090401 shutdown_connections 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 --2- 172.21.15.57:0/1205090401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28010a0b0 0x7ff280110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 --2- 172.21.15.57:0/1205090401 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801096f0 0x7ff280109b70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 --2- 172.21.15.57:0/1205090401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff280108500 0x7ff280108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 -- 172.21.15.57:0/1205090401 >> 172.21.15.57:0/1205090401 conn(0x7ff280077490 msgr2=0x7ff2800798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 -- 172.21.15.57:0/1205090401 shutdown_connections 2024-04-13T05:25:19.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.511+0000 7ff27fe00640 1 -- 172.21.15.57:0/1205090401 wait complete. 2024-04-13T05:25:19.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.512+0000 7ff27fe00640 1 Processor -- start 2024-04-13T05:25:19.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.512+0000 7ff27fe00640 1 -- start start 2024-04-13T05:25:19.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff280108500 0x7ff28019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2801096f0 0x7ff28019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 0x7ff2801a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114040 con 0x7ff28010a0b0 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff280113ec0 con 0x7ff280108500 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801141c0 con 0x7ff2801096f0 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 0x7ff2801a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 0x7ff2801a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48766/0 (socket says 172.21.15.57:48766) 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27f400640 1 -- 172.21.15.57:0/832508458 learned_addr learned my addr 172.21.15.57:0/832508458 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27e000640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2801096f0 0x7ff28019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.513+0000 7ff27ea00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff280108500 0x7ff28019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27f400640 1 -- 172.21.15.57:0/832508458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2801096f0 msgr2=0x7ff28019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27f400640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2801096f0 0x7ff28019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27f400640 1 -- 172.21.15.57:0/832508458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff280108500 msgr2=0x7ff28019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27f400640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff280108500 0x7ff28019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27f400640 1 -- 172.21.15.57:0/832508458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a6350 con 0x7ff28010a0b0 2024-04-13T05:25:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27e000640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2801096f0 0x7ff28019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27f400640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 0x7ff2801a5ce0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff278009fd0 tx=0x7ff27800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff277400640 1 -- 172.21.15.57:0/832508458 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x7ff28010a0b0 2024-04-13T05:25:19.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff277400640 1 -- 172.21.15.57:0/832508458 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff278003e20 con 0x7ff28010a0b0 2024-04-13T05:25:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a6640 con 0x7ff28010a0b0 2024-04-13T05:25:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.514+0000 7ff277400640 1 -- 172.21.15.57:0/832508458 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278010460 con 0x7ff28010a0b0 2024-04-13T05:25:19.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.515+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a6b40 con 0x7ff28010a0b0 2024-04-13T05:25:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 Processor -- start 2024-04-13T05:25:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff277400640 1 -- 172.21.15.57:0/832508458 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff278003970 con 0x7ff28010a0b0 2024-04-13T05:25:19.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 -- start start 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986420 0x559036986840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 0x55903698b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x55903698e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5590368ecbc0 con 0x559036986e10 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559036893660 con 0x55903698bf80 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.518+0000 7ff288793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590368e8b80 con 0x559036986420 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 0x55903698b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 0x55903698b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48780/0 (socket says 172.21.15.57:48780) 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 -- 172.21.15.57:0/4215378659 learned_addr learned my addr 172.21.15.57:0/4215378659 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27e000640 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986420 0x559036986840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27e000640 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x55903698e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 -- 172.21.15.57:0/4215378659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986420 msgr2=0x559036986840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986420 0x559036986840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 -- 172.21.15.57:0/4215378659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 msgr2=0x55903698e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x55903698e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.519+0000 7ff27ea00640 1 -- 172.21.15.57:0/4215378659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590368b6220 con 0x559036986e10 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff277400640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2640a3bb0 0x7ff2640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff27ea00640 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 0x55903698b910 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff27000bc80 tx=0x7ff27000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1721df1d0529339 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff277400640 1 -- 172.21.15.57:0/832508458 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff2780d0650 con 0x7ff28010a0b0 2024-04-13T05:25:19.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff27e000640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2640a3bb0 0x7ff2640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff26be00640 1 -- 172.21.15.57:0/4215378659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270015070 con 0x559036986e10 2024-04-13T05:25:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff26be00640 1 -- 172.21.15.57:0/4215378659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff27001a910 con 0x559036986e10 2024-04-13T05:25:19.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.520+0000 7ff26be00640 1 -- 172.21.15.57:0/4215378659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2700118d0 con 0x559036986e10 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 -- 172.21.15.57:0/4215378659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 msgr2=0x55903698b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff27e000640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2640a3bb0 0x7ff2640a6070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff28019fc40 tx=0x7ff26c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 0x55903698b910 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff27000bc80 tx=0x7ff27000f730 comp rx=0 tx=0).stop 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 -- 172.21.15.57:0/4215378659 shutdown_connections 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x55903698e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986e10 0x55903698b910 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 --2- 172.21.15.57:0/4215378659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986420 0x559036986840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 -- 172.21.15.57:0/4215378659 >> 172.21.15.57:0/4215378659 conn(0x559036981990 msgr2=0x559036981d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 -- 172.21.15.57:0/4215378659 shutdown_connections 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.521+0000 7ff288793740 1 -- 172.21.15.57:0/4215378659 wait complete. 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 -- start start 2024-04-13T05:25:19.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 Processor -- start 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036990960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036990ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036997ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55903698ef30 con 0x559036986420 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590368a6a30 con 0x559036986e10 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55903698f0b0 con 0x55903698bf80 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.522+0000 7ff288793740 10 client.0 _ll_register_callbacks cb 0x5590365fec60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036990960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036990960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48782/0 (socket says 172.21.15.57:48782) 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 -- 172.21.15.57:0/1113523637 learned_addr learned my addr 172.21.15.57:0/1113523637 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27e000640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036990ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 -- 172.21.15.57:0/1113523637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 msgr2=0x559036997ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27f400640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036997ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036997ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 -- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 msgr2=0x559036990ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036990ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 -- 172.21.15.57:0/1113523637 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590366c77e0 con 0x559036986420 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27e000640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036990ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff27ea00640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036990960 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff27000ba40 tx=0x7ff27000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cda4194a6df501 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.523+0000 7ff268c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff268c00640 1 -- 172.21.15.57:0/1113523637 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270015070 con 0x559036986420 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff268c00640 1 -- 172.21.15.57:0/1113523637 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff27001acc0 con 0x559036986420 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff27001a910 con 0x559036986420 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff268c00640 1 -- 172.21.15.57:0/1113523637 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff270004020 con 0x559036986420 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff288793740 10 client.17028 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559036727f70 con 0x559036986420 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.524+0000 7ff268c00640 1 -- 172.21.15.57:0/1113523637 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff27001a910 con 0x559036986420 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff268c00640 1 -- 172.21.15.57:0/1113523637 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff27002b030 con 0x559036986420 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff288793740 10 client.17028 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff288793740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff288793740 20 client.17028 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff288793740 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55903699c480 0x55903699e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff288793740 4 client.17028 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590368fa5e0 con 0x55903699c480 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.525+0000 7ff27e000640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55903699c480 0x55903699e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.526+0000 7ff27e000640 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55903699c480 0x55903699e870 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1d4a6ab38e5c86 server_cookie=d38f3f37f4c557af in_seq=0 out_seq=0 2024-04-13T05:25:19.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.526+0000 7ff268c00640 10 client.17028 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.528+0000 7ff268c00640 1 -- 172.21.15.57:0/1113523637 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590368fa5e0 con 0x55903699c480 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.528+0000 7ff268c00640 10 client.17028 handle_command_reply: tid=0 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 client.17028 shutdown 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 msgr2=0x559036990960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036990960 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff27000ba40 tx=0x7ff27000bfa0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 shutdown_connections 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55903699c480 0x55903699e870 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036990ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036997ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 --2- 172.21.15.57:0/1113523637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036990960 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff268c00640 0 client.17028 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 >> 172.21.15.57:0/1113523637 conn(0x559036981990 msgr2=0x559036980b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 shutdown_connections 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 1 -- 172.21.15.57:0/1113523637 wait complete. 2024-04-13T05:25:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.529+0000 7ff288793740 20 client.17028 trim_cache size 0 max 16384 2024-04-13T05:25:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:19 smithi057 ceph-mon[32331]: pgmap v687: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 Processor -- start 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 -- start start 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x5590369a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986e10 0x5590369a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x5590369a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b3860 con 0x559036986420 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b10d0 con 0x55903698bf80 2024-04-13T05:25:19.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.549+0000 7ff288793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801aef20 con 0x559036986e10 2024-04-13T05:25:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x5590369a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x5590369a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48798/0 (socket says 172.21.15.57:48798) 2024-04-13T05:25:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 -- 172.21.15.57:0/1517674642 learned_addr learned my addr 172.21.15.57:0/1517674642 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27e000640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986e10 0x5590369a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27f400640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x5590369a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 -- 172.21.15.57:0/1517674642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986e10 msgr2=0x5590369a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986e10 0x5590369a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 -- 172.21.15.57:0/1517674642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 msgr2=0x5590369a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x5590369a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 -- 172.21.15.57:0/1517674642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590365bf170 con 0x559036986420 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27e000640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986e10 0x5590369a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27f400640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x5590369a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.550+0000 7ff27ea00640 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x5590369a3100 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff27000bbd0 tx=0x7ff270004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeca52000e757b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.551+0000 7ff26b400640 1 -- 172.21.15.57:0/1517674642 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270015040 con 0x559036986420 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.551+0000 7ff26b400640 1 -- 172.21.15.57:0/1517674642 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff270039070 con 0x559036986420 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.551+0000 7ff26b400640 1 -- 172.21.15.57:0/1517674642 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270004dd0 con 0x559036986420 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.551+0000 7ff288793740 1 -- 172.21.15.57:0/1517674642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 msgr2=0x5590369a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.551+0000 7ff288793740 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x5590369a3100 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff27000bbd0 tx=0x7ff270004c60 comp rx=0 tx=0).stop 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 -- 172.21.15.57:0/1517674642 shutdown_connections 2024-04-13T05:25:19.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55903698bf80 0x5590369a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559036986e10 0x5590369a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 --2- 172.21.15.57:0/1517674642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x5590369a3100 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 -- 172.21.15.57:0/1517674642 >> 172.21.15.57:0/1517674642 conn(0x559036981990 msgr2=0x559036985480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 -- 172.21.15.57:0/1517674642 shutdown_connections 2024-04-13T05:25:19.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 -- 172.21.15.57:0/1517674642 wait complete. 2024-04-13T05:25:19.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 -- start start 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.552+0000 7ff288793740 1 Processor -- start 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036994cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036995210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036996220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801af590 con 0x559036986420 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b0a00 con 0x559036986e10 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801afc60 con 0x55903698bf80 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff288793740 10 client.0 _ll_register_callbacks cb 0x7ff27000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036994cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036994cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48800/0 (socket says 172.21.15.57:48800) 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff27ea00640 1 -- 172.21.15.57:0/3110422443 learned_addr learned my addr 172.21.15.57:0/3110422443 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.553+0000 7ff27e000640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036995210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27ea00640 1 -- 172.21.15.57:0/3110422443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 msgr2=0x559036996220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27f400640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036996220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27ea00640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036996220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27ea00640 1 -- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 msgr2=0x559036995210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27ea00640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036995210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27ea00640 1 -- 172.21.15.57:0/3110422443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590368a7be0 con 0x559036986420 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27f400640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036996220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27e000640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036995210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff27ea00640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036994cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff270004200 tx=0x7ff27001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1252cedeabcbfc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff268c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff268c00640 1 -- 172.21.15.57:0/3110422443 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270041070 con 0x559036986420 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff268c00640 1 -- 172.21.15.57:0/3110422443 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff270005a70 con 0x559036986420 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.554+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff270039070 con 0x559036986420 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff268c00640 1 -- 172.21.15.57:0/3110422443 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff27004d050 con 0x559036986420 2024-04-13T05:25:19.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff288793740 10 client.17034 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5590369978f0 con 0x559036986420 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff268c00640 1 -- 172.21.15.57:0/3110422443 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff270039070 con 0x559036986420 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff268c00640 1 -- 172.21.15.57:0/3110422443 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff27002b040 con 0x559036986420 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff288793740 10 client.17034 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff288793740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.555+0000 7ff288793740 20 client.17034 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.556+0000 7ff288793740 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559036a02980 0x559036a04d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.556+0000 7ff288793740 4 client.17034 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.556+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559036987e20 con 0x559036a02980 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.556+0000 7ff27e000640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559036a02980 0x559036a04d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.557+0000 7ff27e000640 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559036a02980 0x559036a04d70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379596cb9d6875a3 server_cookie=c75f9a990dc25839 in_seq=0 out_seq=0 2024-04-13T05:25:19.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.557+0000 7ff268c00640 10 client.17034 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff268c00640 1 -- 172.21.15.57:0/3110422443 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559036987e20 con 0x559036a02980 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff268c00640 10 client.17034 handle_command_reply: tid=0 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 client.17034 shutdown 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 msgr2=0x559036994cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036994cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff270004200 tx=0x7ff27001acd0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 shutdown_connections 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559036a02980 0x559036a04d70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559036986e10 0x559036995210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55903698bf80 0x559036996220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff268c00640 0 client.17034 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 --2- 172.21.15.57:0/3110422443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559036986420 0x559036994cd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 >> 172.21.15.57:0/3110422443 conn(0x559036981990 msgr2=0x559036980300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.561+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 shutdown_connections 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff288793740 1 -- 172.21.15.57:0/3110422443 wait complete. 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff288793740 20 client.17034 trim_cache size 0 max 16384 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:25:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "dd8805b3-efbc-4383-b505-f86cef3f2a6b", 2024-04-13T05:25:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:25:19.566 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:25:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2640a3bb0 msgr2=0x7ff2640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2640a3bb0 0x7ff2640a6070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff28019fc40 tx=0x7ff26c000bc0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 msgr2=0x7ff2801a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 0x7ff2801a5ce0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff278009fd0 tx=0x7ff27800b260 comp rx=0 tx=0).stop 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 shutdown_connections 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2640a3bb0 0x7ff2640a6070 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff28010a0b0 0x7ff2801a5ce0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2801096f0 0x7ff28019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.562+0000 7ff27fe00640 1 --2- 172.21.15.57:0/832508458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff280108500 0x7ff28019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.563+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 >> 172.21.15.57:0/832508458 conn(0x7ff280077490 msgr2=0x7ff28010cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.563+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 shutdown_connections 2024-04-13T05:25:19.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.563+0000 7ff27fe00640 1 -- 172.21.15.57:0/832508458 wait complete. 2024-04-13T05:25:19.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:19 smithi189 ceph-mon[35531]: pgmap v687: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-04-13T05:25:19.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:25:19.587 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:25:19.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.825+0000 7f4344e00640 1 -- 172.21.15.57:0/4179307707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434010e520 msgr2=0x7f4340110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.825+0000 7f4344e00640 1 --2- 172.21.15.57:0/4179307707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434010e520 0x7f4340110910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4338009cd0 tx=0x7f433802c0e0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 -- 172.21.15.57:0/4179307707 shutdown_connections 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 --2- 172.21.15.57:0/4179307707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434010e520 0x7f4340110910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 --2- 172.21.15.57:0/4179307707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f4340074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 --2- 172.21.15.57:0/4179307707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4340076000 0x7f4340074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 -- 172.21.15.57:0/4179307707 >> 172.21.15.57:0/4179307707 conn(0x7f43400fd4c0 msgr2=0x7f43400ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 -- 172.21.15.57:0/4179307707 shutdown_connections 2024-04-13T05:25:19.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.826+0000 7f4344e00640 1 -- 172.21.15.57:0/4179307707 wait complete. 2024-04-13T05:25:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.827+0000 7f4344e00640 1 Processor -- start 2024-04-13T05:25:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.827+0000 7f4344e00640 1 -- start start 2024-04-13T05:25:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f4344e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f434019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f4344e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4340076000 0x7f434019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f4344e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f434010e520 0x7f43401a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f4344e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4340114020 con 0x7f4340074a20 2024-04-13T05:25:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f4344e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4340113ea0 con 0x7f4340076000 2024-04-13T05:25:19.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f4344e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43401141a0 con 0x7f434010e520 2024-04-13T05:25:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f433f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f434019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f433f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f434019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48808/0 (socket says 172.21.15.57:48808) 2024-04-13T05:25:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f433f400640 1 -- 172.21.15.57:0/2551773177 learned_addr learned my addr 172.21.15.57:0/2551773177 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.828+0000 7f433fe00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f434010e520 0x7f43401a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f433f400640 1 -- 172.21.15.57:0/2551773177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f434010e520 msgr2=0x7f43401a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f433f400640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f434010e520 0x7f43401a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f433f400640 1 -- 172.21.15.57:0/2551773177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4340076000 msgr2=0x7f434019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f433f400640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4340076000 0x7f434019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f433f400640 1 -- 172.21.15.57:0/2551773177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43401a62e0 con 0x7f4340074a20 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f433f400640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f434019e710 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f4337e00640 1 -- 172.21.15.57:0/2551773177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd80 con 0x7f4340074a20 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43401a65d0 con 0x7f4340074a20 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f4337e00640 1 -- 172.21.15.57:0/2551773177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43300108e0 con 0x7f4340074a20 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f4337e00640 1 -- 172.21.15.57:0/2551773177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330018920 con 0x7f4340074a20 2024-04-13T05:25:19.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.829+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a6b70 con 0x7f4340074a20 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.833+0000 7f43491f7740 1 Processor -- start 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.833+0000 7f43491f7740 1 -- start start 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.833+0000 7f4337e00640 1 -- 172.21.15.57:0/2551773177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4330010a90 con 0x7f4340074a20 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f43491f7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b091b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f43491f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b096be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f43491f7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f43491f7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ec7aff8020 con 0x55ec7b0916f0 2024-04-13T05:25:19.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f43491f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec7af9e6d0 con 0x55ec7b0920e0 2024-04-13T05:25:19.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f43491f7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec7aff3fe0 con 0x55ec7b097250 2024-04-13T05:25:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f433fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b091b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f433fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b091b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48814/0 (socket says 172.21.15.57:48814) 2024-04-13T05:25:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f433fe00640 1 -- 172.21.15.57:0/3148172549 learned_addr learned my addr 172.21.15.57:0/3148172549 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f433f400640 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b096be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.834+0000 7f433ea00640 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433fe00640 1 -- 172.21.15.57:0/3148172549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 msgr2=0x55ec7b0996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433fe00640 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433fe00640 1 -- 172.21.15.57:0/3148172549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 msgr2=0x55ec7b096be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433fe00640 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b096be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433fe00640 1 -- 172.21.15.57:0/3148172549 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7add47d0 con 0x55ec7b0916f0 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433f400640 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b096be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f433fe00640 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b091b10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4338032380 tx=0x7f4338005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1cbd57bf3439f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f4334c00640 1 -- 172.21.15.57:0/3148172549 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338043020 con 0x55ec7b0916f0 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.835+0000 7f4337e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43240a3bb0 0x7f43240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f4334c00640 1 -- 172.21.15.57:0/3148172549 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4338032df0 con 0x55ec7b0916f0 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f4334c00640 1 -- 172.21.15.57:0/3148172549 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338039040 con 0x55ec7b0916f0 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f4337e00640 1 -- 172.21.15.57:0/2551773177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f43300d0be0 con 0x7f4340074a20 2024-04-13T05:25:19.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f433f400640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43240a3bb0 0x7f43240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f43491f7740 1 -- 172.21.15.57:0/3148172549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 msgr2=0x55ec7b091b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f433f400640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43240a3bb0 0x7f43240a6070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f43300ce6b0 tx=0x7f4330000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.836+0000 7f43491f7740 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b091b10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4338032380 tx=0x7f4338005f80 comp rx=0 tx=0).stop 2024-04-13T05:25:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 -- 172.21.15.57:0/3148172549 shutdown_connections 2024-04-13T05:25:19.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b096be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 --2- 172.21.15.57:0/3148172549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b091b10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 -- 172.21.15.57:0/3148172549 >> 172.21.15.57:0/3148172549 conn(0x55ec7b08cc60 msgr2=0x55ec7b08d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 -- 172.21.15.57:0/3148172549 shutdown_connections 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 -- 172.21.15.57:0/3148172549 wait complete. 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.837+0000 7f43491f7740 1 -- start start 2024-04-13T05:25:19.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 Processor -- start 2024-04-13T05:25:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b09b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b09be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ec7b09a1a0 con 0x55ec7b0916f0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec7afb1a90 con 0x55ec7b0920e0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec7b09a320 con 0x55ec7b097250 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.838+0000 7f43491f7740 10 client.0 _ll_register_callbacks cb 0x55ec7afb1480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b09be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b09be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:52150/0 (socket says 172.21.15.57:52150) 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b09b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 -- 172.21.15.57:0/1886831097 learned_addr learned my addr 172.21.15.57:0/1886831097 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 -- 172.21.15.57:0/1886831097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 msgr2=0x55ec7b0a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 -- 172.21.15.57:0/1886831097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 msgr2=0x55ec7b09b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b09b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433ea00640 1 -- 172.21.15.57:0/1886831097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7ad0a3f0 con 0x55ec7b0920e0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.839+0000 7f433fe00640 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b09b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.840+0000 7f433ea00640 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b09be80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f432c010370 tx=0x7f432c010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04f639af6ed8bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.840+0000 7f4329600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.841+0000 7f4329600640 1 -- 172.21.15.57:0/1886831097 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c004230 con 0x55ec7b0920e0 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.841+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4338032df0 con 0x55ec7b0920e0 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.841+0000 7f4329600640 1 -- 172.21.15.57:0/1886831097 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f432c0043d0 con 0x55ec7b0920e0 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.841+0000 7f4329600640 1 -- 172.21.15.57:0/1886831097 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f432c012660 con 0x55ec7b0920e0 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.841+0000 7f43491f7740 10 client.36389 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.841+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ec7acca0f0 con 0x55ec7b0920e0 2024-04-13T05:25:19.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f4329600640 1 -- 172.21.15.57:0/1886831097 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4338032df0 con 0x55ec7b0920e0 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f4329600640 1 -- 172.21.15.57:0/1886831097 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f432c00f040 con 0x55ec7b0920e0 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f43491f7740 10 client.36389 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f43491f7740 10 client.36389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f43491f7740 20 client.36389 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f43491f7740 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b0a7790 0x55ec7b0a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f43491f7740 4 client.36389 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.842+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec7af97600 con 0x55ec7b0a7790 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.843+0000 7f433fe00640 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b0a7790 0x55ec7b0a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.843+0000 7f433fe00640 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b0a7790 0x55ec7b0a9bf0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b2b3914b8cbd1d server_cookie=598403cc6e1c2108 in_seq=0 out_seq=0 2024-04-13T05:25:19.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.843+0000 7f4329600640 10 client.36389 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.844+0000 7f4329600640 1 -- 172.21.15.57:0/1886831097 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec7af97600 con 0x55ec7b0a7790 2024-04-13T05:25:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.844+0000 7f4329600640 10 client.36389 handle_command_reply: tid=0 2024-04-13T05:25:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 client.36389 shutdown 2024-04-13T05:25:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 msgr2=0x55ec7b09be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b09be80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f432c010370 tx=0x7f432c010830 comp rx=0 tx=0).stop 2024-04-13T05:25:19.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 shutdown_connections 2024-04-13T05:25:19.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b0a7790 0x55ec7b0a9bf0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0920e0 0x55ec7b09be80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 --2- 172.21.15.57:0/1886831097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b09b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f4329600640 0 client.36389 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 >> 172.21.15.57:0/1886831097 conn(0x55ec7b08cc60 msgr2=0x55ec7b08be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 shutdown_connections 2024-04-13T05:25:19.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.845+0000 7f43491f7740 1 -- 172.21.15.57:0/1886831097 wait complete. 2024-04-13T05:25:19.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.846+0000 7f43491f7740 20 client.36389 trim_cache size 0 max 16384 2024-04-13T05:25:19.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.870+0000 7f43491f7740 1 Processor -- start 2024-04-13T05:25:19.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.870+0000 7f43491f7740 1 -- start start 2024-04-13T05:25:19.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f43491f7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b0916f0 0x55ec7b09a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f43491f7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 0x55ec7b0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f43491f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b097250 0x55ec7b093360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f43491f7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f43401b3870 con 0x55ec7b0920e0 2024-04-13T05:25:19.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f43491f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43401b10e0 con 0x55ec7b097250 2024-04-13T05:25:19.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f43491f7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43401aef30 con 0x55ec7b0916f0 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f433ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 0x55ec7b0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f433ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 0x55ec7b0a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48828/0 (socket says 172.21.15.57:48828) 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f433fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b097250 0x55ec7b093360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.871+0000 7f433ea00640 1 -- 172.21.15.57:0/998756742 learned_addr learned my addr 172.21.15.57:0/998756742 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433fe00640 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b0916f0 0x55ec7b09a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433ea00640 1 -- 172.21.15.57:0/998756742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b0916f0 msgr2=0x55ec7b09a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433ea00640 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b0916f0 0x55ec7b09a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433ea00640 1 -- 172.21.15.57:0/998756742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b097250 msgr2=0x55ec7b093360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433ea00640 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b097250 0x55ec7b093360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433ea00640 1 -- 172.21.15.57:0/998756742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7b092970 con 0x55ec7b0920e0 2024-04-13T05:25:19.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.872+0000 7f433ea00640 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 0x55ec7b0a2e80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f432c003c50 tx=0x7f432c010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7abc0525b134d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.873+0000 7f432be00640 1 -- 172.21.15.57:0/998756742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c02a070 con 0x55ec7b0920e0 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.873+0000 7f432be00640 1 -- 172.21.15.57:0/998756742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f432c003ec0 con 0x55ec7b0920e0 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.873+0000 7f432be00640 1 -- 172.21.15.57:0/998756742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c012b80 con 0x55ec7b0920e0 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 -- 172.21.15.57:0/998756742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 msgr2=0x55ec7b0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 0x55ec7b0a2e80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f432c003c50 tx=0x7f432c010db0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 -- 172.21.15.57:0/998756742 shutdown_connections 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b097250 0x55ec7b093360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0920e0 0x55ec7b0a2e80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 --2- 172.21.15.57:0/998756742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b0916f0 0x55ec7b09a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 -- 172.21.15.57:0/998756742 >> 172.21.15.57:0/998756742 conn(0x55ec7b08cc60 msgr2=0x55ec7b08d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 -- 172.21.15.57:0/998756742 shutdown_connections 2024-04-13T05:25:19.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.874+0000 7f43491f7740 1 -- 172.21.15.57:0/998756742 wait complete. 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 -- start start 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 Processor -- start 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b0a0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0a1600 0x55ec7b0a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f43401af5a0 con 0x55ec7b0916f0 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43401b0a10 con 0x55ec7b0a1600 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43401afc70 con 0x55ec7b097250 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.875+0000 7f43491f7740 10 client.0 _ll_register_callbacks cb 0x7f432c009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b0a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b0a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48844/0 (socket says 172.21.15.57:48844) 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 -- 172.21.15.57:0/643686203 learned_addr learned my addr 172.21.15.57:0/643686203 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:19.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433ea00640 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 -- 172.21.15.57:0/643686203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 msgr2=0x55ec7b0a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 -- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0a1600 msgr2=0x55ec7b0a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:19.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0a1600 0x55ec7b0a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.876+0000 7f433fe00640 1 -- 172.21.15.57:0/643686203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7b0a1fc0 con 0x55ec7b0916f0 2024-04-13T05:25:19.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.877+0000 7f433fe00640 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b0a0170 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f433802c120 tx=0x7f4338005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788917927eb28bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:19.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.877+0000 7f4329600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:19.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.877+0000 7f4329600640 1 -- 172.21.15.57:0/643686203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338043040 con 0x55ec7b0916f0 2024-04-13T05:25:19.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.877+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432c003ec0 con 0x55ec7b0916f0 2024-04-13T05:25:19.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.877+0000 7f4329600640 1 -- 172.21.15.57:0/643686203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4338050bc0 con 0x55ec7b0916f0 2024-04-13T05:25:19.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.877+0000 7f4329600640 1 -- 172.21.15.57:0/643686203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4338039040 con 0x55ec7b0916f0 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f43491f7740 10 client.17052 fetch_fsmap learned FSMap version 31 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ec7b10dd30 con 0x55ec7b0916f0 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f4329600640 1 -- 172.21.15.57:0/643686203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f432c003ec0 con 0x55ec7b0916f0 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f4329600640 1 -- 172.21.15.57:0/643686203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4338032790 con 0x55ec7b0916f0 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f43491f7740 10 client.17052 fetch_fsmap finished waiting for FSMap version 31 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f43491f7740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f43491f7740 20 client.17052 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.878+0000 7f43491f7740 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b10e2e0 0x55ec7b110700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.879+0000 7f43491f7740 4 client.17052 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.879+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec7b005a40 con 0x55ec7b10e2e0 2024-04-13T05:25:19.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.879+0000 7f433ea00640 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b10e2e0 0x55ec7b110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:19.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.881+0000 7f433ea00640 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b10e2e0 0x55ec7b110700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b75ae5a48578db server_cookie=7fb6ee8086eebf8a in_seq=0 out_seq=0 2024-04-13T05:25:19.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.881+0000 7f4329600640 10 client.17052 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f4329600640 1 -- 172.21.15.57:0/643686203 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55ec7b005a40 con 0x55ec7b10e2e0 2024-04-13T05:25:19.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f4329600640 10 client.17052 handle_command_reply: tid=0 2024-04-13T05:25:19.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 client.17052 shutdown 2024-04-13T05:25:19.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 msgr2=0x55ec7b0a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b0a0170 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f433802c120 tx=0x7f4338005c60 comp rx=0 tx=0).stop 2024-04-13T05:25:19.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 shutdown_connections 2024-04-13T05:25:19.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55ec7b10e2e0 0x55ec7b110700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec7b0a1600 0x55ec7b0a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec7b097250 0x55ec7b0a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 --2- 172.21.15.57:0/643686203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ec7b0916f0 0x55ec7b0a0170 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f4329600640 0 client.17052 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 >> 172.21.15.57:0/643686203 conn(0x55ec7b08cc60 msgr2=0x55ec7b093360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 shutdown_connections 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 1 -- 172.21.15.57:0/643686203 wait complete. 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.882+0000 7f43491f7740 20 client.17052 trim_cache size 0 max 16384 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1064 inodes in the stack)", 2024-04-13T05:25:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "dd8805b3-efbc-4383-b505-f86cef3f2a6b": { 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "dd8805b3-efbc-4383-b505-f86cef3f2a6b", 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:25:19.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43240a3bb0 msgr2=0x7f43240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43240a3bb0 0x7f43240a6070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f43300ce6b0 tx=0x7f4330000bc0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 msgr2=0x7f434019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f434019e710 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 shutdown_connections 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f43240a3bb0 0x7f43240a6070 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4340076000 0x7f434019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f434010e520 0x7f43401a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 --2- 172.21.15.57:0/2551773177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4340074a20 0x7f434019e710 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.883+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 >> 172.21.15.57:0/2551773177 conn(0x7f43400fd4c0 msgr2=0x7f43401017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.884+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 shutdown_connections 2024-04-13T05:25:19.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:19.884+0000 7f4344e00640 1 -- 172.21.15.57:0/2551773177 wait complete. 2024-04-13T05:25:19.906 INFO:tasks.cephfs.filesystem:scrub status for tag:dd8805b3-efbc-4383-b505-f86cef3f2a6b - {'path': '/', 'tag': 'dd8805b3-efbc-4383-b505-f86cef3f2a6b', 'options': 'recursive,force'} 2024-04-13T05:25:20.002 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:25:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:25:19.661+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000021b [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0/ auth{0=1,2=1} fragtree_t(*^3) v47842 ap=2 f(v342 m2024-04-13T05:21:28.256974+0000 273=273+0) n(v10 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1) (inest sync->mix g=0,2) caps={15447=pAsLsXs/-@1608} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559b052eb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x2000000021b:[<0x2000000021a/blog-0 v47842>,<0x10000000208/blogtest_in v11726>,<0x10000000201/src v2821>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x2000000021b:[<0x2000000021a/blog-0 v47842>,<0x10000000208/blogtest_in v15445>,<0x10000000201/src v2837>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:21:28.196434+0000 274=274+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1)","memory_value.dirstat":"f(v342 m2024-04-13T05:21:28.256974+0000 273=273+0)","memory_value.rstat":"n(v10 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:25:20.002 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:25:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:25:19.669+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001a4d [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44/ auth{2=1} v81253 ap=2 DIRTYPARENT f(v12 m2024-04-13T05:23:31.482085+0000 85=85+0) n(v5 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@357} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06149180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000001a4d:[<0x2000000021a/blog-44 v81253>,<0x10000000208/blogtest_in v15445>,<0x10000000201/src v2837>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:23:31.370751+0000 86=86+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1)","memory_value.dirstat":"f(v12 m2024-04-13T05:23:31.482085+0000 85=85+0)","memory_value.rstat":"n(v5 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-13T05:25:20.002 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:25:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:25:19.707+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001100 [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/ auth{2=1} v73177 ap=2 f(v16 m2024-04-13T05:22:48.044991+0000 79=79+0) n(v5 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@198} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06366580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000001100:[<0x2000000021a/blog-25 v73177>,<0x10000000208/blogtest_in v13864>,<0x10000000201/src v2827>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x20000001100:[<0x2000000021a/blog-25 v73177>,<0x10000000208/blogtest_in v15445>,<0x10000000201/src v2837>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:22:47.912987+0000 80=80+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1)","memory_value.dirstat":"f(v16 m2024-04-13T05:22:48.044991+0000 79=79+0)","memory_value.rstat":"n(v5 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:25:20.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:20 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:20.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:20 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:20.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:20 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: pgmap v688: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-13T05:25:21.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:21 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:25:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: pgmap v688: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-13T05:25:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:21 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:25:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:21.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: pgmap v688: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-13T05:25:21.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:21 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:23.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:23 smithi083 ceph-mon[35133]: pgmap v689: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 929 KiB/s wr, 119 op/s 2024-04-13T05:25:23.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:23 smithi057 ceph-mon[32331]: pgmap v689: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 929 KiB/s wr, 119 op/s 2024-04-13T05:25:23.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:23 smithi189 ceph-mon[35531]: pgmap v689: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 929 KiB/s wr, 119 op/s 2024-04-13T05:25:24.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:24 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:24 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:24.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:24 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:25.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:25 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:25.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:25 smithi083 ceph-mon[35133]: pgmap v690: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 897 KiB/s wr, 195 op/s 2024-04-13T05:25:25.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:25 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:25.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:25 smithi057 ceph-mon[32331]: pgmap v690: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 897 KiB/s wr, 195 op/s 2024-04-13T05:25:25.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:25 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:25.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:25 smithi189 ceph-mon[35531]: pgmap v690: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 897 KiB/s wr, 195 op/s 2024-04-13T05:25:27.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:27 smithi083 ceph-mon[35133]: pgmap v691: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 232 KiB/s wr, 198 op/s 2024-04-13T05:25:27.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:27 smithi057 ceph-mon[32331]: pgmap v691: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 232 KiB/s wr, 198 op/s 2024-04-13T05:25:27.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:27 smithi189 ceph-mon[35531]: pgmap v691: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 232 KiB/s wr, 198 op/s 2024-04-13T05:25:29.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:29 smithi083 ceph-mon[35133]: pgmap v692: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 85 KiB/s wr, 178 op/s 2024-04-13T05:25:29.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:29 smithi057 ceph-mon[32331]: pgmap v692: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 85 KiB/s wr, 178 op/s 2024-04-13T05:25:29.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:29 smithi189 ceph-mon[35531]: pgmap v692: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 85 KiB/s wr, 178 op/s 2024-04-13T05:25:31.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:31 smithi083 ceph-mon[35133]: pgmap v693: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 107 KiB/s wr, 183 op/s 2024-04-13T05:25:31.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:31 smithi057 ceph-mon[32331]: pgmap v693: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 107 KiB/s wr, 183 op/s 2024-04-13T05:25:31.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:31 smithi189 ceph-mon[35531]: pgmap v693: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 107 KiB/s wr, 183 op/s 2024-04-13T05:25:33.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:33 smithi083 ceph-mon[35133]: pgmap v694: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 44 KiB/s wr, 168 op/s 2024-04-13T05:25:33.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:33 smithi057 ceph-mon[32331]: pgmap v694: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 44 KiB/s wr, 168 op/s 2024-04-13T05:25:33.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:33 smithi189 ceph-mon[35531]: pgmap v694: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 44 KiB/s wr, 168 op/s 2024-04-13T05:25:35.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:35 smithi083 ceph-mon[35133]: pgmap v695: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 44 KiB/s wr, 145 op/s 2024-04-13T05:25:35.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:35 smithi057 ceph-mon[32331]: pgmap v695: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 44 KiB/s wr, 145 op/s 2024-04-13T05:25:35.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:35 smithi189 ceph-mon[35531]: pgmap v695: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 44 KiB/s wr, 145 op/s 2024-04-13T05:25:37.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:37 smithi083 ceph-mon[35133]: pgmap v696: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 36 KiB/s wr, 64 op/s 2024-04-13T05:25:37.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:37 smithi057 ceph-mon[32331]: pgmap v696: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 36 KiB/s wr, 64 op/s 2024-04-13T05:25:37.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:37 smithi189 ceph-mon[35531]: pgmap v696: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 36 KiB/s wr, 64 op/s 2024-04-13T05:25:38.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:38 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:25:38.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:38 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:25:38.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:38 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:25:38.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:38 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:25:38.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:38 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:25:38.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:38 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:25:38.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:38 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:25:38.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:38 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:25:38.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:38 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:25:39.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:39 smithi083 ceph-mon[35133]: pgmap v697: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-13T05:25:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:39 smithi057 ceph-mon[32331]: pgmap v697: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-13T05:25:39.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:39 smithi189 ceph-mon[35531]: pgmap v697: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-13T05:25:41.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:41 smithi057 ceph-mon[32331]: pgmap v698: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-13T05:25:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:41 smithi189 ceph-mon[35531]: pgmap v698: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-13T05:25:41.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:41 smithi083 ceph-mon[35133]: pgmap v698: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-13T05:25:41.975 INFO:tasks.workunit.client.0.smithi057.stdout: 2024-04-13T05:25:41.975 INFO:tasks.workunit.client.0.smithi057.stdout:Final score for writes: 59 2024-04-13T05:25:41.975 INFO:tasks.workunit.client.0.smithi057.stdout:Final score for reads : 5360 2024-04-13T05:25:41.975 INFO:tasks.workunit.client.0.smithi057.stdout: 2024-04-13T05:25:41.978 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-13T05:25:41.978 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-13T05:25:43.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:43 smithi057 ceph-mon[32331]: pgmap v699: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-13T05:25:43.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:43 smithi189 ceph-mon[35531]: pgmap v699: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-13T05:25:43.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:43 smithi083 ceph-mon[35133]: pgmap v699: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-13T05:25:45.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:45 smithi057 ceph-mon[32331]: pgmap v700: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-13T05:25:45.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:45 smithi189 ceph-mon[35531]: pgmap v700: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-13T05:25:45.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:45 smithi083 ceph-mon[35133]: pgmap v700: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-13T05:25:47.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:47 smithi057 ceph-mon[32331]: pgmap v701: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-13T05:25:47.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:47 smithi189 ceph-mon[35531]: pgmap v701: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-13T05:25:47.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:47 smithi083 ceph-mon[35133]: pgmap v701: 129 pgs: 129 active+clean; 463 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-13T05:25:49.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:49 smithi057 ceph-mon[32331]: pgmap v702: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-13T05:25:49.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:49 smithi189 ceph-mon[35531]: pgmap v702: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-13T05:25:49.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:49 smithi083 ceph-mon[35133]: pgmap v702: 129 pgs: 129 active+clean; 462 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-13T05:25:49.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:25:49.907 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 -- 172.21.15.57:0/861701206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c104100 msgr2=0x7f0e5c104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 --2- 172.21.15.57:0/861701206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c104100 0x7f0e5c104520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009900 tx=0x7f0e4c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 -- 172.21.15.57:0/861701206 shutdown_connections 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 --2- 172.21.15.57:0/861701206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 --2- 172.21.15.57:0/861701206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e5c105cb0 0x7f0e5c10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 --2- 172.21.15.57:0/861701206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c104100 0x7f0e5c104520 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 -- 172.21.15.57:0/861701206 >> 172.21.15.57:0/861701206 conn(0x7f0e5c0ff670 msgr2=0x7f0e5c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 -- 172.21.15.57:0/861701206 shutdown_connections 2024-04-13T05:25:50.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.152+0000 7f0e61000640 1 -- 172.21.15.57:0/861701206 wait complete. 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.153+0000 7f0e61000640 1 Processor -- start 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.153+0000 7f0e61000640 1 -- start start 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e61000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e5c104100 0x7f0e5c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e61000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e61000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c105cb0 0x7f0e5c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e61000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c10fc90 con 0x7f0e5c1052f0 2024-04-13T05:25:50.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e61000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c10fb10 con 0x7f0e5c105cb0 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e61000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c10fe10 con 0x7f0e5c104100 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e5aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e5aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56866/0 (socket says 172.21.15.57:56866) 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e5aa00640 1 -- 172.21.15.57:0/148993395 learned_addr learned my addr 172.21.15.57:0/148993395 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.154+0000 7f0e5be00640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c105cb0 0x7f0e5c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5b400640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e5c104100 0x7f0e5c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5aa00640 1 -- 172.21.15.57:0/148993395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e5c104100 msgr2=0x7f0e5c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e5c104100 0x7f0e5c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5aa00640 1 -- 172.21.15.57:0/148993395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c105cb0 msgr2=0x7f0e5c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c105cb0 0x7f0e5c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5aa00640 1 -- 172.21.15.57:0/148993395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c1a1f50 con 0x7f0e5c1052f0 2024-04-13T05:25:50.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c19a870 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e9f0 tx=0x7f0e4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e53e00640 1 -- 172.21.15.57:0/148993395 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4800cd80 con 0x7f0e5c1052f0 2024-04-13T05:25:50.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e53e00640 1 -- 172.21.15.57:0/148993395 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e480108e0 con 0x7f0e5c1052f0 2024-04-13T05:25:50.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e53e00640 1 -- 172.21.15.57:0/148993395 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48018920 con 0x7f0e5c1052f0 2024-04-13T05:25:50.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.155+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c1a2240 con 0x7f0e5c1052f0 2024-04-13T05:25:50.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.156+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c1a2790 con 0x7f0e5c1052f0 2024-04-13T05:25:50.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.160+0000 7f0e6556f740 1 Processor -- start 2024-04-13T05:25:50.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.160+0000 7f0e6556f740 1 -- start start 2024-04-13T05:25:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.160+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 0x559e77766b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.160+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e7776bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e7776c250 0x559e7776e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e6556f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559e776cd020 con 0x559e777666f0 2024-04-13T05:25:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e6556f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e776736d0 con 0x559e7776c250 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e6556f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e776c8fe0 con 0x559e777670e0 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 0x559e77766b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 0x559e77766b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56878/0 (socket says 172.21.15.57:56878) 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 -- 172.21.15.57:0/3272376947 learned_addr learned my addr 172.21.15.57:0/3272376947 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e53e00640 1 -- 172.21.15.57:0/148993395 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0e48010a90 con 0x7f0e5c1052f0 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5b400640 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e7776bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 -- 172.21.15.57:0/3272376947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 msgr2=0x559e7776bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e7776bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 -- 172.21.15.57:0/3272376947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e7776c250 msgr2=0x559e7776e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e7776c250 0x559e7776e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.161+0000 7f0e5be00640 1 -- 172.21.15.57:0/3272376947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e774a97d0 con 0x559e777666f0 2024-04-13T05:25:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.162+0000 7f0e5be00640 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 0x559e77766b10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009ee0 tx=0x7f0e54010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bb4ab55561e3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.162+0000 7f0e50c00640 1 -- 172.21.15.57:0/3272376947 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54016070 con 0x559e777666f0 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.162+0000 7f0e50c00640 1 -- 172.21.15.57:0/3272376947 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e54004d20 con 0x559e777666f0 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.162+0000 7f0e50c00640 1 -- 172.21.15.57:0/3272376947 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54012720 con 0x559e777666f0 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e53e00640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0e400a3bb0 0x7f0e400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e53e00640 1 -- 172.21.15.57:0/148993395 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0e480d0c40 con 0x7f0e5c1052f0 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 -- 172.21.15.57:0/3272376947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 msgr2=0x559e77766b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 0x559e77766b10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009ee0 tx=0x7f0e54010740 comp rx=0 tx=0).stop 2024-04-13T05:25:50.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e5b400640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0e400a3bb0 0x7f0e400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 -- 172.21.15.57:0/3272376947 shutdown_connections 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e7776c250 0x559e7776e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e7776bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3272376947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777666f0 0x559e77766b10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 -- 172.21.15.57:0/3272376947 >> 172.21.15.57:0/3272376947 conn(0x559e77761c60 msgr2=0x559e77762060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 -- 172.21.15.57:0/3272376947 shutdown_connections 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.163+0000 7f0e6556f740 1 -- 172.21.15.57:0/3272376947 wait complete. 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.164+0000 7f0e6556f740 1 -- start start 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.164+0000 7f0e5b400640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0e400a3bb0 0x7f0e400a6070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c02c240 tx=0x7f0e4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.164+0000 7f0e6556f740 1 Processor -- start 2024-04-13T05:25:50.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e777709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e77770f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e7776c250 0x559e77777f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559e7776f200 con 0x559e777670e0 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e77686a90 con 0x559e777666f0 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e7776f380 con 0x559e7776c250 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e6556f740 10 client.0 _ll_register_callbacks cb 0x559e77686480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e77770f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e77770f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56892/0 (socket says 172.21.15.57:56892) 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 -- 172.21.15.57:0/1772706852 learned_addr learned my addr 172.21.15.57:0/1772706852 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5b400640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e777709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5be00640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e7776c250 0x559e77777f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 -- 172.21.15.57:0/1772706852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e7776c250 msgr2=0x559e77777f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e7776c250 0x559e77777f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 -- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 msgr2=0x559e777709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e777709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.165+0000 7f0e5aa00640 1 -- 172.21.15.57:0/1772706852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e773df3f0 con 0x559e777670e0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e77770f30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0e480994f0 tx=0x7f0e4809c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ca2a791e2fe27c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e45600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e45600640 1 -- 172.21.15.57:0/1772706852 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4809cb20 con 0x559e777670e0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e45600640 1 -- 172.21.15.57:0/1772706852 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0e48010430 con 0x559e777670e0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e54004d20 con 0x559e777670e0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e45600640 1 -- 172.21.15.57:0/1772706852 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e480a0ed0 con 0x559e777670e0 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e6556f740 10 client.17064 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:50.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.166+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e7739f0f0 con 0x559e777670e0 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e45600640 1 -- 172.21.15.57:0/1772706852 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e54004d20 con 0x559e777670e0 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e45600640 1 -- 172.21.15.57:0/1772706852 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f0e480bbce0 con 0x559e777670e0 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e6556f740 10 client.17064 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e6556f740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e6556f740 20 client.17064 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e6556f740 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7777c8a0 0x559e7777ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e6556f740 4 client.17064 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.167+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e7766c600 con 0x559e7777c8a0 2024-04-13T05:25:50.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.168+0000 7f0e5b400640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7777c8a0 0x559e7777ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.169+0000 7f0e5b400640 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7777c8a0 0x559e7777ece0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1979e1b63bbbfe0a server_cookie=5ad17e577436af8c in_seq=0 out_seq=0 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.169+0000 7f0e45600640 10 client.17064 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e45600640 1 -- 172.21.15.57:0/1772706852 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e7766c600 con 0x559e7777c8a0 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e45600640 10 client.17064 handle_command_reply: tid=0 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 client.17064 shutdown 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 msgr2=0x559e77770f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e77770f30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0e480994f0 tx=0x7f0e4809c750 comp rx=0 tx=0).stop 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 shutdown_connections 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7777c8a0 0x559e7777ece0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e7776c250 0x559e77777f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e77770f30 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 --2- 172.21.15.57:0/1772706852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e777709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 >> 172.21.15.57:0/1772706852 conn(0x559e77761c60 msgr2=0x559e77760e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e45600640 0 client.17064 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:50.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 shutdown_connections 2024-04-13T05:25:50.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 1 -- 172.21.15.57:0/1772706852 wait complete. 2024-04-13T05:25:50.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.171+0000 7f0e6556f740 20 client.17064 trim_cache size 0 max 16384 2024-04-13T05:25:50.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.192+0000 7f0e6556f740 1 Processor -- start 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.192+0000 7f0e6556f740 1 -- start start 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77761c60 0x559e777655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77777f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e777835a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e6556f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1af5e0 con 0x559e777670e0 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e6556f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ace20 con 0x559e777666f0 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e6556f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1aac70 con 0x559e77761c60 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e777835a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e777835a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56902/0 (socket says 172.21.15.57:56902) 2024-04-13T05:25:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e5be00640 1 -- 172.21.15.57:0/3187571348 learned_addr learned my addr 172.21.15.57:0/3187571348 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77777f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e5be00640 1 -- 172.21.15.57:0/3187571348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77761c60 msgr2=0x559e777655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.193+0000 7f0e5be00640 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77761c60 0x559e777655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e5be00640 1 -- 172.21.15.57:0/3187571348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 msgr2=0x559e77777f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e5be00640 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77777f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e5be00640 1 -- 172.21.15.57:0/3187571348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7775d4e0 con 0x559e777670e0 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e5be00640 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e777835a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0e54005f40 tx=0x7f0e54005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e88f696c9fcb77 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e47e00640 1 -- 172.21.15.57:0/3187571348 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54028070 con 0x559e777670e0 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e47e00640 1 -- 172.21.15.57:0/3187571348 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e54005040 con 0x559e777670e0 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.194+0000 7f0e47e00640 1 -- 172.21.15.57:0/3187571348 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54016070 con 0x559e777670e0 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 -- 172.21.15.57:0/3187571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 msgr2=0x559e777835a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e777835a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0e54005f40 tx=0x7f0e54005c80 comp rx=0 tx=0).stop 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 -- 172.21.15.57:0/3187571348 shutdown_connections 2024-04-13T05:25:50.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77777f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e777670e0 0x559e777835a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3187571348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e77761c60 0x559e777655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 -- 172.21.15.57:0/3187571348 >> 172.21.15.57:0/3187571348 conn(0x559e7776bd90 msgr2=0x559e77764ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 -- 172.21.15.57:0/3187571348 shutdown_connections 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.195+0000 7f0e6556f740 1 -- 172.21.15.57:0/3187571348 wait complete. 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 -- start start 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 Processor -- start 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 0x559e77775dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77776310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e77777320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ab2e0 con 0x559e77761c60 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ac750 con 0x559e777666f0 2024-04-13T05:25:50.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ab9b0 con 0x559e777670e0 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e6556f740 10 client.0 _ll_register_callbacks cb 0x7f0e4809c980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.196+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 0x559e77775dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 0x559e77775dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56908/0 (socket says 172.21.15.57:56908) 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 -- 172.21.15.57:0/3965805024 learned_addr learned my addr 172.21.15.57:0/3965805024 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 -- 172.21.15.57:0/3965805024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 msgr2=0x559e77777320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e77777320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77776310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 -- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 msgr2=0x559e77776310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77776310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 -- 172.21.15.57:0/3965805024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e77777910 con 0x559e77761c60 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e5b400640 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 0x559e77775dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c02ce10 tx=0x7f0e4c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533bb4e51cd4d52b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e45600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.197+0000 7f0e45600640 1 -- 172.21.15.57:0/3965805024 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0042d0 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e45600640 1 -- 172.21.15.57:0/3965805024 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0e4c004470 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e54005040 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e45600640 1 -- 172.21.15.57:0/3965805024 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c062050 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e6556f740 10 client.17070 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e777e3600 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e45600640 1 -- 172.21.15.57:0/3965805024 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e54005040 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.198+0000 7f0e45600640 1 -- 172.21.15.57:0/3965805024 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f0e4c0046c0 con 0x559e77761c60 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e6556f740 10 client.17070 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e6556f740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e6556f740 20 client.17070 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7776c250 0x559e777e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e6556f740 4 client.17070 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:25:50.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e776daa40 con 0x559e7776c250 2024-04-13T05:25:50.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.199+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7776c250 0x559e777e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.200+0000 7f0e5aa00640 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7776c250 0x559e777e6080 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2738dc1572657e server_cookie=e9dd83f174907a55 in_seq=0 out_seq=0 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.200+0000 7f0e45600640 10 client.17070 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.200+0000 7f0e45600640 1 -- 172.21.15.57:0/3965805024 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e776daa40 con 0x559e7776c250 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.200+0000 7f0e45600640 10 client.17070 handle_command_reply: tid=0 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.200+0000 7f0e6556f740 1 client.17070 shutdown 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 msgr2=0x559e77775dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 0x559e77775dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c02ce10 tx=0x7f0e4c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 shutdown_connections 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559e7776c250 0x559e777e6080 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e777666f0 0x559e77776310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e777670e0 0x559e77777320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 --2- 172.21.15.57:0/3965805024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e77761c60 0x559e77775dd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 >> 172.21.15.57:0/3965805024 conn(0x559e7776bd90 msgr2=0x559e77765750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e45600640 0 client.17070 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 shutdown_connections 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 1 -- 172.21.15.57:0/3965805024 wait complete. 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.201+0000 7f0e6556f740 20 client.17070 trim_cache size 0 max 16384 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:25:50.208 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0e400a3bb0 msgr2=0x7f0e400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0e400a3bb0 0x7f0e400a6070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c02c240 tx=0x7f0e4c005fd0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 msgr2=0x7f0e5c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c19a870 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e9f0 tx=0x7f0e4800eeb0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 shutdown_connections 2024-04-13T05:25:50.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0e400a3bb0 0x7f0e400a6070 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c105cb0 0x7f0e5c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e5c1052f0 0x7f0e5c19a870 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 --2- 172.21.15.57:0/148993395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e5c104100 0x7f0e5c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 >> 172.21.15.57:0/148993395 conn(0x7f0e5c0ff670 msgr2=0x7f0e5c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 shutdown_connections 2024-04-13T05:25:50.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.202+0000 7f0e61000640 1 -- 172.21.15.57:0/148993395 wait complete. 2024-04-13T05:25:50.224 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:25:50.225 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 -- 172.21.15.57:0/2429569781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109ff0 msgr2=0x7f961c110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 --2- 172.21.15.57:0/2429569781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109ff0 0x7f961c110880 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9614009ca0 tx=0x7f961402c100 comp rx=0 tx=0).stop 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 -- 172.21.15.57:0/2429569781 shutdown_connections 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 --2- 172.21.15.57:0/2429569781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109630 0x7f961c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 --2- 172.21.15.57:0/2429569781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c109ff0 0x7f961c110880 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 --2- 172.21.15.57:0/2429569781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c108440 0x7f961c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 -- 172.21.15.57:0/2429569781 >> 172.21.15.57:0/2429569781 conn(0x7f961c077400 msgr2=0x7f961c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 -- 172.21.15.57:0/2429569781 shutdown_connections 2024-04-13T05:25:50.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.443+0000 7f9621400640 1 -- 172.21.15.57:0/2429569781 wait complete. 2024-04-13T05:25:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.444+0000 7f9621400640 1 Processor -- start 2024-04-13T05:25:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 -- start start 2024-04-13T05:25:50.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 0x7f961c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c109630 0x7f961c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109ff0 0x7f961c1a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f961c113f80 con 0x7f961c108440 2024-04-13T05:25:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f961c113e00 con 0x7f961c109ff0 2024-04-13T05:25:50.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f9621400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f961c114100 con 0x7f961c109630 2024-04-13T05:25:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f961be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 0x7f961c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f961be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 0x7f961c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56936/0 (socket says 172.21.15.57:56936) 2024-04-13T05:25:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.445+0000 7f961be00640 1 -- 172.21.15.57:0/1838595560 learned_addr learned my addr 172.21.15.57:0/1838595560 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f9620a00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109ff0 0x7f961c1a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f9613e00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c109630 0x7f961c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f961be00640 1 -- 172.21.15.57:0/1838595560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c109630 msgr2=0x7f961c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f961be00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c109630 0x7f961c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f961be00640 1 -- 172.21.15.57:0/1838595560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109ff0 msgr2=0x7f961c1a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f961be00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109ff0 0x7f961c1a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f961be00640 1 -- 172.21.15.57:0/1838595560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1a6200 con 0x7f961c108440 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f9613e00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c109630 0x7f961c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f9620a00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109ff0 0x7f961c1a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.446+0000 7f961be00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 0x7f961c19e5e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c970 tx=0x7f960c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.447+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c004220 con 0x7f961c108440 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.447+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c1a64f0 con 0x7f961c108440 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.447+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f960c0043c0 con 0x7f961c108440 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.447+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c01d060 con 0x7f961c108440 2024-04-13T05:25:50.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.447+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1a6a90 con 0x7f961c108440 2024-04-13T05:25:50.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.448+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c10f7e0 con 0x7f961c108440 2024-04-13T05:25:50.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.450+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f960c0079b0 con 0x7f961c108440 2024-04-13T05:25:50.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.452+0000 7f9619600640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.452+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f960c0d0690 con 0x7f961c108440 2024-04-13T05:25:50.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.452+0000 7f9613e00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.453+0000 7f9613e00640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f961c19faf0 tx=0x7f9604005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.457+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f960c014030 con 0x7f961c108440 2024-04-13T05:25:50.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:50 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:25:50.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.719+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f961c02d5e0 con 0x7f961c108440 2024-04-13T05:25:50.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.720+0000 7f9619600640 1 -- 172.21.15.57:0/1838595560 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11862 (secure 0 0 0) 0x7f960c09cc70 con 0x7f961c108440 2024-04-13T05:25:50.723 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:25:50.724 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:48.741921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34499,34457]},"id":1}]} 2024-04-13T05:25:50.724 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 32 2024-04-13T05:25:50.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 msgr2=0x7f96000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f961c19faf0 tx=0x7f9604005d30 comp rx=0 tx=0).stop 2024-04-13T05:25:50.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 msgr2=0x7f961c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 0x7f961c19e5e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c970 tx=0x7f960c00ce30 comp rx=0 tx=0).stop 2024-04-13T05:25:50.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 shutdown_connections 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f96000a3bb0 0x7f96000a6070 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f961c109ff0 0x7f961c1a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f961c109630 0x7f961c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 --2- 172.21.15.57:0/1838595560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f961c108440 0x7f961c19e5e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.723+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 >> 172.21.15.57:0/1838595560 conn(0x7f961c077400 msgr2=0x7f961c10c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.724+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 shutdown_connections 2024-04-13T05:25:50.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.724+0000 7f9621400640 1 -- 172.21.15.57:0/1838595560 wait complete. 2024-04-13T05:25:50.742 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:50 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:25:50.747 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:25:50.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:50 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.980+0000 7f3939a00640 1 -- 172.21.15.57:0/3668365146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934109e10 msgr2=0x7f393410c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.980+0000 7f3939a00640 1 --2- 172.21.15.57:0/3668365146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934109e10 0x7f393410c2a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f392c009ca0 tx=0x7f392c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 -- 172.21.15.57:0/3668365146 shutdown_connections 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 --2- 172.21.15.57:0/3668365146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934074a20 0x7f3934074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 --2- 172.21.15.57:0/3668365146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934109e10 0x7f393410c2a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 --2- 172.21.15.57:0/3668365146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3934076000 0x7f3934074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 -- 172.21.15.57:0/3668365146 >> 172.21.15.57:0/3668365146 conn(0x7f39340fd4c0 msgr2=0x7f39340ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 -- 172.21.15.57:0/3668365146 shutdown_connections 2024-04-13T05:25:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.981+0000 7f3939a00640 1 -- 172.21.15.57:0/3668365146 wait complete. 2024-04-13T05:25:50.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.983+0000 7f3939a00640 1 Processor -- start 2024-04-13T05:25:50.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.983+0000 7f3939a00640 1 -- start start 2024-04-13T05:25:50.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 0x7f393419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3934076000 0x7f393419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934109e10 0x7f39341a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f393410fc10 con 0x7f3934074a20 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f393410fa90 con 0x7f3934109e10 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f393410fd90 con 0x7f3934076000 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3933e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 0x7f393419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3933e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 0x7f393419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56956/0 (socket says 172.21.15.57:56956) 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3933e00640 1 -- 172.21.15.57:0/3304393597 learned_addr learned my addr 172.21.15.57:0/3304393597 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3939000640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934109e10 0x7f39341a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.984+0000 7f3933400640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3934076000 0x7f393419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3933e00640 1 -- 172.21.15.57:0/3304393597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3934076000 msgr2=0x7f393419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3933e00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3934076000 0x7f393419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3933e00640 1 -- 172.21.15.57:0/3304393597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934109e10 msgr2=0x7f39341a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3933e00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934109e10 0x7f39341a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3933e00640 1 -- 172.21.15.57:0/3304393597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39341a1ea0 con 0x7f3934074a20 2024-04-13T05:25:50.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3939000640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934109e10 0x7f39341a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:50.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3933e00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 0x7f393419a280 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f392400d770 tx=0x7f392400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3930c00640 1 -- 172.21.15.57:0/3304393597 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924017070 con 0x7f3934074a20 2024-04-13T05:25:50.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3930c00640 1 -- 172.21.15.57:0/3304393597 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39240054d0 con 0x7f3934074a20 2024-04-13T05:25:50.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39341a2190 con 0x7f3934074a20 2024-04-13T05:25:50.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3930c00640 1 -- 172.21.15.57:0/3304393597 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924004cf0 con 0x7f3934074a20 2024-04-13T05:25:50.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.985+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39341a2690 con 0x7f3934074a20 2024-04-13T05:25:50.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.989+0000 7f393de70740 1 Processor -- start 2024-04-13T05:25:50.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.989+0000 7f3930c00640 1 -- 172.21.15.57:0/3304393597 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3924004e90 con 0x7f3934074a20 2024-04-13T05:25:50.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.989+0000 7f393de70740 1 -- start start 2024-04-13T05:25:50.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f393de70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b69860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f393de70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b6e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f393de70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f393de70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5598e2acf020 con 0x5598e2b6efa0 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f393de70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598e2a75950 con 0x5598e2b69440 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f393de70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598e2acafe0 con 0x5598e2b69e30 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f3933400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f3933400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b71430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56960/0 (socket says 172.21.15.57:56960) 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f3933400640 1 -- 172.21.15.57:0/2334382795 learned_addr learned my addr 172.21.15.57:0/2334382795 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f3939000640 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b69860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.990+0000 7f3933e00640 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b6e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3933400640 1 -- 172.21.15.57:0/2334382795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 msgr2=0x5598e2b6e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3933400640 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b6e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3933400640 1 -- 172.21.15.57:0/2334382795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 msgr2=0x5598e2b69860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:50.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3933400640 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b69860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3933400640 1 -- 172.21.15.57:0/2334382795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e27e13f0 con 0x5598e2b6efa0 2024-04-13T05:25:50.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3930c00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f39180a3bb0 0x7f39180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3930c00640 1 -- 172.21.15.57:0/3304393597 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f39240d02c0 con 0x7f3934074a20 2024-04-13T05:25:50.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3933400640 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b71430 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f392000bc80 tx=0x7f392000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4bd49ffbbcc8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3939000640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f39180a3bb0 0x7f39180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3929600640 1 -- 172.21.15.57:0/2334382795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920015070 con 0x5598e2b6efa0 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.991+0000 7f3929600640 1 -- 172.21.15.57:0/2334382795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392001a910 con 0x5598e2b6efa0 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f3929600640 1 -- 172.21.15.57:0/2334382795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39200118d0 con 0x5598e2b6efa0 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 -- 172.21.15.57:0/2334382795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 msgr2=0x5598e2b71430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f3939000640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f39180a3bb0 0x7f39180a6070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f392c02c630 tx=0x7f392c005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b71430 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f392000bc80 tx=0x7f392000f730 comp rx=0 tx=0).stop 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 -- 172.21.15.57:0/2334382795 shutdown_connections 2024-04-13T05:25:50.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b71430 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b6e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 --2- 172.21.15.57:0/2334382795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b69860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.992+0000 7f393de70740 1 -- 172.21.15.57:0/2334382795 >> 172.21.15.57:0/2334382795 conn(0x5598e2b649b0 msgr2=0x5598e2b64db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.993+0000 7f393de70740 1 -- 172.21.15.57:0/2334382795 shutdown_connections 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.993+0000 7f393de70740 1 -- 172.21.15.57:0/2334382795 wait complete. 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.993+0000 7f393de70740 1 -- start start 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.993+0000 7f393de70740 1 Processor -- start 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b73950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 0x5598e2b73e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b6efa0 0x5598e2b7aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5598e2b71f50 con 0x5598e2b69e30 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598e2a88c80 con 0x5598e2b69440 2024-04-13T05:25:50.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598e2b720d0 con 0x5598e2b6efa0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f393de70740 10 client.0 _ll_register_callbacks cb 0x5598e2a88670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f3933400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 0x5598e2b73e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f3933400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 0x5598e2b73e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56962/0 (socket says 172.21.15.57:56962) 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f3933400640 1 -- 172.21.15.57:0/926223777 learned_addr learned my addr 172.21.15.57:0/926223777 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f3933e00640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b73950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.994+0000 7f3939000640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b6efa0 0x5598e2b7aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f3933400640 1 -- 172.21.15.57:0/926223777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b6efa0 msgr2=0x5598e2b7aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f3933400640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b6efa0 0x5598e2b7aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f3933400640 1 -- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 msgr2=0x5598e2b73950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f3933400640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b73950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f3933400640 1 -- 172.21.15.57:0/926223777 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e28ab7d0 con 0x5598e2b69e30 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f3933400640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 0x5598e2b73e90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3920004630 tx=0x7f392000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666d0580e55609f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f391e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:51.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f391e000640 1 -- 172.21.15.57:0/926223777 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920015070 con 0x5598e2b69e30 2024-04-13T05:25:51.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f391e000640 1 -- 172.21.15.57:0/926223777 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f392001acc0 con 0x5598e2b69e30 2024-04-13T05:25:51.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.995+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392001a910 con 0x5598e2b69e30 2024-04-13T05:25:51.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f391e000640 1 -- 172.21.15.57:0/926223777 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3920030050 con 0x5598e2b69e30 2024-04-13T05:25:51.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 10 client.17088 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:51.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5598e2b7e290 con 0x5598e2b69e30 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f391e000640 1 -- 172.21.15.57:0/926223777 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392001a910 con 0x5598e2b69e30 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f391e000640 1 -- 172.21.15.57:0/926223777 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f392002b030 con 0x5598e2b69e30 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 10 client.17088 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 10 client.17088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 20 client.17088 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2b7f690 0x5598e2b81a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 4 client.17088 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.996+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e2a6e750 con 0x5598e2b7f690 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.997+0000 7f3933e00640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2b7f690 0x5598e2b81a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.998+0000 7f3933e00640 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2b7f690 0x5598e2b81a80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f19a028325d18cd3 server_cookie=5d0fb1b7ee7bed3e in_seq=0 out_seq=0 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:50.998+0000 7f391e000640 10 client.17088 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f391e000640 1 -- 172.21.15.57:0/926223777 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598e2a6e750 con 0x5598e2b7f690 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f391e000640 10 client.17088 handle_command_reply: tid=0 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 client.17088 shutdown 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 msgr2=0x5598e2b73e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 0x5598e2b73e90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3920004630 tx=0x7f392000bfa0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 shutdown_connections 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2b7f690 0x5598e2b81a80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b6efa0 0x5598e2b7aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69e30 0x5598e2b73e90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 --2- 172.21.15.57:0/926223777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b73950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f391e000640 0 client.17088 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 >> 172.21.15.57:0/926223777 conn(0x5598e2b649b0 msgr2=0x5598e2b63cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.000+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 shutdown_connections 2024-04-13T05:25:51.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.001+0000 7f393de70740 1 -- 172.21.15.57:0/926223777 wait complete. 2024-04-13T05:25:51.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.001+0000 7f393de70740 20 client.17088 trim_cache size 0 max 16384 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 Processor -- start 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 -- start start 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b7ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b63980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f39341af430 con 0x5598e2b6efa0 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39341acc70 con 0x5598e2b69440 2024-04-13T05:25:51.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.020+0000 7f393de70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39341aaac0 con 0x5598e2b69e30 2024-04-13T05:25:51.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b63980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b63980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56976/0 (socket says 172.21.15.57:56976) 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 -- 172.21.15.57:0/2004836970 learned_addr learned my addr 172.21.15.57:0/2004836970 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3933400640 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b7ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3933e00640 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 -- 172.21.15.57:0/2004836970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 msgr2=0x5598e2b7ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b7ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 -- 172.21.15.57:0/2004836970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 msgr2=0x5598e2b63440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b63440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.021+0000 7f3939000640 1 -- 172.21.15.57:0/2004836970 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e2b6a210 con 0x5598e2b6efa0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.022+0000 7f3939000640 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b63980 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f392c009ca0 tx=0x7f392c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eacdb94b2826c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.022+0000 7f3928c00640 1 -- 172.21.15.57:0/2004836970 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c039040 con 0x5598e2b6efa0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.022+0000 7f3928c00640 1 -- 172.21.15.57:0/2004836970 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392c052070 con 0x5598e2b6efa0 2024-04-13T05:25:51.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.022+0000 7f3928c00640 1 -- 172.21.15.57:0/2004836970 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c040750 con 0x5598e2b6efa0 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 -- 172.21.15.57:0/2004836970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 msgr2=0x5598e2b63980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b63980 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f392c009ca0 tx=0x7f392c040000 comp rx=0 tx=0).stop 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 -- 172.21.15.57:0/2004836970 shutdown_connections 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b6efa0 0x5598e2b63980 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b7ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 --2- 172.21.15.57:0/2004836970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b69440 0x5598e2b63440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 -- 172.21.15.57:0/2004836970 >> 172.21.15.57:0/2004836970 conn(0x5598e2b649b0 msgr2=0x5598e2b64d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 -- 172.21.15.57:0/2004836970 shutdown_connections 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.023+0000 7f393de70740 1 -- 172.21.15.57:0/2004836970 wait complete. 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.024+0000 7f393de70740 1 -- start start 2024-04-13T05:25:51.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.024+0000 7f393de70740 1 Processor -- start 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.024+0000 7f393de70740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 0x5598e2b78b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.024+0000 7f393de70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f393de70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b6efa0 0x5598e2b7a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f393de70740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f39341ab130 con 0x5598e2b69440 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f393de70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39341ac5a0 con 0x5598e2b6efa0 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f393de70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39341ab800 con 0x5598e2b69e30 2024-04-13T05:25:51.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f393de70740 10 client.0 _ll_register_callbacks cb 0x7f392000b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:51.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f3933e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 0x5598e2b78b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f3933e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 0x5598e2b78b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56978/0 (socket says 172.21.15.57:56978) 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.025+0000 7f3933e00640 1 -- 172.21.15.57:0/282892825 learned_addr learned my addr 172.21.15.57:0/282892825 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.026+0000 7f3939000640 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b6efa0 0x5598e2b7a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.026+0000 7f3933e00640 1 -- 172.21.15.57:0/282892825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 msgr2=0x5598e2b790b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.026+0000 7f3933e00640 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.026+0000 7f3933e00640 1 -- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b6efa0 msgr2=0x5598e2b7a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.026+0000 7f3933e00640 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b6efa0 0x5598e2b7a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.026+0000 7f3933e00640 1 -- 172.21.15.57:0/282892825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e2b7a6b0 con 0x5598e2b69440 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.027+0000 7f3933e00640 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 0x5598e2b78b70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f392409bf00 tx=0x7f39240d04c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1893d85d4274be64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.027+0000 7f391e000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.027+0000 7f391e000640 1 -- 172.21.15.57:0/282892825 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924020040 con 0x5598e2b69440 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.027+0000 7f391e000640 1 -- 172.21.15.57:0/282892825 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3924098870 con 0x5598e2b69440 2024-04-13T05:25:51.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.027+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392c052070 con 0x5598e2b69440 2024-04-13T05:25:51.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.028+0000 7f391e000640 1 -- 172.21.15.57:0/282892825 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3924005180 con 0x5598e2b69440 2024-04-13T05:25:51.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.028+0000 7f393de70740 10 client.17094 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:51.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.028+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5598e2bee9b0 con 0x5598e2b69440 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.028+0000 7f391e000640 1 -- 172.21.15.57:0/282892825 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392c052070 con 0x5598e2b69440 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f391e000640 1 -- 172.21.15.57:0/282892825 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f3924083c40 con 0x5598e2b69440 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f393de70740 10 client.17094 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f393de70740 10 client.17094 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f393de70740 20 client.17094 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f393de70740 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2bef360 0x5598e2bf1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f393de70740 4 client.17094 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.029+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598e2adca40 con 0x5598e2bef360 2024-04-13T05:25:51.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.030+0000 7f3933400640 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2bef360 0x5598e2bf1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.032+0000 7f3933400640 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2bef360 0x5598e2bf1780 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b1499b5fe9e46b1 server_cookie=fffdee09703983f7 in_seq=0 out_seq=0 2024-04-13T05:25:51.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.032+0000 7f391e000640 10 client.17094 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:51.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.032+0000 7f391e000640 1 -- 172.21.15.57:0/282892825 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598e2adca40 con 0x5598e2bef360 2024-04-13T05:25:51.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.032+0000 7f391e000640 10 client.17094 handle_command_reply: tid=0 2024-04-13T05:25:51.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.032+0000 7f393de70740 1 client.17094 shutdown 2024-04-13T05:25:51.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 msgr2=0x5598e2b78b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 0x5598e2b78b70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f392409bf00 tx=0x7f39240d04c0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 shutdown_connections 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5598e2bef360 0x5598e2bf1780 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598e2b6efa0 0x5598e2b7a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598e2b69e30 0x5598e2b790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 --2- 172.21.15.57:0/282892825 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598e2b69440 0x5598e2b78b70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 >> 172.21.15.57:0/282892825 conn(0x5598e2b649b0 msgr2=0x5598e2b67c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.033+0000 7f391e000640 0 client.17094 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:25:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.034+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 shutdown_connections 2024-04-13T05:25:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.034+0000 7f393de70740 1 -- 172.21.15.57:0/282892825 wait complete. 2024-04-13T05:25:51.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.034+0000 7f393de70740 20 client.17094 trim_cache size 0 max 16384 2024-04-13T05:25:51.037 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.035+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f39180a3bb0 msgr2=0x7f39180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.035+0000 7f3939a00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f39180a3bb0 0x7f39180a6070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f392c02c630 tx=0x7f392c005d80 comp rx=0 tx=0).stop 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.035+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 msgr2=0x7f393419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.035+0000 7f3939a00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 0x7f393419a280 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f392400d770 tx=0x7f392400dc30 comp rx=0 tx=0).stop 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 shutdown_connections 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f39180a3bb0 0x7f39180a6070 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3934109e10 0x7f39341a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3934076000 0x7f393419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 --2- 172.21.15.57:0/3304393597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3934074a20 0x7f393419a280 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 >> 172.21.15.57:0/3304393597 conn(0x7f39340fd4c0 msgr2=0x7f3934105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 shutdown_connections 2024-04-13T05:25:51.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.036+0000 7f3939a00640 1 -- 172.21.15.57:0/3304393597 wait complete. 2024-04-13T05:25:51.069 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:25:51.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.286+0000 7f09f8a00640 1 -- 172.21.15.57:0/72882161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4078560 msgr2=0x7f09f4078e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.286+0000 7f09f8a00640 1 --2- 172.21.15.57:0/72882161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4078560 0x7f09f4078e00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009ca0 tx=0x7f09ec02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 -- 172.21.15.57:0/72882161 shutdown_connections 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 --2- 172.21.15.57:0/72882161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09f4077bc0 0x7f09f4078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 --2- 172.21.15.57:0/72882161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4078560 0x7f09f4078e00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 --2- 172.21.15.57:0/72882161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4076980 0x7f09f4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 -- 172.21.15.57:0/72882161 >> 172.21.15.57:0/72882161 conn(0x7f09f40ff7b0 msgr2=0x7f09f4101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 -- 172.21.15.57:0/72882161 shutdown_connections 2024-04-13T05:25:51.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.287+0000 7f09f8a00640 1 -- 172.21.15.57:0/72882161 wait complete. 2024-04-13T05:25:51.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 Processor -- start 2024-04-13T05:25:51.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 -- start start 2024-04-13T05:25:51.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09f4076980 0x7f09f419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 0x7f09f419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4078560 0x7f09f41a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4114160 con 0x7f09f4077bc0 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4113fe0 con 0x7f09f4076980 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.288+0000 7f09f8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41142e0 con 0x7f09f4078560 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 0x7f09f419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 0x7f09f419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57000/0 (socket says 172.21.15.57:57000) 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 -- 172.21.15.57:0/4175196519 learned_addr learned my addr 172.21.15.57:0/4175196519 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09f1e00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09f4076980 0x7f09f419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09f2800640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4078560 0x7f09f41a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 -- 172.21.15.57:0/4175196519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4078560 msgr2=0x7f09f41a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4078560 0x7f09f41a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 -- 172.21.15.57:0/4175196519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09f4076980 msgr2=0x7f09f419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09f4076980 0x7f09f419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09e9400640 1 -- 172.21.15.57:0/4175196519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f41a6410 con 0x7f09f4077bc0 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.289+0000 7f09f2800640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4078560 0x7f09f41a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.290+0000 7f09e9400640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 0x7f09f419ed30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00d950 tx=0x7f09dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.290+0000 7f09eb400640 1 -- 172.21.15.57:0/4175196519 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc017070 con 0x7f09f4077bc0 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.290+0000 7f09eb400640 1 -- 172.21.15.57:0/4175196519 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09dc009c30 con 0x7f09f4077bc0 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.290+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f41a6700 con 0x7f09f4077bc0 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.290+0000 7f09eb400640 1 -- 172.21.15.57:0/4175196519 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00f620 con 0x7f09f4077bc0 2024-04-13T05:25:51.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.290+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f41a6c50 con 0x7f09f4077bc0 2024-04-13T05:25:51.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09eb400640 1 -- 172.21.15.57:0/4175196519 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f09dc0109a0 con 0x7f09f4077bc0 2024-04-13T05:25:51.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 Processor -- start 2024-04-13T05:25:51.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 -- start start 2024-04-13T05:25:51.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fcfb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd00740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd03240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55be1fc61020 con 0x55be1fcfbc40 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be1fc07950 con 0x55be1fcfb250 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55be1fc5cfe0 con 0x55be1fd00db0 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.294+0000 7f09f1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd00740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd00740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57012/0 (socket says 172.21.15.57:57012) 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 -- 172.21.15.57:0/2134485625 learned_addr learned my addr 172.21.15.57:0/2134485625 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f2800640 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fcfb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09e9400640 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd03240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 -- 172.21.15.57:0/2134485625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 msgr2=0x55be1fd03240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd03240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 -- 172.21.15.57:0/2134485625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 msgr2=0x55be1fcfb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fcfb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 -- 172.21.15.57:0/2134485625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be1f9733f0 con 0x55be1fcfbc40 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09e9400640 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd03240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.295+0000 7f09f1e00640 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd00740 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f09e400bcb0 tx=0x7f09e400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24526742aade1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.296+0000 7f09eb400640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09d80a3bb0 0x7f09d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.296+0000 7f09e3000640 1 -- 172.21.15.57:0/2134485625 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e4015070 con 0x55be1fcfbc40 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.296+0000 7f09e3000640 1 -- 172.21.15.57:0/2134485625 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09e40118f0 con 0x55be1fcfbc40 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.296+0000 7f09e3000640 1 -- 172.21.15.57:0/2134485625 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e40108b0 con 0x55be1fcfbc40 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.296+0000 7f09eb400640 1 -- 172.21.15.57:0/4175196519 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f09dc0d04e0 con 0x7f09f4077bc0 2024-04-13T05:25:51.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.296+0000 7f09f2800640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09d80a3bb0 0x7f09d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2134485625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 msgr2=0x55be1fd00740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd00740 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f09e400bcb0 tx=0x7f09e400f730 comp rx=0 tx=0).stop 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2134485625 shutdown_connections 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd03240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd00740 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2134485625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fcfb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2134485625 >> 172.21.15.57:0/2134485625 conn(0x55be1fcf67c0 msgr2=0x55be1fcf6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09f2800640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09d80a3bb0 0x7f09d80a6070 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09ec032420 tx=0x7f09ec005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2134485625 shutdown_connections 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2134485625 wait complete. 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.297+0000 7f09fb9dd740 1 -- start start 2024-04-13T05:25:51.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 Processor -- start 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd05770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55be1fd03d60 con 0x55be1fcfbc40 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be1fc1ac80 con 0x55be1fcfb250 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55be1fd03ee0 con 0x55be1fd00db0 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.298+0000 7f09fb9dd740 10 client.0 _ll_register_callbacks cb 0x55be1fc1a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd05cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57018/0 (socket says 172.21.15.57:57018) 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 -- 172.21.15.57:0/3681109618 learned_addr learned my addr 172.21.15.57:0/3681109618 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09f1e00640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd05770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09f2800640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 -- 172.21.15.57:0/3681109618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 msgr2=0x55be1fd0ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 -- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 msgr2=0x55be1fd05770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd05770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 -- 172.21.15.57:0/3681109618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be1fa3d7d0 con 0x55be1fcfbc40 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09e9400640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd05cb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0cdbe0 tx=0x7f09dc009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c98bf5b8e2b6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.299+0000 7f09d7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09d7400640 1 -- 172.21.15.57:0/3681109618 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0b0e60 con 0x55be1fcfbc40 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09d7400640 1 -- 172.21.15.57:0/3681109618 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f09dc098a30 con 0x55be1fcfbc40 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e40118f0 con 0x55be1fcfbc40 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09d7400640 1 -- 172.21.15.57:0/3681109618 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f09dc020070 con 0x55be1fcfbc40 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09fb9dd740 10 client.17106 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55be1fd10040 con 0x55be1fcfbc40 2024-04-13T05:25:51.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.300+0000 7f09d7400640 1 -- 172.21.15.57:0/3681109618 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09e40118f0 con 0x55be1fcfbc40 2024-04-13T05:25:51.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09d7400640 1 -- 172.21.15.57:0/3681109618 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f09dc09b9a0 con 0x55be1fcfbc40 2024-04-13T05:25:51.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09fb9dd740 10 client.17106 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:51.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09fb9dd740 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:25:51.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09fb9dd740 20 client.17106 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd116b0 0x55be1fd13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09fb9dd740 4 client.17106 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.301+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be1fc00750 con 0x55be1fd116b0 2024-04-13T05:25:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.302+0000 7f09f1e00640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd116b0 0x55be1fd13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.303+0000 7f09f1e00640 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd116b0 0x55be1fd13b70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9a97b8f653bf835 server_cookie=b3d7adb8da40a824 in_seq=0 out_seq=0 2024-04-13T05:25:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.303+0000 7f09d7400640 10 client.17106 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.304+0000 7f09d7400640 1 -- 172.21.15.57:0/3681109618 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be1fc00750 con 0x55be1fd116b0 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.304+0000 7f09d7400640 10 client.17106 handle_command_reply: tid=0 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.304+0000 7f09fb9dd740 1 client.17106 shutdown 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 msgr2=0x55be1fd05cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd05cb0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0cdbe0 tx=0x7f09dc009de0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 shutdown_connections 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd116b0 0x55be1fd13b70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd05cb0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09d7400640 0 client.17106 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/3681109618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd05770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 >> 172.21.15.57:0/3681109618 conn(0x55be1fcf67c0 msgr2=0x55be1fcf5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 shutdown_connections 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 1 -- 172.21.15.57:0/3681109618 wait complete. 2024-04-13T05:25:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.305+0000 7f09fb9dd740 20 client.17106 trim_cache size 0 max 16384 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.323+0000 7f09fb9dd740 1 Processor -- start 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.323+0000 7f09fb9dd740 1 -- start start 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.323+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd18430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 0x55be1fd18970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b3b40 con 0x55be1fd00db0 2024-04-13T05:25:51.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b1380 con 0x55be1fcfb250 2024-04-13T05:25:51.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41af1d0 con 0x55be1fcfbc40 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 0x55be1fd18970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 0x55be1fd18970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57032/0 (socket says 172.21.15.57:57032) 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 -- 172.21.15.57:0/2003142559 learned_addr learned my addr 172.21.15.57:0/2003142559 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09e9400640 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f1e00640 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd18430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 -- 172.21.15.57:0/2003142559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fcfbc40 msgr2=0x55be1fd0ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 -- 172.21.15.57:0/2003142559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 msgr2=0x55be1fd18430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd18430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.324+0000 7f09f2800640 1 -- 172.21.15.57:0/2003142559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be1fd04850 con 0x55be1fd00db0 2024-04-13T05:25:51.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.325+0000 7f09f2800640 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 0x55be1fd18970 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f09ec036000 tx=0x7f09ec03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f05bfc388e8aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.325+0000 7f09e2600640 1 -- 172.21.15.57:0/2003142559 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec050070 con 0x55be1fd00db0 2024-04-13T05:25:51.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.325+0000 7f09e2600640 1 -- 172.21.15.57:0/2003142559 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09ec045040 con 0x55be1fd00db0 2024-04-13T05:25:51.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.325+0000 7f09e2600640 1 -- 172.21.15.57:0/2003142559 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec039040 con 0x55be1fd00db0 2024-04-13T05:25:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2003142559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 msgr2=0x55be1fd18970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 0x55be1fd18970 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f09ec036000 tx=0x7f09ec03e000 comp rx=0 tx=0).stop 2024-04-13T05:25:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2003142559 shutdown_connections 2024-04-13T05:25:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fd00db0 0x55be1fd18970 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/2003142559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd18430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2003142559 >> 172.21.15.57:0/2003142559 conn(0x55be1fcf67c0 msgr2=0x55be1fcfa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2003142559 shutdown_connections 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.326+0000 7f09fb9dd740 1 -- 172.21.15.57:0/2003142559 wait complete. 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 -- start start 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 Processor -- start 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd0c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41af840 con 0x55be1fcfbc40 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41b0cb0 con 0x55be1fcfb250 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.327+0000 7f09fb9dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41aff10 con 0x55be1fd00db0 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09fb9dd740 10 client.0 _ll_register_callbacks cb 0x7f09dc09bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57044/0 (socket says 172.21.15.57:57044) 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 -- 172.21.15.57:0/251908091 learned_addr learned my addr 172.21.15.57:0/251908091 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09f1e00640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd0c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09f2800640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 -- 172.21.15.57:0/251908091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 msgr2=0x55be1fd0ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 -- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 msgr2=0x55be1fd0c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd0c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09e9400640 1 -- 172.21.15.57:0/251908091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be1fd0b180 con 0x55be1fcfbc40 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.328+0000 7f09f1e00640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd0c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09e9400640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ca70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0981b0 tx=0x7f09dc09cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a74d7276ffb76c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09d7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09d7400640 1 -- 172.21.15.57:0/251908091 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0b0e60 con 0x55be1fcfbc40 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09d7400640 1 -- 172.21.15.57:0/251908091 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f09dc09b810 con 0x55be1fcfbc40 2024-04-13T05:25:51.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09ec045040 con 0x55be1fcfbc40 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09d7400640 1 -- 172.21.15.57:0/251908091 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f09dc098450 con 0x55be1fcfbc40 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09fb9dd740 10 client.17112 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55be1fd78660 con 0x55be1fcfbc40 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.329+0000 7f09d7400640 1 -- 172.21.15.57:0/251908091 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09ec045040 con 0x55be1fcfbc40 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09d7400640 1 -- 172.21.15.57:0/251908091 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f09dc09b280 con 0x55be1fcfbc40 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09fb9dd740 10 client.17112 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09fb9dd740 10 client.17112 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09fb9dd740 20 client.17112 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd78ec0 0x55be1fd7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09fb9dd740 4 client.17112 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:25:51.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be1fc6ea40 con 0x55be1fd78ec0 2024-04-13T05:25:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.330+0000 7f09f1e00640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd78ec0 0x55be1fd7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.331+0000 7f09f1e00640 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd78ec0 0x55be1fd7b2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3da3b68a97f71f9 server_cookie=748c14b55f189cc2 in_seq=0 out_seq=0 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.331+0000 7f09d7400640 10 client.17112 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09d7400640 1 -- 172.21.15.57:0/251908091 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be1fc6ea40 con 0x55be1fd78ec0 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09d7400640 10 client.17112 handle_command_reply: tid=0 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 client.17112 shutdown 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 msgr2=0x55be1fd0ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ca70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0981b0 tx=0x7f09dc09cb30 comp rx=0 tx=0).stop 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 shutdown_connections 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55be1fd78ec0 0x55be1fd7b2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be1fd00db0 0x55be1fd0ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be1fcfbc40 0x55be1fd0ca70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 --2- 172.21.15.57:0/251908091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be1fcfb250 0x55be1fd0c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09d7400640 0 client.17112 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.332+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 >> 172.21.15.57:0/251908091 conn(0x55be1fcf67c0 msgr2=0x55be1fcf51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 shutdown_connections 2024-04-13T05:25:51.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09fb9dd740 1 -- 172.21.15.57:0/251908091 wait complete. 2024-04-13T05:25:51.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09fb9dd740 20 client.17112 trim_cache size 0 max 16384 2024-04-13T05:25:51.338 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:25:51.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09d80a3bb0 msgr2=0x7f09d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09f8a00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09d80a3bb0 0x7f09d80a6070 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09ec032420 tx=0x7f09ec005f80 comp rx=0 tx=0).stop 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 msgr2=0x7f09f419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.333+0000 7f09f8a00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 0x7f09f419ed30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00d950 tx=0x7f09dc00de10 comp rx=0 tx=0).stop 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 shutdown_connections 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f09d80a3bb0 0x7f09d80a6070 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f4078560 0x7f09f41a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09f4077bc0 0x7f09f419ed30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 --2- 172.21.15.57:0/4175196519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09f4076980 0x7f09f419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 >> 172.21.15.57:0/4175196519 conn(0x7f09f40ff7b0 msgr2=0x7f09f41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 shutdown_connections 2024-04-13T05:25:51.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.334+0000 7f09f8a00640 1 -- 172.21.15.57:0/4175196519 wait complete. 2024-04-13T05:25:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:51 smithi057 ceph-mon[32331]: pgmap v703: 129 pgs: 129 active+clean; 460 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 126 KiB/s wr, 40 op/s 2024-04-13T05:25:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:51 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1838595560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:25:51.354 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:25:51.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 -- 172.21.15.57:0/1226942869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e980749c0 msgr2=0x7f1e98074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 --2- 172.21.15.57:0/1226942869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e980749c0 0x7f1e98074e20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009900 tx=0x7f1e8402bcd0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 -- 172.21.15.57:0/1226942869 shutdown_connections 2024-04-13T05:25:51.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 --2- 172.21.15.57:0/1226942869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 0x7f1e9810c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 --2- 172.21.15.57:0/1226942869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e980749c0 0x7f1e98074e20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 --2- 172.21.15.57:0/1226942869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e98075fa0 0x7f1e980743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 -- 172.21.15.57:0/1226942869 >> 172.21.15.57:0/1226942869 conn(0x7f1e980fd460 msgr2=0x7f1e980ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 -- 172.21.15.57:0/1226942869 shutdown_connections 2024-04-13T05:25:51.579 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.575+0000 7f1e97200640 1 -- 172.21.15.57:0/1226942869 wait complete. 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.576+0000 7f1e97200640 1 Processor -- start 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 -- start start 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e980749c0 0x7f1e9819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 0x7f1e9819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 0x7f1e981a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9810fc80 con 0x7f1e98075fa0 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9810fb00 con 0x7f1e98109d80 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e97200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9810fe00 con 0x7f1e980749c0 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e95400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 0x7f1e9819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e95400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 0x7f1e9819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57060/0 (socket says 172.21.15.57:57060) 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.577+0000 7f1e95400640 1 -- 172.21.15.57:0/1146723849 learned_addr learned my addr 172.21.15.57:0/1146723849 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e96800640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 0x7f1e981a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e980749c0 0x7f1e9819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95400640 1 -- 172.21.15.57:0/1146723849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e980749c0 msgr2=0x7f1e9819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95400640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e980749c0 0x7f1e9819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95400640 1 -- 172.21.15.57:0/1146723849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 msgr2=0x7f1e981a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95400640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 0x7f1e981a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95400640 1 -- 172.21.15.57:0/1146723849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e981a1f40 con 0x7f1e98075fa0 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e96800640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 0x7f1e981a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:51.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.578+0000 7f1e95400640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 0x7f1e9819a860 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1e84000c00 tx=0x7f1e84005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.579+0000 7f1e8ea00640 1 -- 172.21.15.57:0/1146723849 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84043020 con 0x7f1e98075fa0 2024-04-13T05:25:51.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.579+0000 7f1e8ea00640 1 -- 172.21.15.57:0/1146723849 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e8402dd20 con 0x7f1e98075fa0 2024-04-13T05:25:51.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.579+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e981a21d0 con 0x7f1e98075fa0 2024-04-13T05:25:51.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.579+0000 7f1e8ea00640 1 -- 172.21.15.57:0/1146723849 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84003bb0 con 0x7f1e98075fa0 2024-04-13T05:25:51.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.579+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e981a2720 con 0x7f1e98075fa0 2024-04-13T05:25:51.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:51 smithi189 ceph-mon[35531]: pgmap v703: 129 pgs: 129 active+clean; 460 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 126 KiB/s wr, 40 op/s 2024-04-13T05:25:51.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:51 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1838595560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.582+0000 7f1e9fd8f740 1 Processor -- start 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.582+0000 7f1e9fd8f740 1 -- start start 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.582+0000 7f1e8ea00640 1 -- 172.21.15.57:0/1146723849 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1e84046020 con 0x7f1e98075fa0 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e10860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e15930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e18430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55af17d76020 con 0x55af17e15fa0 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af17d1c950 con 0x55af17e10440 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55af17d71fe0 con 0x55af17e10e30 2024-04-13T05:25:51.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e95400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e18430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e95400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e18430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57064/0 (socket says 172.21.15.57:57064) 2024-04-13T05:25:51.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e95400640 1 -- 172.21.15.57:0/2156132287 learned_addr learned my addr 172.21.15.57:0/2156132287 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e96800640 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e10860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.583+0000 7f1e95e00640 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e15930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e95400640 1 -- 172.21.15.57:0/2156132287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 msgr2=0x55af17e15930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e95400640 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e15930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e95400640 1 -- 172.21.15.57:0/2156132287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 msgr2=0x55af17e10860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e95400640 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e10860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e95400640 1 -- 172.21.15.57:0/2156132287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af17a883f0 con 0x55af17e15fa0 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e95400640 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e18430 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1e840f8a60 tx=0x7f1e840f8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202d1797601bbaf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.584+0000 7f1e8ea00640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1e7c0a3bb0 0x7f1e7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e8ea00640 1 -- 172.21.15.57:0/1146723849 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1e840f2a60 con 0x7f1e98075fa0 2024-04-13T05:25:51.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e83400640 1 -- 172.21.15.57:0/2156132287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84101070 con 0x55af17e15fa0 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e96800640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1e7c0a3bb0 0x7f1e7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e83400640 1 -- 172.21.15.57:0/2156132287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e840f2e50 con 0x55af17e15fa0 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e83400640 1 -- 172.21.15.57:0/2156132287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e840be710 con 0x55af17e15fa0 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/2156132287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 msgr2=0x55af17e18430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e96800640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1e7c0a3bb0 0x7f1e7c0a6070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55af17e11690 tx=0x7f1e90009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.585+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e18430 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1e840f8a60 tx=0x7f1e840f8b50 comp rx=0 tx=0).stop 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/2156132287 shutdown_connections 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e18430 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e15930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/2156132287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e10860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/2156132287 >> 172.21.15.57:0/2156132287 conn(0x55af17e0b9b0 msgr2=0x55af17e0bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/2156132287 shutdown_connections 2024-04-13T05:25:51.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/2156132287 wait complete. 2024-04-13T05:25:51.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 -- start start 2024-04-13T05:25:51.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.586+0000 7f1e9fd8f740 1 Processor -- start 2024-04-13T05:25:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10440 0x55af17e1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 0x55af17e1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e21ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55af17e18d10 con 0x55af17e10e30 2024-04-13T05:25:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af17d2fc80 con 0x55af17e15fa0 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55af17e18e90 con 0x55af17e10440 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e9fd8f740 10 client.0 _ll_register_callbacks cb 0x55af17d2f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e95400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 0x55af17e1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e95400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 0x55af17e1aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57066/0 (socket says 172.21.15.57:57066) 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.587+0000 7f1e95400640 1 -- 172.21.15.57:0/3677565768 learned_addr learned my addr 172.21.15.57:0/3677565768 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95e00640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10440 0x55af17e1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e96800640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e21ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95400640 1 -- 172.21.15.57:0/3677565768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10440 msgr2=0x55af17e1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95400640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10440 0x55af17e1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95400640 1 -- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 msgr2=0x55af17e21ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95400640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e21ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95400640 1 -- 172.21.15.57:0/3677565768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af17b527d0 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95e00640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10440 0x55af17e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e95400640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 0x55af17e1aca0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1e8402c2b0 tx=0x7f1e8402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3619fd36f0484e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e77e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e77e00640 1 -- 172.21.15.57:0/3677565768 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84101030 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e77e00640 1 -- 172.21.15.57:0/3677565768 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1e84003d50 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.588+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e840f2e50 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e77e00640 1 -- 172.21.15.57:0/3677565768 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1e840be2a0 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 10 client.17124 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af17e18d10 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e77e00640 1 -- 172.21.15.57:0/3677565768 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e840f2e50 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e77e00640 1 -- 172.21.15.57:0/3677565768 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f1e840beba0 con 0x55af17e10e30 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 10 client.17124 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 20 client.17124 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e26570 0x55af17e289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 4 client.17124 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.589+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af17d15750 con 0x55af17e26570 2024-04-13T05:25:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.590+0000 7f1e95e00640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e26570 0x55af17e289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.591+0000 7f1e95e00640 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e26570 0x55af17e289d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83119314615a11c4 server_cookie=f6b218977214717a in_seq=0 out_seq=0 2024-04-13T05:25:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.591+0000 7f1e77e00640 10 client.17124 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e77e00640 1 -- 172.21.15.57:0/3677565768 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af17d15750 con 0x55af17e26570 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e77e00640 10 client.17124 handle_command_reply: tid=0 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 client.17124 shutdown 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 msgr2=0x55af17e1aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 0x55af17e1aca0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1e8402c2b0 tx=0x7f1e8402bcd0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 shutdown_connections 2024-04-13T05:25:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e26570 0x55af17e289d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e21ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10e30 0x55af17e1aca0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/3677565768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10440 0x55af17e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e77e00640 0 client.17124 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 >> 172.21.15.57:0/3677565768 conn(0x55af17e0b9b0 msgr2=0x55af17e0acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 shutdown_connections 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.593+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/3677565768 wait complete. 2024-04-13T05:25:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.594+0000 7f1e9fd8f740 20 client.17124 trim_cache size 0 max 16384 2024-04-13T05:25:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.609+0000 7f1e9fd8f740 1 Processor -- start 2024-04-13T05:25:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.609+0000 7f1e9fd8f740 1 -- start start 2024-04-13T05:25:51.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 0x55af17e0a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e21ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e0a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981af540 con 0x55af17e10440 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981acd80 con 0x55af17e15fa0 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981aabd0 con 0x55af17e10e30 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 0x55af17e0a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 0x55af17e0a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57068/0 (socket says 172.21.15.57:57068) 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 -- 172.21.15.57:0/1624032040 learned_addr learned my addr 172.21.15.57:0/1624032040 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e96800640 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e0a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 -- 172.21.15.57:0/1624032040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 msgr2=0x55af17e21ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e21ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 -- 172.21.15.57:0/1624032040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 msgr2=0x55af17e0a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.610+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e0a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.611+0000 7f1e95e00640 1 -- 172.21.15.57:0/1624032040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af17e29050 con 0x55af17e10440 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.611+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 0x55af17e0a440 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x55af17e29280 tx=0x7f1e88005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3cca61192327d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.611+0000 7f1e82a00640 1 -- 172.21.15.57:0/1624032040 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88004050 con 0x55af17e10440 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.611+0000 7f1e82a00640 1 -- 172.21.15.57:0/1624032040 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e88021070 con 0x55af17e10440 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.611+0000 7f1e82a00640 1 -- 172.21.15.57:0/1624032040 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8801b770 con 0x55af17e10440 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1624032040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 msgr2=0x55af17e0a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 0x55af17e0a440 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x55af17e29280 tx=0x7f1e88005f90 comp rx=0 tx=0).stop 2024-04-13T05:25:51.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1624032040 shutdown_connections 2024-04-13T05:25:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e15fa0 0x55af17e0a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e21ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1624032040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e10440 0x55af17e0a440 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1624032040 >> 172.21.15.57:0/1624032040 conn(0x55af17e0b9b0 msgr2=0x55af17e191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1624032040 shutdown_connections 2024-04-13T05:25:51.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.612+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1624032040 wait complete. 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 -- start start 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 Processor -- start 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e1ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e1f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981ab240 con 0x55af17e15fa0 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981ac6b0 con 0x55af17e10440 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981ab910 con 0x55af17e10e30 2024-04-13T05:25:51.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.613+0000 7f1e9fd8f740 10 client.0 _ll_register_callbacks cb 0x7f1e840035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e8ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e8ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57070/0 (socket says 172.21.15.57:57070) 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 -- 172.21.15.57:0/1229307962 learned_addr learned my addr 172.21.15.57:0/1229307962 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e1ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e95400640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e1f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 -- 172.21.15.57:0/1229307962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 msgr2=0x55af17e1f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e1f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 -- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 msgr2=0x55af17e1ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e1ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.614+0000 7f1e96800640 1 -- 172.21.15.57:0/1229307962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af17e8b100 con 0x55af17e15fa0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e96800640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e8ab40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1e900129a0 tx=0x7f1e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943c8518d19ca7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e77e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e77e00640 1 -- 172.21.15.57:0/1229307962 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90020dc0 con 0x55af17e15fa0 2024-04-13T05:25:51.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e77e00640 1 -- 172.21.15.57:0/1229307962 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1e900283e0 con 0x55af17e15fa0 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88021070 con 0x55af17e15fa0 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e77e00640 1 -- 172.21.15.57:0/1229307962 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1e90028620 con 0x55af17e15fa0 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e9fd8f740 10 client.17130 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.615+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af17e8c470 con 0x55af17e15fa0 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e77e00640 1 -- 172.21.15.57:0/1229307962 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e88021070 con 0x55af17e15fa0 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e77e00640 1 -- 172.21.15.57:0/1229307962 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f1e90027070 con 0x55af17e15fa0 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e9fd8f740 10 client.17130 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e9fd8f740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e9fd8f740 20 client.17130 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e8cf00 0x55af17e8f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e9fd8f740 4 client.17130 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.616+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af17d83a40 con 0x55af17e8cf00 2024-04-13T05:25:51.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.617+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e8cf00 0x55af17e8f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.618+0000 7f1e77e00640 10 client.17130 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:51.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.618+0000 7f1e95e00640 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e8cf00 0x55af17e8f320 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578e27868d52a0c9 server_cookie=b822eb479880633c in_seq=0 out_seq=0 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e77e00640 1 -- 172.21.15.57:0/1229307962 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af17d83a40 con 0x55af17e8cf00 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e77e00640 10 client.17130 handle_command_reply: tid=0 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 client.17130 shutdown 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 msgr2=0x55af17e8ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e8ab40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1e900129a0 tx=0x7f1e90004a40 comp rx=0 tx=0).stop 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 shutdown_connections 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55af17e8cf00 0x55af17e8f320 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55af17e15fa0 0x55af17e8ab40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af17e10e30 0x55af17e1f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 --2- 172.21.15.57:0/1229307962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af17e10440 0x55af17e1ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 >> 172.21.15.57:0/1229307962 conn(0x55af17e0b9b0 msgr2=0x55af17e0ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.620+0000 7f1e77e00640 0 client.17130 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 shutdown_connections 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e9fd8f740 1 -- 172.21.15.57:0/1229307962 wait complete. 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e9fd8f740 20 client.17130 trim_cache size 0 max 16384 2024-04-13T05:25:51.624 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:25:51.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1e7c0a3bb0 msgr2=0x7f1e7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e97200640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1e7c0a3bb0 0x7f1e7c0a6070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55af17e11690 tx=0x7f1e90009500 comp rx=0 tx=0).stop 2024-04-13T05:25:51.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 msgr2=0x7f1e9819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:51.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.621+0000 7f1e97200640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 0x7f1e9819a860 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1e84000c00 tx=0x7f1e84005e30 comp rx=0 tx=0).stop 2024-04-13T05:25:51.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 shutdown_connections 2024-04-13T05:25:51.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1e7c0a3bb0 0x7f1e7c0a6070 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e98109d80 0x7f1e981a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e98075fa0 0x7f1e9819a860 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 --2- 172.21.15.57:0/1146723849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e980749c0 0x7f1e9819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 >> 172.21.15.57:0/1146723849 conn(0x7f1e980fd460 msgr2=0x7f1e98105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 shutdown_connections 2024-04-13T05:25:51.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:51.622+0000 7f1e97200640 1 -- 172.21.15.57:0/1146723849 wait complete. 2024-04-13T05:25:51.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:51 smithi083 ceph-mon[35133]: pgmap v703: 129 pgs: 129 active+clean; 460 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 126 KiB/s wr, 40 op/s 2024-04-13T05:25:51.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:51 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1838595560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:25:52.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:25:52.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:25:52.646 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:25:52.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 -- 172.21.15.57:0/2074166037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8101e80 msgr2=0x7f53c81022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 --2- 172.21.15.57:0/2074166037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8101e80 0x7f53c81022a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009900 tx=0x7f53b802bc30 comp rx=0 tx=0).stop 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 -- 172.21.15.57:0/2074166037 shutdown_connections 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 --2- 172.21.15.57:0/2074166037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8103a20 0x7f53c810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 --2- 172.21.15.57:0/2074166037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 0x7f53c81034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 --2- 172.21.15.57:0/2074166037 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8101e80 0x7f53c81022a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.894+0000 7f53c7200640 1 -- 172.21.15.57:0/2074166037 >> 172.21.15.57:0/2074166037 conn(0x7f53c80fd460 msgr2=0x7f53c80ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.895+0000 7f53c7200640 1 -- 172.21.15.57:0/2074166037 shutdown_connections 2024-04-13T05:25:52.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.895+0000 7f53c7200640 1 -- 172.21.15.57:0/2074166037 wait complete. 2024-04-13T05:25:52.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 Processor -- start 2024-04-13T05:25:52.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 -- start start 2024-04-13T05:25:52.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8101e80 0x7f53c81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 0x7f53c81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 0x7f53c819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f53c810da70 con 0x7f53c8103a20 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53c810d8f0 con 0x7f53c8101e80 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.896+0000 7f53c7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53c810dbf0 con 0x7f53c8103060 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 0x7f53c819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 0x7f53c819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57094/0 (socket says 172.21.15.57:57094) 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 -- 172.21.15.57:0/2876125672 learned_addr learned my addr 172.21.15.57:0/2876125672 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c5e00640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8101e80 0x7f53c81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c5400640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 0x7f53c81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 -- 172.21.15.57:0/2876125672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 msgr2=0x7f53c81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 0x7f53c81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 -- 172.21.15.57:0/2876125672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8101e80 msgr2=0x7f53c81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8101e80 0x7f53c81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 -- 172.21.15.57:0/2876125672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c819fcd0 con 0x7f53c8103a20 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c5400640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 0x7f53c81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.897+0000 7f53c6800640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 0x7f53c819f660 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f53c000cce0 tx=0x7f53c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.898+0000 7f53bea00640 1 -- 172.21.15.57:0/2876125672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c000f450 con 0x7f53c8103a20 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.898+0000 7f53bea00640 1 -- 172.21.15.57:0/2876125672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c000faa0 con 0x7f53c8103a20 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.898+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c819ffc0 con 0x7f53c8103a20 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.898+0000 7f53bea00640 1 -- 172.21.15.57:0/2876125672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0018b70 con 0x7f53c8103a20 2024-04-13T05:25:52.902 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.898+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81a0480 con 0x7f53c8103a20 2024-04-13T05:25:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.901+0000 7f53cfdb9740 1 Processor -- start 2024-04-13T05:25:52.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.901+0000 7f53bea00640 1 -- 172.21.15.57:0/2876125672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f53c0018d10 con 0x7f53c8103a20 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 -- start start 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af094c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af099d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af09c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5586aeff6bc0 con 0x5586af094830 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5586aef9d660 con 0x5586af095220 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586aeff2b80 con 0x5586af09a390 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53c5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af094c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53c5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af094c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57100/0 (socket says 172.21.15.57:57100) 2024-04-13T05:25:52.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.902+0000 7f53c5400640 1 -- 172.21.15.57:0/3873849017 learned_addr learned my addr 172.21.15.57:0/3873849017 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:52.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5e00640 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af099d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af09c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 -- 172.21.15.57:0/3873849017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 msgr2=0x5586af09c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af09c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 -- 172.21.15.57:0/3873849017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 msgr2=0x5586af099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af099d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 -- 172.21.15.57:0/3873849017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586aefc0220 con 0x5586af094830 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5e00640 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af099d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53c5400640 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af094c50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f53b400bfa0 tx=0x7f53b400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727cea1b83e97c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.903+0000 7f53bea00640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53ac0a3c30 0x7f53ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.904+0000 7f53b3400640 1 -- 172.21.15.57:0/3873849017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4005020 con 0x5586af094830 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.904+0000 7f53bea00640 1 -- 172.21.15.57:0/2876125672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f53c00d0be0 con 0x7f53c8103a20 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.904+0000 7f53b3400640 1 -- 172.21.15.57:0/3873849017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b40047a0 con 0x5586af094830 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.904+0000 7f53b3400640 1 -- 172.21.15.57:0/3873849017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b400f650 con 0x5586af094830 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.904+0000 7f53c5e00640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53ac0a3c30 0x7f53ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3873849017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 msgr2=0x5586af094c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af094c50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f53b400bfa0 tx=0x7f53b400b730 comp rx=0 tx=0).stop 2024-04-13T05:25:52.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53c5e00640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53ac0a3c30 0x7f53ac0a60f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f53b802c240 tx=0x7f53b802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3873849017 shutdown_connections 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af099d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af09c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3873849017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af094c50 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3873849017 >> 172.21.15.57:0/3873849017 conn(0x5586af08fda0 msgr2=0x5586af0901a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3873849017 shutdown_connections 2024-04-13T05:25:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3873849017 wait complete. 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.905+0000 7f53cfdb9740 1 -- start start 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 Processor -- start 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af09ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5586af09d340 con 0x5586af094830 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5586aefb0a30 con 0x5586af095220 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586af09d4c0 con 0x5586af09a390 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53cfdb9740 10 client.0 _ll_register_callbacks cb 0x5586aed08c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53c5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af09ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53c5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af09ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57110/0 (socket says 172.21.15.57:57110) 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.906+0000 7f53c5e00640 1 -- 172.21.15.57:0/2820803975 learned_addr learned my addr 172.21.15.57:0/2820803975 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5400640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c6800640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5e00640 1 -- 172.21.15.57:0/2820803975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 msgr2=0x5586af0a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5e00640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af0a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5e00640 1 -- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 msgr2=0x5586af09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5e00640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5e00640 1 -- 172.21.15.57:0/2820803975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586aedd17e0 con 0x5586af094830 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5400640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53c5e00640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af09ed80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f53c8102bd0 tx=0x7f53b802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad737029b5c6679a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.907+0000 7f53a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53a7e00640 1 -- 172.21.15.57:0/2820803975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b804e030 con 0x5586af094830 2024-04-13T05:25:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53a7e00640 1 -- 172.21.15.57:0/2820803975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f53b80048d0 con 0x5586af094830 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b40047a0 con 0x5586af094830 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53a7e00640 1 -- 172.21.15.57:0/2820803975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53b805b050 con 0x5586af094830 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53cfdb9740 10 client.17142 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5586aee31f70 con 0x5586af094830 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.908+0000 7f53a7e00640 1 -- 172.21.15.57:0/2820803975 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b40047a0 con 0x5586af094830 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53a7e00640 1 -- 172.21.15.57:0/2820803975 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f53b8003b20 con 0x5586af094830 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53cfdb9740 10 client.17142 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53cfdb9740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53cfdb9740 20 client.17142 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af0aacf0 0x5586af0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53cfdb9740 4 client.17142 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586af0045e0 con 0x5586af0aacf0 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.909+0000 7f53c5400640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af0aacf0 0x5586af0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.910+0000 7f53c5400640 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af0aacf0 0x5586af0ad170 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38729898f572a2e server_cookie=19cb180be2037394 in_seq=0 out_seq=0 2024-04-13T05:25:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.910+0000 7f53a7e00640 10 client.17142 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53a7e00640 1 -- 172.21.15.57:0/2820803975 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586af0045e0 con 0x5586af0aacf0 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53a7e00640 10 client.17142 handle_command_reply: tid=0 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 client.17142 shutdown 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 msgr2=0x5586af09ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af09ed80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f53c8102bd0 tx=0x7f53b802bc30 comp rx=0 tx=0).stop 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 shutdown_connections 2024-04-13T05:25:52.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af0aacf0 0x5586af0ad170 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af09a390 0x5586af0a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/2820803975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af094830 0x5586af09ed80 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 >> 172.21.15.57:0/2820803975 conn(0x5586af08fda0 msgr2=0x5586af08ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53a7e00640 0 client.17142 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 shutdown_connections 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 1 -- 172.21.15.57:0/2820803975 wait complete. 2024-04-13T05:25:52.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.914+0000 7f53cfdb9740 20 client.17142 trim_cache size 0 max 16384 2024-04-13T05:25:52.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 Processor -- start 2024-04-13T05:25:52.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 -- start start 2024-04-13T05:25:52.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af094830 0x5586af09f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af095220 0x5586af0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af093710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81aaf60 con 0x5586af09a390 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81a87a0 con 0x5586af094830 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.935+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81a65f0 con 0x5586af095220 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af093710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af093710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57120/0 (socket says 172.21.15.57:57120) 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 -- 172.21.15.57:0/669245211 learned_addr learned my addr 172.21.15.57:0/669245211 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c5e00640 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af094830 0x5586af09f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c5400640 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af095220 0x5586af0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 -- 172.21.15.57:0/669245211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af095220 msgr2=0x5586af0a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af095220 0x5586af0a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 -- 172.21.15.57:0/669245211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af094830 msgr2=0x5586af09f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af094830 0x5586af09f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c6800640 1 -- 172.21.15.57:0/669245211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586aecc9170 con 0x5586af09a390 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.936+0000 7f53c5400640 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af095220 0x5586af0a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:52.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.937+0000 7f53c6800640 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af093710 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f53c000ccb0 tx=0x7f53c000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45f1343ca5b2a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:52.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.937+0000 7f53b2a00640 1 -- 172.21.15.57:0/669245211 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a0ed0 con 0x5586af09a390 2024-04-13T05:25:52.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.937+0000 7f53b2a00640 1 -- 172.21.15.57:0/669245211 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c00b6070 con 0x5586af09a390 2024-04-13T05:25:52.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.937+0000 7f53b2a00640 1 -- 172.21.15.57:0/669245211 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c009cfe0 con 0x5586af09a390 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 -- 172.21.15.57:0/669245211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 msgr2=0x5586af093710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af093710 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f53c000ccb0 tx=0x7f53c000fd10 comp rx=0 tx=0).stop 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 -- 172.21.15.57:0/669245211 shutdown_connections 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af093710 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af095220 0x5586af0a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/669245211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af094830 0x5586af09f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 -- 172.21.15.57:0/669245211 >> 172.21.15.57:0/669245211 conn(0x5586af08fda0 msgr2=0x5586af093000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:52.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 -- 172.21.15.57:0/669245211 shutdown_connections 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 -- 172.21.15.57:0/669245211 wait complete. 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 -- start start 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.938+0000 7f53cfdb9740 1 Processor -- start 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af094830 0x5586af0a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af0a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af0a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81a6c60 con 0x5586af09a390 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81a80d0 con 0x5586af095220 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81a7330 con 0x5586af094830 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53cfdb9740 10 client.0 _ll_register_callbacks cb 0x7f53b8042050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53c6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af0a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53c6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af0a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57136/0 (socket says 172.21.15.57:57136) 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53c5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af0a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.939+0000 7f53c6800640 1 -- 172.21.15.57:0/3284646407 learned_addr learned my addr 172.21.15.57:0/3284646407 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:52.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c5e00640 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af094830 0x5586af0a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c6800640 1 -- 172.21.15.57:0/3284646407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af094830 msgr2=0x5586af0a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c6800640 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af094830 0x5586af0a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c6800640 1 -- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 msgr2=0x5586af0a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c6800640 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af0a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c6800640 1 -- 172.21.15.57:0/3284646407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586aefb1be0 con 0x5586af09a390 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53c6800640 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af0a4aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f53c00a5260 tx=0x7f53c00185b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ee8ac853f90d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53a7e00640 1 -- 172.21.15.57:0/3284646407 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c007f070 con 0x5586af09a390 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.940+0000 7f53a7e00640 1 -- 172.21.15.57:0/3284646407 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f53c00a5300 con 0x5586af09a390 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c00b6070 con 0x5586af09a390 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53a7e00640 1 -- 172.21.15.57:0/3284646407 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53c00a0ed0 con 0x5586af09a390 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 10 client.17148 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:52.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5586af111d80 con 0x5586af09a390 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53a7e00640 1 -- 172.21.15.57:0/3284646407 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c00b6070 con 0x5586af09a390 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53a7e00640 1 -- 172.21.15.57:0/3284646407 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f53c00b8070 con 0x5586af09a390 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 10 client.17148 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 20 client.17148 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af112790 0x5586af114bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 4 client.17148 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.941+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586aeffe450 con 0x5586af112790 2024-04-13T05:25:52.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.942+0000 7f53c5e00640 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af112790 0x5586af114bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:52.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.943+0000 7f53c5e00640 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af112790 0x5586af114bb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99cea1f3329cd433 server_cookie=c5bc9dccad489dee in_seq=0 out_seq=0 2024-04-13T05:25:52.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.943+0000 7f53a7e00640 10 client.17148 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53a7e00640 1 -- 172.21.15.57:0/3284646407 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586aeffe450 con 0x5586af112790 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53a7e00640 10 client.17148 handle_command_reply: tid=0 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 client.17148 shutdown 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 msgr2=0x5586af0a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af0a4aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f53c00a5260 tx=0x7f53c00185b0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 shutdown_connections 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5586af112790 0x5586af114bb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586af095220 0x5586af0a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586af09a390 0x5586af0a4aa0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53a7e00640 0 client.17148 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 --2- 172.21.15.57:0/3284646407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586af094830 0x5586af0a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 >> 172.21.15.57:0/3284646407 conn(0x5586af08fda0 msgr2=0x5586af09f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 shutdown_connections 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 1 -- 172.21.15.57:0/3284646407 wait complete. 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.948+0000 7f53cfdb9740 20 client.17148 trim_cache size 0 max 16384 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "6e1ffafa-6252-4882-aca6-bd984132bff5", 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:25:52.952 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53ac0a3c30 msgr2=0x7f53ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53ac0a3c30 0x7f53ac0a60f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f53b802c240 tx=0x7f53b802c7e0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 msgr2=0x7f53c819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 0x7f53c819f660 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f53c000cce0 tx=0x7f53c000a620 comp rx=0 tx=0).stop 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 shutdown_connections 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f53ac0a3c30 0x7f53ac0a60f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c8103a20 0x7f53c819f660 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53c8103060 0x7f53c81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 --2- 172.21.15.57:0/2876125672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8101e80 0x7f53c81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:52.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.949+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 >> 172.21.15.57:0/2876125672 conn(0x7f53c80fd460 msgr2=0x7f53c81063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:52.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.950+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 shutdown_connections 2024-04-13T05:25:52.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:52.950+0000 7f53c7200640 1 -- 172.21.15.57:0/2876125672 wait complete. 2024-04-13T05:25:52.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:25:52.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:25:53.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 -- 172.21.15.57:0/3145797473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 msgr2=0x7f5de8076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3145797473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 0x7f5de8076da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009900 tx=0x7f5dd802bc40 comp rx=0 tx=0).stop 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 -- 172.21.15.57:0/3145797473 shutdown_connections 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3145797473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8077bc0 0x7f5de8078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3145797473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8078560 0x7f5de8078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3145797473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 0x7f5de8076da0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.246+0000 7f5dedc00640 1 -- 172.21.15.57:0/3145797473 >> 172.21.15.57:0/3145797473 conn(0x7f5de80ff7b0 msgr2=0x7f5de8101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:53.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.247+0000 7f5dedc00640 1 -- 172.21.15.57:0/3145797473 shutdown_connections 2024-04-13T05:25:53.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.247+0000 7f5dedc00640 1 -- 172.21.15.57:0/3145797473 wait complete. 2024-04-13T05:25:53.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.249+0000 7f5dedc00640 1 Processor -- start 2024-04-13T05:25:53.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.249+0000 7f5dedc00640 1 -- start start 2024-04-13T05:25:53.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.249+0000 7f5dedc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 0x7f5de819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.249+0000 7f5dedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8077bc0 0x7f5de819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5dedc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8078560 0x7f5de81a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5dedc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81141e0 con 0x7f5de8077bc0 2024-04-13T05:25:53.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5dedc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8114060 con 0x7f5de8078560 2024-04-13T05:25:53.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5dedc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8114360 con 0x7f5de8076980 2024-04-13T05:25:53.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5de6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8077bc0 0x7f5de819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5de7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8078560 0x7f5de81a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5de7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8078560 0x7f5de81a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47836/0 (socket says 172.21.15.57:47836) 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5de7400640 1 -- 172.21.15.57:0/3487765121 learned_addr learned my addr 172.21.15.57:0/3487765121 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.250+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 0x7f5de819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.251+0000 7f5de6000640 1 -- 172.21.15.57:0/3487765121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 msgr2=0x7f5de819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.251+0000 7f5de6000640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 0x7f5de819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.251+0000 7f5de6000640 1 -- 172.21.15.57:0/3487765121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8078560 msgr2=0x7f5de81a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.251+0000 7f5de6000640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8078560 0x7f5de81a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.251+0000 7f5de6000640 1 -- 172.21.15.57:0/3487765121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de81a64d0 con 0x7f5de8077bc0 2024-04-13T05:25:53.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.252+0000 7f5de6000640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8077bc0 0x7f5de819edf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b3f0 tx=0x7f5dd400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:53.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.252+0000 7f5ddf400640 1 -- 172.21.15.57:0/3487765121 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4015020 con 0x7f5de8077bc0 2024-04-13T05:25:53.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.253+0000 7f5ddf400640 1 -- 172.21.15.57:0/3487765121 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd4011ce0 con 0x7f5de8077bc0 2024-04-13T05:25:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.253+0000 7f5ddf400640 1 -- 172.21.15.57:0/3487765121 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4010d80 con 0x7f5de8077bc0 2024-04-13T05:25:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.253+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de81a67c0 con 0x7f5de8077bc0 2024-04-13T05:25:53.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.253+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de81a6d40 con 0x7f5de8077bc0 2024-04-13T05:25:53.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.261+0000 7f5ddf400640 1 -- 172.21.15.57:0/3487765121 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5dd4018020 con 0x7f5de8077bc0 2024-04-13T05:25:53.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.264+0000 7f5ddf400640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5dcc0a3cb0 0x7f5dcc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.264+0000 7f5ddf400640 1 -- 172.21.15.57:0/3487765121 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5dd40cffb0 con 0x7f5de8077bc0 2024-04-13T05:25:53.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.265+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5dcc0a3cb0 0x7f5dcc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.266+0000 7f5df0bb0740 1 Processor -- start 2024-04-13T05:25:53.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.267+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5dcc0a3cb0 0x7f5dcc0a6170 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8006010 tx=0x7f5dd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.267+0000 7f5df0bb0740 1 -- start start 2024-04-13T05:25:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.269+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056b94d0 0x55d1056b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.269+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 0x55d1056be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.269+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056bf030 0x55d1056c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.269+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d105620020 con 0x55d1056b94d0 2024-04-13T05:25:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.269+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1055c66d0 con 0x55d1056b9ec0 2024-04-13T05:25:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.269+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d10561bfe0 con 0x55d1056bf030 2024-04-13T05:25:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.270+0000 7f5de6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 0x55d1056be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.270+0000 7f5de6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 0x55d1056be9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47842/0 (socket says 172.21.15.57:47842) 2024-04-13T05:25:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.270+0000 7f5de6000640 1 -- 172.21.15.57:0/1698066261 learned_addr learned my addr 172.21.15.57:0/1698066261 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.271+0000 7f5de6a00640 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056b94d0 0x55d1056b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.272+0000 7f5de7400640 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056bf030 0x55d1056c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.272+0000 7f5de6000640 1 -- 172.21.15.57:0/1698066261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056bf030 msgr2=0x55d1056c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.272+0000 7f5de6000640 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056bf030 0x55d1056c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.272+0000 7f5de6000640 1 -- 172.21.15.57:0/1698066261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056b94d0 msgr2=0x55d1056b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.273+0000 7f5de6000640 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056b94d0 0x55d1056b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.273+0000 7f5de6000640 1 -- 172.21.15.57:0/1698066261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1053fc7d0 con 0x55d1056b9ec0 2024-04-13T05:25:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.273+0000 7f5de6a00640 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056b94d0 0x55d1056b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:25:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.273+0000 7f5de6000640 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 0x55d1056be9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4098860 tx=0x7f5dd40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62bda39d95c2ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.274+0000 7f5dd3e00640 1 -- 172.21.15.57:0/1698066261 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd409c590 con 0x55d1056b9ec0 2024-04-13T05:25:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.274+0000 7f5dd3e00640 1 -- 172.21.15.57:0/1698066261 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd409be90 con 0x55d1056b9ec0 2024-04-13T05:25:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.274+0000 7f5dd3e00640 1 -- 172.21.15.57:0/1698066261 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40b5220 con 0x55d1056b9ec0 2024-04-13T05:25:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.276+0000 7f5df0bb0740 1 -- 172.21.15.57:0/1698066261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 msgr2=0x55d1056be9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.277+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 0x55d1056be9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4098860 tx=0x7f5dd40046c0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.278+0000 7f5df0bb0740 1 -- 172.21.15.57:0/1698066261 shutdown_connections 2024-04-13T05:25:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.278+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b9ec0 0x55d1056be9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.278+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056bf030 0x55d1056c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.278+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/1698066261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056b94d0 0x55d1056b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.278+0000 7f5df0bb0740 1 -- 172.21.15.57:0/1698066261 >> 172.21.15.57:0/1698066261 conn(0x55d1056b4a40 msgr2=0x55d1056b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.279+0000 7f5df0bb0740 1 -- 172.21.15.57:0/1698066261 shutdown_connections 2024-04-13T05:25:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.279+0000 7f5df0bb0740 1 -- 172.21.15.57:0/1698066261 wait complete. 2024-04-13T05:25:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.280+0000 7f5df0bb0740 1 -- start start 2024-04-13T05:25:53.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.281+0000 7f5df0bb0740 1 Processor -- start 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 0x55d1056c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d1056c1fe0 con 0x55d1056bf030 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1055d9a90 con 0x55d1056b94d0 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d1056c2160 con 0x55d1056b9ec0 2024-04-13T05:25:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5df0bb0740 10 client.0 _ll_register_callbacks cb 0x55d1055d9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.283+0000 7f5de6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 0x55d1056c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 0x55d1056c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:39566/0 (socket says 172.21.15.57:39566) 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 -- 172.21.15.57:0/2018123200 learned_addr learned my addr 172.21.15.57:0/2018123200 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de7400640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 -- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 msgr2=0x55d1056c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 -- 172.21.15.57:0/2018123200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 msgr2=0x55d1056cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de6000640 1 -- 172.21.15.57:0/2018123200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1053323f0 con 0x55d1056b9ec0 2024-04-13T05:25:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.284+0000 7f5de7400640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:25:53.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.285+0000 7f5de6000640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 0x55d1056c3dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400bdb0 tx=0x7f5dd40a4b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7b3712b4ce07e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:53.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.285+0000 7f5dd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:25:53.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.285+0000 7f5dd0c00640 1 -- 172.21.15.57:0/2018123200 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40b9070 con 0x55d1056b9ec0 2024-04-13T05:25:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.285+0000 7f5dd0c00640 1 -- 172.21.15.57:0/2018123200 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5dd40b5240 con 0x55d1056b9ec0 2024-04-13T05:25:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.285+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd409be90 con 0x55d1056b9ec0 2024-04-13T05:25:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.285+0000 7f5dd0c00640 1 -- 172.21.15.57:0/2018123200 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5dd4085050 con 0x55d1056b9ec0 2024-04-13T05:25:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.286+0000 7f5df0bb0740 10 client.26317 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.286+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1052f20f0 con 0x55d1056b9ec0 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.286+0000 7f5dd0c00640 1 -- 172.21.15.57:0/2018123200 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd409be90 con 0x55d1056b9ec0 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5dd0c00640 1 -- 172.21.15.57:0/2018123200 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f5dd409b2e0 con 0x55d1056b9ec0 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5df0bb0740 10 client.26317 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5df0bb0740 10 client.26317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5df0bb0740 20 client.26317 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d1056cf300 0x55d1056d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5df0bb0740 4 client.26317 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:25:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.287+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1055bf600 con 0x55d1056cf300 2024-04-13T05:25:53.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.288+0000 7f5de6a00640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d1056cf300 0x55d1056d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.292+0000 7f5de6a00640 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d1056cf300 0x55d1056d1750 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469c344dd7ec15ed server_cookie=d163da1a9f7c442 in_seq=0 out_seq=0 2024-04-13T05:25:53.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.292+0000 7f5dd0c00640 10 client.26317 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:53.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5dd0c00640 1 -- 172.21.15.57:0/2018123200 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1055bf600 con 0x55d1056cf300 2024-04-13T05:25:53.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5dd0c00640 10 client.26317 handle_command_reply: tid=0 2024-04-13T05:25:53.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 client.26317 shutdown 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 msgr2=0x55d1056c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 0x55d1056c3dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400bdb0 tx=0x7f5dd40a4b00 comp rx=0 tx=0).stop 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 shutdown_connections 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d1056cf300 0x55d1056d1750 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b9ec0 0x55d1056c3dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/2018123200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 >> 172.21.15.57:0/2018123200 conn(0x55d1056b4a40 msgr2=0x55d1056b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:53.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.299+0000 7f5dd0c00640 0 client.26317 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:53.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.300+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 shutdown_connections 2024-04-13T05:25:53.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.300+0000 7f5df0bb0740 1 -- 172.21.15.57:0/2018123200 wait complete. 2024-04-13T05:25:53.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.300+0000 7f5df0bb0740 20 client.26317 trim_cache size 0 max 16384 2024-04-13T05:25:53.314 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:25:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:25:53.005+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001a4d [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44/ auth{2=1} v81253 ap=2 DIRTYPARENT f(v12 m2024-04-13T05:23:31.482085+0000 85=85+0) n(v5 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@359} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06149180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x20000001a4d:[<0x2000000021a/blog-44 v81253>,<0x10000000208/blogtest_in v15593>,<0x10000000201/src v2855>,<0x10000000006/blogbench-1.0 v2723>,<0x10000000005/tmp v182>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:23:31.370751+0000 86=86+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1)","memory_value.dirstat":"f(v12 m2024-04-13T05:23:31.482085+0000 85=85+0)","memory_value.rstat":"n(v5 rc2024-04-13T05:23:31.482085+0000 b1755820 86=85+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-13T05:25:53.314 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:25:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:25:53.012+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000021b [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0/ auth{0=1,2=1} fragtree_t(*^3) v47842 ap=2 f(v342 m2024-04-13T05:21:28.256974+0000 273=273+0) n(v10 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1) (inest sync->mix g=0,2) caps={15447=pAsLsXsFs/-@1610} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x559b052eb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x2000000021b:[<0x2000000021a/blog-0 v47842>,<0x10000000208/blogtest_in v11726>,<0x10000000201/src v2821>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x2000000021b:[<0x2000000021a/blog-0 v47842>,<0x10000000208/blogtest_in v15593>,<0x10000000201/src v2855>,<0x10000000006/blogbench-1.0 v2723>,<0x10000000005/tmp v182>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:21:28.196434+0000 274=274+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1)","memory_value.dirstat":"f(v342 m2024-04-13T05:21:28.256974+0000 273=273+0)","memory_value.rstat":"n(v10 rc2024-04-13T05:21:28.256974+0000 b5102842 274=273+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:25:53.314 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:25:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:25:53.071+0000 7f34a7600700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001100 [...2,head] /volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25/ auth{2=1} v73177 ap=2 f(v16 m2024-04-13T05:22:48.044991+0000 79=79+0) n(v5 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1) (inest sync->mix g=2) caps={15447=pAsLsXsFs/-@200} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x559b06366580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x20000001100:[<0x2000000021a/blog-25 v73177>,<0x10000000208/blogtest_in v13864>,<0x10000000201/src v2827>,<0x10000000006/blogbench-1.0 v2595>,<0x10000000005/tmp v97>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","memoryvalue":"(2)0x20000001100:[<0x2000000021a/blog-25 v73177>,<0x10000000208/blogtest_in v15593>,<0x10000000201/src v2855>,<0x10000000006/blogbench-1.0 v2723>,<0x10000000005/tmp v182>,<0x10000000003/client.0 v91>,<0x10000000002/695f39a4-c9d5-4816-845e-3fe961904a07 v109>,<0x10000000001/sv_0 v125>,<0x10000000000/qa v93>,<0x1/volumes v92>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-13T05:22:47.912987+0000 80=80+0)","ondisk_value.rstat":"n(v0 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1)","memory_value.dirstat":"f(v16 m2024-04-13T05:22:48.044991+0000 79=79+0)","memory_value.rstat":"n(v5 rc2024-04-13T05:22:48.044991+0000 b1611176 80=79+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-13T05:25:53.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.328+0000 7f5df0bb0740 1 Processor -- start 2024-04-13T05:25:53.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.328+0000 7f5df0bb0740 1 -- start start 2024-04-13T05:25:53.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 0x55d1056b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056bf030 0x55d1056cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056cbf70 0x55d1056b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81b3c80 con 0x55d1056cbf70 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81b14c0 con 0x55d1056bf030 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81af310 con 0x55d1056b94d0 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5de6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 0x55d1056b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5de6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 0x55d1056b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:39570/0 (socket says 172.21.15.57:39570) 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5de6a00640 1 -- 172.21.15.57:0/3073460666 learned_addr learned my addr 172.21.15.57:0/3073460666 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:53.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5de7400640 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056cbf70 0x55d1056b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.329+0000 7f5de6000640 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056bf030 0x55d1056cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.330+0000 7f5de6a00640 1 -- 172.21.15.57:0/3073460666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056bf030 msgr2=0x55d1056cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.330+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056bf030 0x55d1056cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.330+0000 7f5de6a00640 1 -- 172.21.15.57:0/3073460666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056cbf70 msgr2=0x55d1056b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.330+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056cbf70 0x55d1056b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.330+0000 7f5de6a00640 1 -- 172.21.15.57:0/3073460666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1056be530 con 0x55d1056b94d0 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.330+0000 7f5de7400640 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056cbf70 0x55d1056b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:25:53.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.331+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 0x55d1056b32f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005c40 tx=0x7f5dd8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e7848f15c22c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:53.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.331+0000 7f5dd3400640 1 -- 172.21.15.57:0/3073460666 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8005300 con 0x55d1056b94d0 2024-04-13T05:25:53.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.331+0000 7f5dd3400640 1 -- 172.21.15.57:0/3073460666 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd80054a0 con 0x55d1056b94d0 2024-04-13T05:25:53.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.331+0000 7f5dd3400640 1 -- 172.21.15.57:0/3073460666 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004ae0 con 0x55d1056b94d0 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3073460666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 msgr2=0x55d1056b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 0x55d1056b32f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005c40 tx=0x7f5dd8009260 comp rx=0 tx=0).stop 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3073460666 shutdown_connections 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056bf030 0x55d1056cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056cbf70 0x55d1056b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3073460666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056b94d0 0x55d1056b32f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3073460666 >> 172.21.15.57:0/3073460666 conn(0x55d1056b4a40 msgr2=0x55d1056cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.332+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3073460666 shutdown_connections 2024-04-13T05:25:53.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.333+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3073460666 wait complete. 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.333+0000 7f5df0bb0740 1 -- start start 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.333+0000 7f5df0bb0740 1 Processor -- start 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056cbf70 0x55d1056c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81af980 con 0x55d1056bf030 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81b0df0 con 0x55d1056b94d0 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81b0050 con 0x55d1056cbf70 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5df0bb0740 10 client.0 _ll_register_callbacks cb 0x7f5dd40981e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5de6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5de6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:47864/0 (socket says 172.21.15.57:47864) 2024-04-13T05:25:53.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5de6a00640 1 -- 172.21.15.57:0/3215758312 learned_addr learned my addr 172.21.15.57:0/3215758312 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:25:53.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5de6000640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.334+0000 7f5de7400640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056cbf70 0x55d1056c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6a00640 1 -- 172.21.15.57:0/3215758312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056cbf70 msgr2=0x55d1056c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056cbf70 0x55d1056c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6a00640 1 -- 172.21.15.57:0/3215758312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 msgr2=0x55d1056c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6a00640 1 -- 172.21.15.57:0/3215758312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1056ca360 con 0x55d1056b94d0 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6000640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5de6a00640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c8870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8036040 tx=0x7f5dd8005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c200e96c8bc3d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.335+0000 7f5dd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.336+0000 7f5dd0c00640 1 -- 172.21.15.57:0/3215758312 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd805f070 con 0x55d1056b94d0 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.336+0000 7f5dd0c00640 1 -- 172.21.15.57:0/3215758312 <== mon.2 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5dd8004a60 con 0x55d1056b94d0 2024-04-13T05:25:53.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.336+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd80054a0 con 0x55d1056b94d0 2024-04-13T05:25:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.337+0000 7f5dd0c00640 1 -- 172.21.15.57:0/3215758312 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5dd8069050 con 0x55d1056b94d0 2024-04-13T05:25:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.337+0000 7f5df0bb0740 10 client.36476 fetch_fsmap learned FSMap version 32 2024-04-13T05:25:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.337+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d105735bb0 con 0x55d1056b94d0 2024-04-13T05:25:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.337+0000 7f5dd0c00640 1 -- 172.21.15.57:0/3215758312 <== mon.2 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd80054a0 con 0x55d1056b94d0 2024-04-13T05:25:53.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.337+0000 7f5dd0c00640 1 -- 172.21.15.57:0/3215758312 <== mon.2 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f5dd80035f0 con 0x55d1056b94d0 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5df0bb0740 10 client.36476 fetch_fsmap finished waiting for FSMap version 32 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5df0bb0740 10 client.36476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5df0bb0740 20 client.36476 populate_metadata read hostname 'smithi057' 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d105736630 0x55d105738a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5df0bb0740 4 client.36476 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1056c3680 con 0x55d105736630 2024-04-13T05:25:53.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.338+0000 7f5de6000640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d105736630 0x55d105738a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:25:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.339+0000 7f5de6000640 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d105736630 0x55d105738a50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f2636b76f3137d server_cookie=21f48c02f4566659 in_seq=0 out_seq=0 2024-04-13T05:25:53.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.339+0000 7f5dd0c00640 10 client.36476 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:53.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.341+0000 7f5dd0c00640 1 -- 172.21.15.57:0/3215758312 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55d1056c3680 con 0x55d105736630 2024-04-13T05:25:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.341+0000 7f5dd0c00640 10 client.36476 handle_command_reply: tid=0 2024-04-13T05:25:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.341+0000 7f5df0bb0740 1 client.36476 shutdown 2024-04-13T05:25:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.341+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 msgr2=0x55d1056c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.341+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c8870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8036040 tx=0x7f5dd8005700 comp rx=0 tx=0).stop 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 shutdown_connections 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55d105736630 0x55d105738a50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1056cbf70 0x55d1056c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1056bf030 0x55d1056c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 --2- 172.21.15.57:0/3215758312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1056b94d0 0x55d1056c8870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 >> 172.21.15.57:0/3215758312 conn(0x55d1056b4a40 msgr2=0x55d1056b7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5dd0c00640 0 client.36476 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 shutdown_connections 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 1 -- 172.21.15.57:0/3215758312 wait complete. 2024-04-13T05:25:53.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.342+0000 7f5df0bb0740 20 client.36476 trim_cache size 0 max 16384 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1167 inodes in the stack)", 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: "6e1ffafa-6252-4882-aca6-bd984132bff5": { 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6e1ffafa-6252-4882-aca6-bd984132bff5", 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:25:53.346 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5dcc0a3cb0 msgr2=0x7f5dcc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5dcc0a3cb0 0x7f5dcc0a6170 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8006010 tx=0x7f5dd8000bc0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8077bc0 msgr2=0x7f5de819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8077bc0 0x7f5de819edf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b3f0 tx=0x7f5dd400b8b0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 shutdown_connections 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5dcc0a3cb0 0x7f5dcc0a6170 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5de8078560 0x7f5de81a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5de8077bc0 0x7f5de819edf0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 --2- 172.21.15.57:0/3487765121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5de8076980 0x7f5de819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:25:53.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.343+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 >> 172.21.15.57:0/3487765121 conn(0x7f5de80ff7b0 msgr2=0x7f5de8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:25:53.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.344+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 shutdown_connections 2024-04-13T05:25:53.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:25:53.344+0000 7f5dedc00640 1 -- 172.21.15.57:0/3487765121 wait complete. 2024-04-13T05:25:53.366 INFO:tasks.cephfs.filesystem:scrub status for tag:6e1ffafa-6252-4882-aca6-bd984132bff5 - {'path': '/', 'tag': '6e1ffafa-6252-4882-aca6-bd984132bff5', 'options': 'recursive,force'} 2024-04-13T05:25:53.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:53 smithi189 ceph-mon[35531]: pgmap v704: 129 pgs: 129 active+clean; 458 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 119 KiB/s wr, 46 op/s 2024-04-13T05:25:53.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:53 smithi083 ceph-mon[35133]: pgmap v704: 129 pgs: 129 active+clean; 458 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 119 KiB/s wr, 46 op/s 2024-04-13T05:25:53.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:53 smithi057 ceph-mon[32331]: pgmap v704: 129 pgs: 129 active+clean; 458 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 119 KiB/s wr, 46 op/s 2024-04-13T05:25:54.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:25:54.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:54.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:54.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:54.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:25:54.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:54 smithi189 ceph-mon[35531]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:54 smithi083 ceph-mon[35133]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:25:54.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:54.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:54.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:54.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:25:54.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: Scrub error on inode 0x2000000021b (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.h log and `damage ls` output for details 2024-04-13T05:25:54.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:54 smithi057 ceph-mon[32331]: Scrub error on inode 0x20000001100 (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-25) see mds.h log and `damage ls` output for details 2024-04-13T05:25:55.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:55 smithi083 ceph-mon[35133]: pgmap v705: 129 pgs: 129 active+clean; 454 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 185 KiB/s wr, 96 op/s 2024-04-13T05:25:55.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:55 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:55 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:55 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:55 smithi083 ceph-mon[35133]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:25:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:55 smithi057 ceph-mon[32331]: pgmap v705: 129 pgs: 129 active+clean; 454 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 185 KiB/s wr, 96 op/s 2024-04-13T05:25:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:55 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:55 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:55 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:55 smithi057 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:25:55.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:55 smithi189 ceph-mon[35531]: pgmap v705: 129 pgs: 129 active+clean; 454 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 185 KiB/s wr, 96 op/s 2024-04-13T05:25:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:55 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:55 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:55 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:25:55.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:55 smithi189 ceph-mon[35531]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-13T05:25:57.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:57 smithi083 ceph-mon[35133]: pgmap v706: 129 pgs: 129 active+clean; 450 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 278 KiB/s wr, 192 op/s 2024-04-13T05:25:57.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:57 smithi057 ceph-mon[32331]: pgmap v706: 129 pgs: 129 active+clean; 450 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 278 KiB/s wr, 192 op/s 2024-04-13T05:25:57.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:57 smithi189 ceph-mon[35531]: pgmap v706: 129 pgs: 129 active+clean; 450 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 278 KiB/s wr, 192 op/s 2024-04-13T05:25:59.636 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:25:59 smithi189 ceph-mon[35531]: pgmap v707: 129 pgs: 129 active+clean; 449 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 275 KiB/s wr, 201 op/s 2024-04-13T05:25:59.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:25:59 smithi083 ceph-mon[35133]: pgmap v707: 129 pgs: 129 active+clean; 449 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 275 KiB/s wr, 201 op/s 2024-04-13T05:25:59.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:25:59 smithi057 ceph-mon[32331]: pgmap v707: 129 pgs: 129 active+clean; 449 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 275 KiB/s wr, 201 op/s 2024-04-13T05:26:01.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:01 smithi083 ceph-mon[35133]: pgmap v708: 129 pgs: 129 active+clean; 442 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 410 KiB/s wr, 294 op/s 2024-04-13T05:26:01.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:01 smithi057 ceph-mon[32331]: pgmap v708: 129 pgs: 129 active+clean; 442 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 410 KiB/s wr, 294 op/s 2024-04-13T05:26:01.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:01 smithi189 ceph-mon[35531]: pgmap v708: 129 pgs: 129 active+clean; 442 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 410 KiB/s wr, 294 op/s 2024-04-13T05:26:03.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:03 smithi083 ceph-mon[35133]: pgmap v709: 129 pgs: 129 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 325 KiB/s wr, 280 op/s 2024-04-13T05:26:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:03 smithi057 ceph-mon[32331]: pgmap v709: 129 pgs: 129 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 325 KiB/s wr, 280 op/s 2024-04-13T05:26:03.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:03 smithi189 ceph-mon[35531]: pgmap v709: 129 pgs: 129 active+clean; 440 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 325 KiB/s wr, 280 op/s 2024-04-13T05:26:04.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:04 smithi083 ceph-mon[35133]: pgmap v710: 129 pgs: 129 active+clean; 436 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 322 KiB/s wr, 284 op/s 2024-04-13T05:26:04.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:04 smithi057 ceph-mon[32331]: pgmap v710: 129 pgs: 129 active+clean; 436 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 322 KiB/s wr, 284 op/s 2024-04-13T05:26:04.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:04 smithi189 ceph-mon[35531]: pgmap v710: 129 pgs: 129 active+clean; 436 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 322 KiB/s wr, 284 op/s 2024-04-13T05:26:05.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:05 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:26:05.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:05 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:26:05.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:05 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:26:07.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:06 smithi083 ceph-mon[35133]: pgmap v711: 129 pgs: 129 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 388 KiB/s wr, 260 op/s 2024-04-13T05:26:07.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:06 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:06 smithi057 ceph-mon[32331]: pgmap v711: 129 pgs: 129 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 388 KiB/s wr, 260 op/s 2024-04-13T05:26:07.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:06 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:06 smithi189 ceph-mon[35531]: pgmap v711: 129 pgs: 129 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 388 KiB/s wr, 260 op/s 2024-04-13T05:26:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:07.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:06 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:08.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:07 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:26:08.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:07 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:26:08.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:26:08.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:26:08.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:07 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:07 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:26:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:07 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:26:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:26:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:26:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:07 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:08.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:07 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:26:08.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:07 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:26:08.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:26:08.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:26:08.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:07 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:26:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:08 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:26:09.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:08 smithi083 ceph-mon[35133]: pgmap v712: 129 pgs: 129 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 291 KiB/s wr, 165 op/s 2024-04-13T05:26:09.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:08 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:26:09.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:08 smithi057 ceph-mon[32331]: pgmap v712: 129 pgs: 129 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 291 KiB/s wr, 165 op/s 2024-04-13T05:26:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:08 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:26:09.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:08 smithi189 ceph-mon[35531]: pgmap v712: 129 pgs: 129 active+clean; 434 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 291 KiB/s wr, 165 op/s 2024-04-13T05:26:11.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:10 smithi083 ceph-mon[35133]: pgmap v713: 129 pgs: 129 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 372 KiB/s wr, 206 op/s 2024-04-13T05:26:11.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:10 smithi057 ceph-mon[32331]: pgmap v713: 129 pgs: 129 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 372 KiB/s wr, 206 op/s 2024-04-13T05:26:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:10 smithi189 ceph-mon[35531]: pgmap v713: 129 pgs: 129 active+clean; 425 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 372 KiB/s wr, 206 op/s 2024-04-13T05:26:13.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:12 smithi083 ceph-mon[35133]: pgmap v714: 129 pgs: 129 active+clean; 423 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 326 KiB/s wr, 126 op/s 2024-04-13T05:26:13.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:12 smithi057 ceph-mon[32331]: pgmap v714: 129 pgs: 129 active+clean; 423 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 326 KiB/s wr, 126 op/s 2024-04-13T05:26:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:12 smithi189 ceph-mon[35531]: pgmap v714: 129 pgs: 129 active+clean; 423 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 326 KiB/s wr, 126 op/s 2024-04-13T05:26:15.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:14 smithi083 ceph-mon[35133]: pgmap v715: 129 pgs: 129 active+clean; 419 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 305 KiB/s wr, 141 op/s 2024-04-13T05:26:15.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:14 smithi057 ceph-mon[32331]: pgmap v715: 129 pgs: 129 active+clean; 419 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 305 KiB/s wr, 141 op/s 2024-04-13T05:26:15.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:14 smithi189 ceph-mon[35531]: pgmap v715: 129 pgs: 129 active+clean; 419 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 305 KiB/s wr, 141 op/s 2024-04-13T05:26:17.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:16 smithi083 ceph-mon[35133]: pgmap v716: 129 pgs: 129 active+clean; 417 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 457 KiB/s wr, 156 op/s 2024-04-13T05:26:17.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:16 smithi057 ceph-mon[32331]: pgmap v716: 129 pgs: 129 active+clean; 417 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 457 KiB/s wr, 156 op/s 2024-04-13T05:26:17.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:16 smithi189 ceph-mon[35531]: pgmap v716: 129 pgs: 129 active+clean; 417 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 457 KiB/s wr, 156 op/s 2024-04-13T05:26:19.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:18 smithi083 ceph-mon[35133]: pgmap v717: 129 pgs: 129 active+clean; 416 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 325 KiB/s wr, 135 op/s 2024-04-13T05:26:19.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:18 smithi057 ceph-mon[32331]: pgmap v717: 129 pgs: 129 active+clean; 416 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 325 KiB/s wr, 135 op/s 2024-04-13T05:26:19.332 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:18 smithi189 ceph-mon[35531]: pgmap v717: 129 pgs: 129 active+clean; 416 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 325 KiB/s wr, 135 op/s 2024-04-13T05:26:21.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:20 smithi083 ceph-mon[35133]: pgmap v718: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 472 KiB/s wr, 198 op/s 2024-04-13T05:26:21.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:20 smithi057 ceph-mon[32331]: pgmap v718: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 472 KiB/s wr, 198 op/s 2024-04-13T05:26:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:20 smithi189 ceph-mon[35531]: pgmap v718: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 472 KiB/s wr, 198 op/s 2024-04-13T05:26:23.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:22 smithi083 ceph-mon[35133]: pgmap v719: 129 pgs: 129 active+clean; 410 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 410 KiB/s wr, 163 op/s 2024-04-13T05:26:23.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:22 smithi057 ceph-mon[32331]: pgmap v719: 129 pgs: 129 active+clean; 410 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 410 KiB/s wr, 163 op/s 2024-04-13T05:26:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:22 smithi189 ceph-mon[35531]: pgmap v719: 129 pgs: 129 active+clean; 410 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 410 KiB/s wr, 163 op/s 2024-04-13T05:26:23.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:26:23.368 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:26:23.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 -- 172.21.15.57:0/1599661878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8105280 msgr2=0x7f08a8107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 --2- 172.21.15.57:0/1599661878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8105280 0x7f08a8107670 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0890009900 tx=0x7f089002bc30 comp rx=0 tx=0).stop 2024-04-13T05:26:23.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 -- 172.21.15.57:0/1599661878 shutdown_connections 2024-04-13T05:26:23.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 --2- 172.21.15.57:0/1599661878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8107bb0 0x7f08a810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 --2- 172.21.15.57:0/1599661878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8105280 0x7f08a8107670 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 --2- 172.21.15.57:0/1599661878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08a8102950 0x7f08a8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 -- 172.21.15.57:0/1599661878 >> 172.21.15.57:0/1599661878 conn(0x7f08a80fc360 msgr2=0x7f08a80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 -- 172.21.15.57:0/1599661878 shutdown_connections 2024-04-13T05:26:23.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.603+0000 7f08ada00640 1 -- 172.21.15.57:0/1599661878 wait complete. 2024-04-13T05:26:23.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.605+0000 7f08ada00640 1 Processor -- start 2024-04-13T05:26:23.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.605+0000 7f08ada00640 1 -- start start 2024-04-13T05:26:23.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.605+0000 7f08ada00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 0x7f08a8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.605+0000 7f08ada00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08a8105280 0x7f08a8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08ada00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8107bb0 0x7f08a819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08ada00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810da40 con 0x7f08a8102950 2024-04-13T05:26:23.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08ada00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810d8c0 con 0x7f08a8105280 2024-04-13T05:26:23.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08ada00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810dbc0 con 0x7f08a8107bb0 2024-04-13T05:26:23.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 0x7f08a8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 0x7f08a8198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53552/0 (socket says 172.21.15.57:53552) 2024-04-13T05:26:23.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 -- 172.21.15.57:0/2042848204 learned_addr learned my addr 172.21.15.57:0/2042848204 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f089f400640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08a8105280 0x7f08a8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08ad000640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8107bb0 0x7f08a819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 -- 172.21.15.57:0/2042848204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8107bb0 msgr2=0x7f08a819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8107bb0 0x7f08a819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 -- 172.21.15.57:0/2042848204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08a8105280 msgr2=0x7f08a8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08a8105280 0x7f08a8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.606+0000 7f08a7e00640 1 -- 172.21.15.57:0/2042848204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a819fd30 con 0x7f08a8102950 2024-04-13T05:26:23.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.607+0000 7f08a7e00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 0x7f08a8198110 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f089800d950 tx=0x7f089800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.607+0000 7f08a5600640 1 -- 172.21.15.57:0/2042848204 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898017070 con 0x7f08a8102950 2024-04-13T05:26:23.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.607+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a819ffc0 con 0x7f08a8102950 2024-04-13T05:26:23.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.607+0000 7f08a5600640 1 -- 172.21.15.57:0/2042848204 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0898009c30 con 0x7f08a8102950 2024-04-13T05:26:23.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.607+0000 7f08a5600640 1 -- 172.21.15.57:0/2042848204 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089800f620 con 0x7f08a8102950 2024-04-13T05:26:23.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.607+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81a04d0 con 0x7f08a8102950 2024-04-13T05:26:23.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 Processor -- start 2024-04-13T05:26:23.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 -- start start 2024-04-13T05:26:23.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f5504f0 0x56252f554ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f557af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56252f4b2020 con 0x56252f555660 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56252f4586d0 con 0x56252f54fb00 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.612+0000 7f08b1e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56252f4adfe0 con 0x56252f5504f0 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f08a5600640 1 -- 172.21.15.57:0/2042848204 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f08980109a0 con 0x7f08a8102950 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f089f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f557af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f089f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f557af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53568/0 (socket says 172.21.15.57:53568) 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f089f400640 1 -- 172.21.15.57:0/1683190237 learned_addr learned my addr 172.21.15.57:0/1683190237 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f08ad000640 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f08a7e00640 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f5504f0 0x56252f554ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f089f400640 1 -- 172.21.15.57:0/1683190237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f5504f0 msgr2=0x56252f554ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.613+0000 7f089f400640 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f5504f0 0x56252f554ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f089f400640 1 -- 172.21.15.57:0/1683190237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 msgr2=0x56252f54ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f089f400640 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f089f400640 1 -- 172.21.15.57:0/1683190237 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252f28e7d0 con 0x56252f555660 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f08ad000640 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f089f400640 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f557af0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0890005ab0 tx=0x7f0890005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd383826f6feff8e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f08a5600640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f088c0a3bb0 0x7f088c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.614+0000 7f089d600640 1 -- 172.21.15.57:0/1683190237 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890044020 con 0x56252f555660 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.615+0000 7f08a5600640 1 -- 172.21.15.57:0/2042848204 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f08980d0440 con 0x7f08a8102950 2024-04-13T05:26:23.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.615+0000 7f089d600640 1 -- 172.21.15.57:0/1683190237 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f089002cbe0 con 0x56252f555660 2024-04-13T05:26:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.615+0000 7f089d600640 1 -- 172.21.15.57:0/1683190237 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890004920 con 0x56252f555660 2024-04-13T05:26:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.615+0000 7f08ad000640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f088c0a3bb0 0x7f088c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 -- 172.21.15.57:0/1683190237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 msgr2=0x56252f557af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f557af0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0890005ab0 tx=0x7f0890005dd0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 -- 172.21.15.57:0/1683190237 shutdown_connections 2024-04-13T05:26:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f557af0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f5504f0 0x56252f554ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1683190237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 -- 172.21.15.57:0/1683190237 >> 172.21.15.57:0/1683190237 conn(0x56252f546c60 msgr2=0x56252f547060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08ad000640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f088c0a3bb0 0x7f088c0a6070 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x56252f550d50 tx=0x7f08a0009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 -- 172.21.15.57:0/1683190237 shutdown_connections 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 -- 172.21.15.57:0/1683190237 wait complete. 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.616+0000 7f08b1e61740 1 -- start start 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 Processor -- start 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f5612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56252f558600 con 0x56252f555660 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56252f46ba90 con 0x56252f5504f0 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56252f558780 con 0x56252f54fb00 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08b1e61740 10 client.0 _ll_register_callbacks cb 0x56252f46b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.617+0000 7f08ad000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f5612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f5612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53582/0 (socket says 172.21.15.57:53582) 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 -- 172.21.15.57:0/69279537 learned_addr learned my addr 172.21.15.57:0/69279537 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f089f400640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08a7e00640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 -- 172.21.15.57:0/69279537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 msgr2=0x56252f559d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 -- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f5504f0 msgr2=0x56252f55a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 -- 172.21.15.57:0/69279537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252f1c43f0 con 0x56252f555660 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f08ad000640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f5612e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f08a00128e0 tx=0x7f08a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b1fee2b0e551b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.618+0000 7f0896000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.619+0000 7f0896000640 1 -- 172.21.15.57:0/69279537 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0012e60 con 0x56252f555660 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.619+0000 7f0896000640 1 -- 172.21.15.57:0/69279537 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f08a000ece0 con 0x56252f555660 2024-04-13T05:26:23.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.619+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f089002cbe0 con 0x56252f555660 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.619+0000 7f0896000640 1 -- 172.21.15.57:0/69279537 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08a0030050 con 0x56252f555660 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.619+0000 7f08b1e61740 10 client.17175 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.619+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56252f1840f0 con 0x56252f555660 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f0896000640 1 -- 172.21.15.57:0/69279537 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f089002cbe0 con 0x56252f555660 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f0896000640 1 -- 172.21.15.57:0/69279537 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f08a000f040 con 0x56252f555660 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f08b1e61740 10 client.17175 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f08b1e61740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f08b1e61740 20 client.17175 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f08b1e61740 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f565cc0 0x56252f568140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f08b1e61740 4 client.17175 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.620+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252f451600 con 0x56252f565cc0 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.621+0000 7f08a7e00640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f565cc0 0x56252f568140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.621+0000 7f08a7e00640 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f565cc0 0x56252f568140 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5753e6df2e29a2 server_cookie=9685ab6fe9604451 in_seq=0 out_seq=0 2024-04-13T05:26:23.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.621+0000 7f0896000640 10 client.17175 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:23.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.624+0000 7f0896000640 1 -- 172.21.15.57:0/69279537 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56252f451600 con 0x56252f565cc0 2024-04-13T05:26:23.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.624+0000 7f0896000640 10 client.17175 handle_command_reply: tid=0 2024-04-13T05:26:23.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.624+0000 7f08b1e61740 1 client.17175 shutdown 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 msgr2=0x56252f5612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f5612e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f08a00128e0 tx=0x7f08a0004a40 comp rx=0 tx=0).stop 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 shutdown_connections 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f565cc0 0x56252f568140 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f555660 0x56252f5612e0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 --2- 172.21.15.57:0/69279537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 >> 172.21.15.57:0/69279537 conn(0x56252f546c60 msgr2=0x56252f545e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f0896000640 0 client.17175 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 shutdown_connections 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 1 -- 172.21.15.57:0/69279537 wait complete. 2024-04-13T05:26:23.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.625+0000 7f08b1e61740 20 client.17175 trim_cache size 0 max 16384 2024-04-13T05:26:23.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 Processor -- start 2024-04-13T05:26:23.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 -- start start 2024-04-13T05:26:23.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f555660 0x56252f551730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81ab170 con 0x56252f5504f0 2024-04-13T05:26:23.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a8980 con 0x56252f555660 2024-04-13T05:26:23.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f08b1e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a67d0 con 0x56252f54fb00 2024-04-13T05:26:23.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f089f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.646+0000 7f089f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53584/0 (socket says 172.21.15.57:53584) 2024-04-13T05:26:23.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 -- 172.21.15.57:0/687956063 learned_addr learned my addr 172.21.15.57:0/687956063 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f08ad000640 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f555660 0x56252f551730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f08a7e00640 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 -- 172.21.15.57:0/687956063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 msgr2=0x56252f559d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 -- 172.21.15.57:0/687956063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f555660 msgr2=0x56252f551730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f555660 0x56252f551730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 -- 172.21.15.57:0/687956063 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252f5426d0 con 0x56252f5504f0 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f08a7e00640 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089f400640 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f08900096c0 tx=0x7f0890000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6230edecebe678d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.647+0000 7f089cc00640 1 -- 172.21.15.57:0/687956063 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089004d030 con 0x56252f5504f0 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.648+0000 7f089cc00640 1 -- 172.21.15.57:0/687956063 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0890005040 con 0x56252f5504f0 2024-04-13T05:26:23.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.648+0000 7f089cc00640 1 -- 172.21.15.57:0/687956063 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890044040 con 0x56252f5504f0 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 -- 172.21.15.57:0/687956063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 msgr2=0x56252f55a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f08900096c0 tx=0x7f0890000bc0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 -- 172.21.15.57:0/687956063 shutdown_connections 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f555660 0x56252f551730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f55a2a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 --2- 172.21.15.57:0/687956063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f54fb00 0x56252f559d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 -- 172.21.15.57:0/687956063 >> 172.21.15.57:0/687956063 conn(0x56252f546c60 msgr2=0x56252f567cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 -- 172.21.15.57:0/687956063 shutdown_connections 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 -- 172.21.15.57:0/687956063 wait complete. 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.649+0000 7f08b1e61740 1 -- start start 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 Processor -- start 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f54e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f555660 0x56252f561200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a6e40 con 0x56252f5504f0 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a82b0 con 0x56252f54fb00 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a7510 con 0x56252f555660 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.650+0000 7f08b1e61740 10 client.0 _ll_register_callbacks cb 0x7f08a00124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f54e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f54e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53590/0 (socket says 172.21.15.57:53590) 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 -- 172.21.15.57:0/1849230310 learned_addr learned my addr 172.21.15.57:0/1849230310 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f08a7e00640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f08ad000640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f555660 0x56252f561200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 -- 172.21.15.57:0/1849230310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f555660 msgr2=0x56252f561200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f555660 0x56252f561200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 -- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 msgr2=0x56252f54e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 -- 172.21.15.57:0/1849230310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252f54ec80 con 0x56252f5504f0 2024-04-13T05:26:23.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f089f400640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f54e640 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0890005b90 tx=0x7f0890005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df15ee0cb54490f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.651+0000 7f0896000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:23.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.652+0000 7f0896000640 1 -- 172.21.15.57:0/1849230310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089004d070 con 0x56252f5504f0 2024-04-13T05:26:23.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.652+0000 7f0896000640 1 -- 172.21.15.57:0/1849230310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f08900046c0 con 0x56252f5504f0 2024-04-13T05:26:23.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.652+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0890005040 con 0x56252f5504f0 2024-04-13T05:26:23.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.652+0000 7f0896000640 1 -- 172.21.15.57:0/1849230310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f089005f050 con 0x56252f5504f0 2024-04-13T05:26:23.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.652+0000 7f08b1e61740 10 client.17181 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:23.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.652+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56252f5cfad0 con 0x56252f5504f0 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f0896000640 1 -- 172.21.15.57:0/1849230310 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0890005040 con 0x56252f5504f0 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f0896000640 1 -- 172.21.15.57:0/1849230310 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f0890003bc0 con 0x56252f5504f0 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08b1e61740 10 client.17181 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08b1e61740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08b1e61740 20 client.17181 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f5cffe0 0x56252f5d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08b1e61740 4 client.17181 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56252f4bfa40 con 0x56252f5cffe0 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.653+0000 7f08a7e00640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f5cffe0 0x56252f5d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08a7e00640 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f5cffe0 0x56252f5d23d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c036d5c79f4a2af server_cookie=b7ccadd75e225084 in_seq=0 out_seq=0 2024-04-13T05:26:23.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f0896000640 10 client.17181 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:23.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f0896000640 1 -- 172.21.15.57:0/1849230310 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56252f4bfa40 con 0x56252f5cffe0 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f0896000640 10 client.17181 handle_command_reply: tid=0 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 client.17181 shutdown 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 msgr2=0x56252f54e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f54e640 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0890005b90 tx=0x7f0890005dd0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 shutdown_connections 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x56252f5cffe0 0x56252f5d23d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56252f555660 0x56252f561200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.655+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56252f5504f0 0x56252f54e640 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.656+0000 7f08b1e61740 1 --2- 172.21.15.57:0/1849230310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56252f54fb00 0x56252f54e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.656+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 >> 172.21.15.57:0/1849230310 conn(0x56252f546c60 msgr2=0x56252f551730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.656+0000 7f0896000640 0 client.17181 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.656+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 shutdown_connections 2024-04-13T05:26:23.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.656+0000 7f08b1e61740 1 -- 172.21.15.57:0/1849230310 wait complete. 2024-04-13T05:26:23.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.656+0000 7f08b1e61740 20 client.17181 trim_cache size 0 max 16384 2024-04-13T05:26:23.661 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:26:23.661 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:26:23.661 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:26:23.661 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f088c0a3bb0 msgr2=0x7f088c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f088c0a3bb0 0x7f088c0a6070 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x56252f550d50 tx=0x7f08a0009580 comp rx=0 tx=0).stop 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 msgr2=0x7f08a8198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 0x7f08a8198110 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f089800d950 tx=0x7f089800de10 comp rx=0 tx=0).stop 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 shutdown_connections 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f088c0a3bb0 0x7f088c0a6070 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08a8105280 0x7f08a8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8107bb0 0x7f08a819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 --2- 172.21.15.57:0/2042848204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08a8102950 0x7f08a8198110 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 >> 172.21.15.57:0/2042848204 conn(0x7f08a80fc360 msgr2=0x7f08a81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 shutdown_connections 2024-04-13T05:26:23.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.657+0000 7f08ada00640 1 -- 172.21.15.57:0/2042848204 wait complete. 2024-04-13T05:26:23.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:26:23.677 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 -- 172.21.15.57:0/2349245703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244105250 msgr2=0x7f0244107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 --2- 172.21.15.57:0/2349245703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244105250 0x7f0244107640 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0230009900 tx=0x7f023002bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 -- 172.21.15.57:0/2349245703 shutdown_connections 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 --2- 172.21.15.57:0/2349245703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244107b80 0x7f0244109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 --2- 172.21.15.57:0/2349245703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244105250 0x7f0244107640 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 --2- 172.21.15.57:0/2349245703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0244102920 0x7f0244104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 -- 172.21.15.57:0/2349245703 >> 172.21.15.57:0/2349245703 conn(0x7f02440fc330 msgr2=0x7f02440fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:23.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 -- 172.21.15.57:0/2349245703 shutdown_connections 2024-04-13T05:26:23.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.901+0000 7f0249600640 1 -- 172.21.15.57:0/2349245703 wait complete. 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.902+0000 7f0249600640 1 Processor -- start 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.902+0000 7f0249600640 1 -- start start 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0249600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0244102920 0x7f024419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0249600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244105250 0x7f024419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0249600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 0x7f02441a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0249600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f024410d9a0 con 0x7f0244107b80 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0249600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f024410d820 con 0x7f0244105250 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0249600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f024410db20 con 0x7f0244102920 2024-04-13T05:26:23.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0248c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 0x7f02441a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0248c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 0x7f02441a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53614/0 (socket says 172.21.15.57:53614) 2024-04-13T05:26:23.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.903+0000 7f0248c00640 1 -- 172.21.15.57:0/2236335969 learned_addr learned my addr 172.21.15.57:0/2236335969 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:23.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0243400640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244105250 0x7f024419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.907 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0243e00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0244102920 0x7f024419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0248c00640 1 -- 172.21.15.57:0/2236335969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0244102920 msgr2=0x7f024419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0248c00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0244102920 0x7f024419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0248c00640 1 -- 172.21.15.57:0/2236335969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244105250 msgr2=0x7f024419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0248c00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244105250 0x7f024419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0248c00640 1 -- 172.21.15.57:0/2236335969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02441a1e50 con 0x7f0244107b80 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0243400640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244105250 0x7f024419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.904+0000 7f0248c00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 0x7f02441a17e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f023c00cce0 tx=0x7f023c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.905+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c00f450 con 0x7f0244107b80 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.905+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f023c00faa0 con 0x7f0244107b80 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.905+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c018a80 con 0x7f0244107b80 2024-04-13T05:26:23.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.905+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02441a2140 con 0x7f0244107b80 2024-04-13T05:26:23.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.905+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02441a2660 con 0x7f0244107b80 2024-04-13T05:26:23.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.906+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02441096d0 con 0x7f0244107b80 2024-04-13T05:26:23.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.908+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f023c00fc50 con 0x7f0244107b80 2024-04-13T05:26:23.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.910+0000 7f0240c00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f02280a3c30 0x7f02280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:23.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.910+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f023c0d1060 con 0x7f0244107b80 2024-04-13T05:26:23.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.910+0000 7f0243e00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f02280a3c30 0x7f02280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:23.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.911+0000 7f0243e00640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f02280a3c30 0x7f02280a60f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f023400abe0 tx=0x7f023400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:23.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:23.921+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f023c014030 con 0x7f0244107b80 2024-04-13T05:26:24.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.187+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f024402d5e0 con 0x7f0244107b80 2024-04-13T05:26:24.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.188+0000 7f0240c00640 1 -- 172.21.15.57:0/2236335969 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11864 (secure 0 0 0) 0x7f023c09d6c0 con 0x7f0244107b80 2024-04-13T05:26:24.191 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:26:24.192 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:53.740918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:26:24.192 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-04-13T05:26:24.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f02280a3c30 msgr2=0x7f02280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f02280a3c30 0x7f02280a60f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f023400abe0 tx=0x7f023400a2d0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 msgr2=0x7f02441a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 0x7f02441a17e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f023c00cce0 tx=0x7f023c00a620 comp rx=0 tx=0).stop 2024-04-13T05:26:24.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 shutdown_connections 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f02280a3c30 0x7f02280a60f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0244105250 0x7f024419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0244107b80 0x7f02441a17e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 --2- 172.21.15.57:0/2236335969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0244102920 0x7f024419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.191+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 >> 172.21.15.57:0/2236335969 conn(0x7f02440fc330 msgr2=0x7f0244102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.192+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 shutdown_connections 2024-04-13T05:26:24.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.192+0000 7f0249600640 1 -- 172.21.15.57:0/2236335969 wait complete. 2024-04-13T05:26:24.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 -- 172.21.15.57:0/2684917234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105410 msgr2=0x7f7a14105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 --2- 172.21.15.57:0/2684917234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105410 0x7f7a14105890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009900 tx=0x7f7a0002bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 -- 172.21.15.57:0/2684917234 shutdown_connections 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 --2- 172.21.15.57:0/2684917234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a14105dd0 0x7f7a1410c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 --2- 172.21.15.57:0/2684917234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105410 0x7f7a14105890 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 --2- 172.21.15.57:0/2684917234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a14104220 0x7f7a14104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 -- 172.21.15.57:0/2684917234 >> 172.21.15.57:0/2684917234 conn(0x7f7a140ff770 msgr2=0x7f7a14101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 -- 172.21.15.57:0/2684917234 shutdown_connections 2024-04-13T05:26:24.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.439+0000 7f7a19200640 1 -- 172.21.15.57:0/2684917234 wait complete. 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.440+0000 7f7a19200640 1 Processor -- start 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.440+0000 7f7a19200640 1 -- start start 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a14104220 0x7f7a1419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a19200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a14105410 0x7f7a1419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a19200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 0x7f7a141a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a19200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1410fe10 con 0x7f7a14105dd0 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a19200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1410fc90 con 0x7f7a14104220 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a19200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1410ff90 con 0x7f7a14105410 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a12800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 0x7f7a141a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a12800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 0x7f7a141a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53632/0 (socket says 172.21.15.57:53632) 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a12800640 1 -- 172.21.15.57:0/182735642 learned_addr learned my addr 172.21.15.57:0/182735642 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a11e00640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a14104220 0x7f7a1419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.441+0000 7f7a11400640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a14105410 0x7f7a1419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a12800640 1 -- 172.21.15.57:0/182735642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a14105410 msgr2=0x7f7a1419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a12800640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a14105410 0x7f7a1419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a12800640 1 -- 172.21.15.57:0/182735642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a14104220 msgr2=0x7f7a1419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a12800640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a14104220 0x7f7a1419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a12800640 1 -- 172.21.15.57:0/182735642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a141a20f0 con 0x7f7a14105dd0 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a12800640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 0x7f7a141a1a80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00eaa0 tx=0x7f7a0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a0aa00640 1 -- 172.21.15.57:0/182735642 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00cdb0 con 0x7f7a14105dd0 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.442+0000 7f7a0aa00640 1 -- 172.21.15.57:0/182735642 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a0c005020 con 0x7f7a14105dd0 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.443+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a141a23e0 con 0x7f7a14105dd0 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.443+0000 7f7a0aa00640 1 -- 172.21.15.57:0/182735642 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0116d0 con 0x7f7a14105dd0 2024-04-13T05:26:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.443+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a141a2930 con 0x7f7a14105dd0 2024-04-13T05:26:24.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.447+0000 7f7a1c192740 1 Processor -- start 2024-04-13T05:26:24.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.447+0000 7f7a0aa00640 1 -- 172.21.15.57:0/182735642 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a0c0051d0 con 0x7f7a14105dd0 2024-04-13T05:26:24.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 -- start start 2024-04-13T05:26:24.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a374670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a379740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a37c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56017a2da020 con 0x56017a379db0 2024-04-13T05:26:24.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56017a280950 con 0x56017a374c40 2024-04-13T05:26:24.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a1c192740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56017a2d5fe0 con 0x56017a374250 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.448+0000 7f7a11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a37c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a379740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a37c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53634/0 (socket says 172.21.15.57:53634) 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11400640 1 -- 172.21.15.57:0/3530777735 learned_addr learned my addr 172.21.15.57:0/3530777735 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11400640 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a374670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11e00640 1 -- 172.21.15.57:0/3530777735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 msgr2=0x56017a374670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11e00640 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a374670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11e00640 1 -- 172.21.15.57:0/3530777735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 msgr2=0x56017a37c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11e00640 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a37c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11e00640 1 -- 172.21.15.57:0/3530777735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560179fec3f0 con 0x56017a374c40 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a11400640 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a37c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:26:24.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.449+0000 7f7a0aa00640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79f80a3bb0 0x7f79f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.450+0000 7f7a0aa00640 1 -- 172.21.15.57:0/182735642 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7a0c0d0aa0 con 0x7f7a14105dd0 2024-04-13T05:26:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.450+0000 7f7a11e00640 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a379740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400b730 tx=0x7f7a0400bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43abe75007101923 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.450+0000 7f7a11400640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79f80a3bb0 0x7f79f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.450+0000 7f79ff400640 1 -- 172.21.15.57:0/3530777735 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400be40 con 0x56017a374c40 2024-04-13T05:26:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.450+0000 7f79ff400640 1 -- 172.21.15.57:0/3530777735 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a04019070 con 0x56017a374c40 2024-04-13T05:26:24.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.450+0000 7f79ff400640 1 -- 172.21.15.57:0/3530777735 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04014c50 con 0x56017a374c40 2024-04-13T05:26:24.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.451+0000 7f7a1c192740 1 -- 172.21.15.57:0/3530777735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 msgr2=0x56017a379740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.451+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a379740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400b730 tx=0x7f7a0400bbf0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.451+0000 7f7a11400640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79f80a3bb0 0x7f79f80a6070 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7a0002c850 tx=0x7f7a00005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.451+0000 7f7a1c192740 1 -- 172.21.15.57:0/3530777735 shutdown_connections 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.451+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a379740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a37c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3530777735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a374670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 -- 172.21.15.57:0/3530777735 >> 172.21.15.57:0/3530777735 conn(0x56017a36f7c0 msgr2=0x56017a36fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 -- 172.21.15.57:0/3530777735 shutdown_connections 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 -- 172.21.15.57:0/3530777735 wait complete. 2024-04-13T05:26:24.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 -- start start 2024-04-13T05:26:24.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.452+0000 7f7a1c192740 1 Processor -- start 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a37e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a37ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a379db0 0x56017a385cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56017a37cd60 con 0x56017a374c40 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56017a293c80 con 0x56017a379db0 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56017a37cee0 con 0x56017a374250 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a1c192740 10 client.0 _ll_register_callbacks cb 0x56017a293670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a37ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a37ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53642/0 (socket says 172.21.15.57:53642) 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.453+0000 7f7a11400640 1 -- 172.21.15.57:0/2214707097 learned_addr learned my addr 172.21.15.57:0/2214707097 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11e00640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a37e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a12800640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a379db0 0x56017a385cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11400640 1 -- 172.21.15.57:0/2214707097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 msgr2=0x56017a37e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11400640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a37e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11400640 1 -- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a379db0 msgr2=0x56017a385cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11400640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a379db0 0x56017a385cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11400640 1 -- 172.21.15.57:0/2214707097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017a0b67d0 con 0x56017a374c40 2024-04-13T05:26:24.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f7a11400640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a37ec90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7a00036040 tx=0x7f7a0002bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748dc9873f1e5778 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.454+0000 7f79f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:24.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f79f3e00640 1 -- 172.21.15.57:0/2214707097 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00053060 con 0x56017a374c40 2024-04-13T05:26:24.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f79f3e00640 1 -- 172.21.15.57:0/2214707097 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7a00004c10 con 0x56017a374c40 2024-04-13T05:26:24.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a04019070 con 0x56017a374c40 2024-04-13T05:26:24.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f79f3e00640 1 -- 172.21.15.57:0/2214707097 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7a0005a050 con 0x56017a374c40 2024-04-13T05:26:24.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f7a1c192740 10 client.17199 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:24.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56017a3890b0 con 0x56017a374c40 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.455+0000 7f79f3e00640 1 -- 172.21.15.57:0/2214707097 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a04019070 con 0x56017a374c40 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f79f3e00640 1 -- 172.21.15.57:0/2214707097 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7a00040040 con 0x56017a374c40 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a1c192740 10 client.17199 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a1c192740 10 client.17199 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a1c192740 20 client.17199 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a1c192740 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a38a820 0x56017a38cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a1c192740 4 client.17199 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56017a279750 con 0x56017a38a820 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.456+0000 7f7a11e00640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a38a820 0x56017a38cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.457+0000 7f7a11e00640 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a38a820 0x56017a38cca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eae2351f2695fb07 server_cookie=6857a7fa044d2f8a in_seq=0 out_seq=0 2024-04-13T05:26:24.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.457+0000 7f79f3e00640 10 client.17199 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f79f3e00640 1 -- 172.21.15.57:0/2214707097 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56017a279750 con 0x56017a38a820 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f79f3e00640 10 client.17199 handle_command_reply: tid=0 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 client.17199 shutdown 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 msgr2=0x56017a37ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a37ec90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7a00036040 tx=0x7f7a0002bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 shutdown_connections 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a38a820 0x56017a38cca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a379db0 0x56017a385cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a37ec90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 --2- 172.21.15.57:0/2214707097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a37e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f79f3e00640 0 client.17199 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 >> 172.21.15.57:0/2214707097 conn(0x56017a36f7c0 msgr2=0x56017a36ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 shutdown_connections 2024-04-13T05:26:24.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 1 -- 172.21.15.57:0/2214707097 wait complete. 2024-04-13T05:26:24.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.460+0000 7f7a1c192740 20 client.17199 trim_cache size 0 max 16384 2024-04-13T05:26:24.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 Processor -- start 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 -- start start 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374250 0x56017a36e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a385c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a379db0 0x56017a36e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141af760 con 0x56017a374c40 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141acf70 con 0x56017a374250 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.476+0000 7f7a1c192740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141aadc0 con 0x56017a379db0 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a385c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a385c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53658/0 (socket says 172.21.15.57:53658) 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 -- 172.21.15.57:0/3608375143 learned_addr learned my addr 172.21.15.57:0/3608375143 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11e00640 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374250 0x56017a36e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a12800640 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a379db0 0x56017a36e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 -- 172.21.15.57:0/3608375143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a379db0 msgr2=0x56017a36e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a379db0 0x56017a36e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 -- 172.21.15.57:0/3608375143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374250 msgr2=0x56017a36e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374250 0x56017a36e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.477+0000 7f7a11400640 1 -- 172.21.15.57:0/3608375143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017a36b230 con 0x56017a374c40 2024-04-13T05:26:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.478+0000 7f7a11400640 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a385c90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a0002c180 tx=0x7f7a0002cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372639c2009bb3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.478+0000 7f79fea00640 1 -- 172.21.15.57:0/3608375143 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00053040 con 0x56017a374c40 2024-04-13T05:26:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.478+0000 7f79fea00640 1 -- 172.21.15.57:0/3608375143 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a00005630 con 0x56017a374c40 2024-04-13T05:26:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.478+0000 7f79fea00640 1 -- 172.21.15.57:0/3608375143 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a000344b0 con 0x56017a374c40 2024-04-13T05:26:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 -- 172.21.15.57:0/3608375143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 msgr2=0x56017a385c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a385c90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a0002c180 tx=0x7f7a0002cc10 comp rx=0 tx=0).stop 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 -- 172.21.15.57:0/3608375143 shutdown_connections 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a379db0 0x56017a36e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a374c40 0x56017a385c90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 --2- 172.21.15.57:0/3608375143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374250 0x56017a36e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 -- 172.21.15.57:0/3608375143 >> 172.21.15.57:0/3608375143 conn(0x56017a36f7c0 msgr2=0x56017a37e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 -- 172.21.15.57:0/3608375143 shutdown_connections 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 -- 172.21.15.57:0/3608375143 wait complete. 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.479+0000 7f7a1c192740 1 -- start start 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 Processor -- start 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a383540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a383a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a384a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141ab430 con 0x56017a379db0 2024-04-13T05:26:24.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141ac8a0 con 0x56017a374c40 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141abb00 con 0x56017a374250 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a1c192740 10 client.0 _ll_register_callbacks cb 0x7f7a00005360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a12800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a384a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a12800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a384a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53660/0 (socket says 172.21.15.57:53660) 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.480+0000 7f7a12800640 1 -- 172.21.15.57:0/4280862972 learned_addr learned my addr 172.21.15.57:0/4280862972 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a11400640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a383a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a11e00640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a383540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a12800640 1 -- 172.21.15.57:0/4280862972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 msgr2=0x56017a383540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a12800640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a383540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a12800640 1 -- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 msgr2=0x56017a383a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a12800640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a383a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a12800640 1 -- 172.21.15.57:0/4280862972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017a385010 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f7a12800640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a384a20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c09c910 tx=0x7f7a0c09d140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211e62869f33f3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f79f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f79f3e00640 1 -- 172.21.15.57:0/4280862972 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c01f040 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.481+0000 7f79f3e00640 1 -- 172.21.15.57:0/4280862972 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7a0c0bb660 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56017a37e540 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f79f3e00640 1 -- 172.21.15.57:0/4280862972 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c0a55f0 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f7a1c192740 10 client.17205 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56017a3f0f50 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f79f3e00640 1 -- 172.21.15.57:0/4280862972 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56017a37e540 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f79f3e00640 1 -- 172.21.15.57:0/4280862972 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7a0c00f040 con 0x56017a379db0 2024-04-13T05:26:24.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f7a1c192740 10 client.17205 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f7a1c192740 10 client.17205 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.482+0000 7f7a1c192740 20 client.17205 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.483+0000 7f7a1c192740 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a3f1960 0x56017a3f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.483+0000 7f7a1c192740 4 client.17205 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.483+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56017a2e7a40 con 0x56017a3f1960 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.483+0000 7f7a11e00640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a3f1960 0x56017a3f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.484+0000 7f7a11e00640 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a3f1960 0x56017a3f3d80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f405c65bfe303709 server_cookie=6b82033c1b3e59dd in_seq=0 out_seq=0 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.484+0000 7f79f3e00640 10 client.17205 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f79f3e00640 1 -- 172.21.15.57:0/4280862972 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56017a2e7a40 con 0x56017a3f1960 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f79f3e00640 10 client.17205 handle_command_reply: tid=0 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 client.17205 shutdown 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 msgr2=0x56017a384a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a384a20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c09c910 tx=0x7f7a0c09d140 comp rx=0 tx=0).stop 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 shutdown_connections 2024-04-13T05:26:24.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x56017a3f1960 0x56017a3f3d80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56017a374c40 0x56017a383a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56017a379db0 0x56017a384a20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 --2- 172.21.15.57:0/4280862972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56017a374250 0x56017a383540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f79f3e00640 0 client.17205 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 >> 172.21.15.57:0/4280862972 conn(0x56017a36f7c0 msgr2=0x56017a37e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 shutdown_connections 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 1 -- 172.21.15.57:0/4280862972 wait complete. 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.485+0000 7f7a1c192740 20 client.17205 trim_cache size 0 max 16384 2024-04-13T05:26:24.490 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:26:24.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79f80a3bb0 msgr2=0x7f79f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79f80a3bb0 0x7f79f80a6070 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7a0002c850 tx=0x7f7a00005b50 comp rx=0 tx=0).stop 2024-04-13T05:26:24.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 msgr2=0x7f7a141a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 0x7f7a141a1a80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00eaa0 tx=0x7f7a0c00ef60 comp rx=0 tx=0).stop 2024-04-13T05:26:24.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 shutdown_connections 2024-04-13T05:26:24.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f79f80a3bb0 0x7f79f80a6070 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a14105dd0 0x7f7a141a1a80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a14105410 0x7f7a1419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 --2- 172.21.15.57:0/182735642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a14104220 0x7f7a1419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 >> 172.21.15.57:0/182735642 conn(0x7f7a140ff770 msgr2=0x7f7a14108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 shutdown_connections 2024-04-13T05:26:24.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.486+0000 7f7a19200640 1 -- 172.21.15.57:0/182735642 wait complete. 2024-04-13T05:26:24.507 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:26:24.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 -- 172.21.15.57:0/2551227232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 msgr2=0x7ff6f0107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 --2- 172.21.15.57:0/2551227232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0107670 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009900 tx=0x7ff6dc02bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:24.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 -- 172.21.15.57:0/2551227232 shutdown_connections 2024-04-13T05:26:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 --2- 172.21.15.57:0/2551227232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6f0107bb0 0x7ff6f010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 --2- 172.21.15.57:0/2551227232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0107670 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 --2- 172.21.15.57:0/2551227232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0102950 0x7ff6f0104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 -- 172.21.15.57:0/2551227232 >> 172.21.15.57:0/2551227232 conn(0x7ff6f00fc360 msgr2=0x7ff6f00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.711+0000 7ff6f6400640 1 -- 172.21.15.57:0/2551227232 shutdown_connections 2024-04-13T05:26:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.712+0000 7ff6f6400640 1 -- 172.21.15.57:0/2551227232 wait complete. 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.712+0000 7ff6f6400640 1 Processor -- start 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 -- start start 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6f0102950 0x7ff6f0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0107bb0 0x7ff6f019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f010d9e0 con 0x7ff6f0105280 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f010d860 con 0x7ff6f0107bb0 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f6400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f010db60 con 0x7ff6f0102950 2024-04-13T05:26:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53688/0 (socket says 172.21.15.57:53688) 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6efe00640 1 -- 172.21.15.57:0/1156243055 learned_addr learned my addr 172.21.15.57:0/1156243055 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0107bb0 0x7ff6f019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.713+0000 7ff6f5000640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6f0102950 0x7ff6f0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6efe00640 1 -- 172.21.15.57:0/1156243055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6f0102950 msgr2=0x7ff6f0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6efe00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6f0102950 0x7ff6f0197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6efe00640 1 -- 172.21.15.57:0/1156243055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0107bb0 msgr2=0x7ff6f019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6efe00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0107bb0 0x7ff6f019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6efe00640 1 -- 172.21.15.57:0/1156243055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f019fbe0 con 0x7ff6f0105280 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0107bb0 0x7ff6f019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6efe00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0198500 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc036040 tx=0x7ff6dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6ed600640 1 -- 172.21.15.57:0/1156243055 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc043020 con 0x7ff6f0105280 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6ed600640 1 -- 172.21.15.57:0/1156243055 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6dc004aa0 con 0x7ff6f0105280 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f019fe70 con 0x7ff6f0105280 2024-04-13T05:26:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6ed600640 1 -- 172.21.15.57:0/1156243055 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc003de0 con 0x7ff6f0105280 2024-04-13T05:26:24.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.714+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f01a0380 con 0x7ff6f0105280 2024-04-13T05:26:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.717+0000 7ff6fa98c740 1 Processor -- start 2024-04-13T05:26:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 -- start start 2024-04-13T05:26:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a338860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6ed600640 1 -- 172.21.15.57:0/1156243055 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff6dc046020 con 0x7ff6f0105280 2024-04-13T05:26:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a33d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a33dfa0 0x56535a340430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56535a29e020 con 0x56535a338e30 2024-04-13T05:26:24.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56535a244950 con 0x56535a33dfa0 2024-04-13T05:26:24.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56535a299fe0 con 0x56535a338440 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6f5000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a33d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6f5000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a33d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53700/0 (socket says 172.21.15.57:53700) 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.718+0000 7ff6f5000640 1 -- 172.21.15.57:0/3023642564 learned_addr learned my addr 172.21.15.57:0/3023642564 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6efe00640 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a33dfa0 0x56535a340430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a338860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5000640 1 -- 172.21.15.57:0/3023642564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 msgr2=0x56535a338860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5000640 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a338860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5000640 1 -- 172.21.15.57:0/3023642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a33dfa0 msgr2=0x56535a340430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5000640 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a33dfa0 0x56535a340430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5000640 1 -- 172.21.15.57:0/3023642564 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565359fb03f0 con 0x56535a338e30 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.719+0000 7ff6f5000640 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a33d930 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400bb10 tx=0x7ff6e400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f7e868899a7832 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.720+0000 7ff6e2000640 1 -- 172.21.15.57:0/3023642564 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4017420 con 0x56535a338e30 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.720+0000 7ff6e2000640 1 -- 172.21.15.57:0/3023642564 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e4017a70 con 0x56535a338e30 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.720+0000 7ff6e2000640 1 -- 172.21.15.57:0/3023642564 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400fa50 con 0x56535a338e30 2024-04-13T05:26:24.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.720+0000 7ff6ed600640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff6d80a3c30 0x7ff6d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.720+0000 7ff6ed600640 1 -- 172.21.15.57:0/1156243055 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff6dc0f2c20 con 0x7ff6f0105280 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.720+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff6d80a3c30 0x7ff6d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 -- 172.21.15.57:0/3023642564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 msgr2=0x56535a33d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a33d930 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400bb10 tx=0x7ff6e400d340 comp rx=0 tx=0).stop 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 -- 172.21.15.57:0/3023642564 shutdown_connections 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a33dfa0 0x56535a340430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a33d930 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/3023642564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a338860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 -- 172.21.15.57:0/3023642564 >> 172.21.15.57:0/3023642564 conn(0x56535a3339b0 msgr2=0x56535a333db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 -- 172.21.15.57:0/3023642564 shutdown_connections 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 -- 172.21.15.57:0/3023642564 wait complete. 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff6d80a3c30 0x7ff6d80a60f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x56535a339690 tx=0x7ff6e8008470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.721+0000 7ff6fa98c740 1 -- start start 2024-04-13T05:26:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 Processor -- start 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a346d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a3472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a34e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56535a340f50 con 0x56535a33dfa0 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56535a257c80 con 0x56535a338e30 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56535a3410d0 con 0x56535a338440 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6fa98c740 10 client.0 _ll_register_callbacks cb 0x56535a257670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a34e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a34e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53706/0 (socket says 172.21.15.57:53706) 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.722+0000 7ff6f5a00640 1 -- 172.21.15.57:0/2615169413 learned_addr learned my addr 172.21.15.57:0/2615169413 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5000640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a346d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6efe00640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a3472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5a00640 1 -- 172.21.15.57:0/2615169413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 msgr2=0x56535a346d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a346d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5a00640 1 -- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 msgr2=0x56535a3472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a3472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5a00640 1 -- 172.21.15.57:0/2615169413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535a07a7d0 con 0x56535a33dfa0 2024-04-13T05:26:24.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6efe00640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a3472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:24.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a34e2e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8010e30 tx=0x7ff6e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b938263b45d9d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.723+0000 7ff6d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6d6a00640 1 -- 172.21.15.57:0/2615169413 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e801d620 con 0x56535a33dfa0 2024-04-13T05:26:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e4017a70 con 0x56535a33dfa0 2024-04-13T05:26:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6d6a00640 1 -- 172.21.15.57:0/2615169413 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff6e801d7c0 con 0x56535a33dfa0 2024-04-13T05:26:24.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6d6a00640 1 -- 172.21.15.57:0/2615169413 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6e801d9c0 con 0x56535a33dfa0 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6fa98c740 10 client.17217 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56535a351720 con 0x56535a33dfa0 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.724+0000 7ff6d6a00640 1 -- 172.21.15.57:0/2615169413 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e4017a70 con 0x56535a33dfa0 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6d6a00640 1 -- 172.21.15.57:0/2615169413 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff6e8025b10 con 0x56535a33dfa0 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6fa98c740 10 client.17217 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6fa98c740 10 client.17217 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6fa98c740 20 client.17217 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a352f10 0x56535a355390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6fa98c740 4 client.17217 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56535a23d750 con 0x56535a352f10 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.725+0000 7ff6f5000640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a352f10 0x56535a355390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.726+0000 7ff6f5000640 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a352f10 0x56535a355390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e34daed274edfba4 server_cookie=50417d27b5fabcd6 in_seq=0 out_seq=0 2024-04-13T05:26:24.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.726+0000 7ff6d6a00640 10 client.17217 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.729+0000 7ff6d6a00640 1 -- 172.21.15.57:0/2615169413 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56535a23d750 con 0x56535a352f10 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6d6a00640 10 client.17217 handle_command_reply: tid=0 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 client.17217 shutdown 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 msgr2=0x56535a34e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a34e2e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8010e30 tx=0x7ff6e8004a40 comp rx=0 tx=0).stop 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 shutdown_connections 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a352f10 0x56535a355390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a3472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a34e2e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/2615169413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a346d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 >> 172.21.15.57:0/2615169413 conn(0x56535a3339b0 msgr2=0x56535a332cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6d6a00640 0 client.17217 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 shutdown_connections 2024-04-13T05:26:24.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 1 -- 172.21.15.57:0/2615169413 wait complete. 2024-04-13T05:26:24.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.730+0000 7ff6fa98c740 20 client.17217 trim_cache size 0 max 16384 2024-04-13T05:26:24.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 Processor -- start 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 -- start start 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a332440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a34e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a332980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01aafb0 con 0x56535a33dfa0 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01a87c0 con 0x56535a338e30 2024-04-13T05:26:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.745+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01a6610 con 0x56535a338440 2024-04-13T05:26:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a332980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a332980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53710/0 (socket says 172.21.15.57:53710) 2024-04-13T05:26:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 -- 172.21.15.57:0/796349179 learned_addr learned my addr 172.21.15.57:0/796349179 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5000640 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a332440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6efe00640 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a34e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 -- 172.21.15.57:0/796349179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 msgr2=0x56535a332440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a332440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 -- 172.21.15.57:0/796349179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 msgr2=0x56535a34e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a34e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 -- 172.21.15.57:0/796349179 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535a3396c0 con 0x56535a33dfa0 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.746+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a332980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8005fc0 tx=0x7ff6e800dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb15627c85bbe80 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6e1600640 1 -- 172.21.15.57:0/796349179 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e801dd60 con 0x56535a33dfa0 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6e1600640 1 -- 172.21.15.57:0/796349179 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e800dc10 con 0x56535a33dfa0 2024-04-13T05:26:24.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6e1600640 1 -- 172.21.15.57:0/796349179 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8025cc0 con 0x56535a33dfa0 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 -- 172.21.15.57:0/796349179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 msgr2=0x56535a332980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a332980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8005fc0 tx=0x7ff6e800dfd0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 -- 172.21.15.57:0/796349179 shutdown_connections 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338e30 0x56535a34e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a33dfa0 0x56535a332980 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/796349179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a338440 0x56535a332440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.747+0000 7ff6fa98c740 1 -- 172.21.15.57:0/796349179 >> 172.21.15.57:0/796349179 conn(0x56535a3339b0 msgr2=0x56535a33a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.753 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.748+0000 7ff6fa98c740 1 -- 172.21.15.57:0/796349179 shutdown_connections 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.748+0000 7ff6fa98c740 1 -- 172.21.15.57:0/796349179 wait complete. 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.748+0000 7ff6fa98c740 1 -- start start 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.748+0000 7ff6fa98c740 1 Processor -- start 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338440 0x56535a34bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a34c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a33dfa0 0x56535a34d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01a6c80 con 0x56535a338e30 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01a80f0 con 0x56535a338440 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01a7350 con 0x56535a33dfa0 2024-04-13T05:26:24.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6fa98c740 10 client.0 _ll_register_callbacks cb 0x7ff6e8010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a34c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a34c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53718/0 (socket says 172.21.15.57:53718) 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 -- 172.21.15.57:0/1002519554 learned_addr learned my addr 172.21.15.57:0/1002519554 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6f5000640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338440 0x56535a34bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a33dfa0 0x56535a34d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 -- 172.21.15.57:0/1002519554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a33dfa0 msgr2=0x56535a34d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a33dfa0 0x56535a34d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 -- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338440 msgr2=0x56535a34bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338440 0x56535a34bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6efe00640 1 -- 172.21.15.57:0/1002519554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535a34d600 con 0x56535a338e30 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6f5000640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338440 0x56535a34bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.749+0000 7ff6f5a00640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a33dfa0 0x56535a34d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:24.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6efe00640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a34c050 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0bb4d0 tx=0x7ff6dc0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09f160dd64900f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6d6a00640 1 -- 172.21.15.57:0/1002519554 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0be460 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6d6a00640 1 -- 172.21.15.57:0/1002519554 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff6dc0bb170 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e800dc10 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6d6a00640 1 -- 172.21.15.57:0/1002519554 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff6dc0bf200 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6fa98c740 10 client.17223 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56535a3b9710 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.750+0000 7ff6d6a00640 1 -- 172.21.15.57:0/1002519554 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6e800dc10 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6d6a00640 1 -- 172.21.15.57:0/1002519554 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff6dc0bed60 con 0x56535a338e30 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6fa98c740 10 client.17223 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6fa98c740 10 client.17223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6fa98c740 20 client.17223 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a3ba1c0 0x56535a3bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6fa98c740 4 client.17223 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:26:24.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56535a2aba40 con 0x56535a3ba1c0 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.751+0000 7ff6f5000640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a3ba1c0 0x56535a3bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.752+0000 7ff6f5000640 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a3ba1c0 0x56535a3bc5e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=867ac438b1e7f751 server_cookie=cce9c20759df4f59 in_seq=0 out_seq=0 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.752+0000 7ff6d6a00640 10 client.17223 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6d6a00640 1 -- 172.21.15.57:0/1002519554 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56535a2aba40 con 0x56535a3ba1c0 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6d6a00640 10 client.17223 handle_command_reply: tid=0 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 client.17223 shutdown 2024-04-13T05:26:24.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 msgr2=0x56535a34c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a34c050 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0bb4d0 tx=0x7ff6dc0095e0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 shutdown_connections 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x56535a3ba1c0 0x56535a3bc5e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56535a33dfa0 0x56535a34d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56535a338e30 0x56535a34c050 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 --2- 172.21.15.57:0/1002519554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56535a338440 0x56535a34bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6d6a00640 0 client.17223 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 >> 172.21.15.57:0/1002519554 conn(0x56535a3339b0 msgr2=0x56535a336c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 shutdown_connections 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 1 -- 172.21.15.57:0/1002519554 wait complete. 2024-04-13T05:26:24.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.753+0000 7ff6fa98c740 20 client.17223 trim_cache size 0 max 16384 2024-04-13T05:26:24.760 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:26:24.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff6d80a3c30 msgr2=0x7ff6d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff6d80a3c30 0x7ff6d80a60f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x56535a339690 tx=0x7ff6e8008470 comp rx=0 tx=0).stop 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 msgr2=0x7ff6f0198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0198500 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc036040 tx=0x7ff6dc0042f0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 shutdown_connections 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff6d80a3c30 0x7ff6d80a60f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f0107bb0 0x7ff6f019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f0105280 0x7ff6f0198500 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 --2- 172.21.15.57:0/1156243055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6f0102950 0x7ff6f0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 >> 172.21.15.57:0/1156243055 conn(0x7ff6f00fc360 msgr2=0x7ff6f0103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 shutdown_connections 2024-04-13T05:26:24.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.754+0000 7ff6f6400640 1 -- 172.21.15.57:0/1156243055 wait complete. 2024-04-13T05:26:24.775 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:26:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 -- 172.21.15.57:0/2521358403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 msgr2=0x7f3dc4105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 --2- 172.21.15.57:0/2521358403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc4105770 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009900 tx=0x7f3db002bc30 comp rx=0 tx=0).stop 2024-04-13T05:26:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 -- 172.21.15.57:0/2521358403 shutdown_connections 2024-04-13T05:26:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 --2- 172.21.15.57:0/2521358403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 0x7f3dc410c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 --2- 172.21.15.57:0/2521358403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc4105770 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 --2- 172.21.15.57:0/2521358403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 0x7f3dc4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 -- 172.21.15.57:0/2521358403 >> 172.21.15.57:0/2521358403 conn(0x7f3dc40ff670 msgr2=0x7f3dc4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 -- 172.21.15.57:0/2521358403 shutdown_connections 2024-04-13T05:26:24.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.968+0000 7f3dc9800640 1 -- 172.21.15.57:0/2521358403 wait complete. 2024-04-13T05:26:24.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.969+0000 7f3dc9800640 1 Processor -- start 2024-04-13T05:26:24.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.969+0000 7f3dc9800640 1 -- start start 2024-04-13T05:26:24.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 0x7f3dc419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc9800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 0x7f3dc41a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc9800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc410fc90 con 0x7f3dc41052f0 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc410fb10 con 0x7f3dc4104100 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc9800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc410fe10 con 0x7f3dc4105cb0 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46198/0 (socket says 172.21.15.57:46198) 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc3400640 1 -- 172.21.15.57:0/287525735 learned_addr learned my addr 172.21.15.57:0/287525735 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.970+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 0x7f3dc419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 0x7f3dc41a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3400640 1 -- 172.21.15.57:0/287525735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 msgr2=0x7f3dc41a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3400640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 0x7f3dc41a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3400640 1 -- 172.21.15.57:0/287525735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 msgr2=0x7f3dc419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3400640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 0x7f3dc419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3400640 1 -- 172.21.15.57:0/287525735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc41a1f40 con 0x7f3dc41052f0 2024-04-13T05:26:24.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 0x7f3dc41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:24.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 0x7f3dc419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:24.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc3400640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc419a860 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3db0000c00 tx=0x7f3db00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc0c00640 1 -- 172.21.15.57:0/287525735 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0043020 con 0x7f3dc41052f0 2024-04-13T05:26:24.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc0c00640 1 -- 172.21.15.57:0/287525735 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db00054c0 con 0x7f3dc41052f0 2024-04-13T05:26:24.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc41a21d0 con 0x7f3dc41052f0 2024-04-13T05:26:24.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.971+0000 7f3dc0c00640 1 -- 172.21.15.57:0/287525735 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003d40 con 0x7f3dc41052f0 2024-04-13T05:26:24.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.972+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc41a2720 con 0x7f3dc41052f0 2024-04-13T05:26:24.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 Processor -- start 2024-04-13T05:26:24.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 -- start start 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dc0c00640 1 -- 172.21.15.57:0/287525735 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3db0046020 con 0x7f3dc41052f0 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc4b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562a7fbac020 con 0x562a7fc46e30 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562a7fb52950 con 0x562a7fc46440 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.975+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a7fba7fe0 con 0x562a7fc4bfa0 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc4b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc4b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46210/0 (socket says 172.21.15.57:46210) 2024-04-13T05:26:24.979 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 -- 172.21.15.57:0/1825152459 learned_addr learned my addr 172.21.15.57:0/1825152459 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3400640 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 -- 172.21.15.57:0/1825152459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 msgr2=0x562a7fc4e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc4e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 -- 172.21.15.57:0/1825152459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 msgr2=0x562a7fc46860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc46860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.976+0000 7f3dc3e00640 1 -- 172.21.15.57:0/1825152459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a7f8be3f0 con 0x562a7fc46e30 2024-04-13T05:26:24.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc4b930 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3db400bd30 tx=0x7f3db400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdee93087c36a9f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3db9600640 1 -- 172.21.15.57:0/1825152459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4015070 con 0x562a7fc46e30 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3db9600640 1 -- 172.21.15.57:0/1825152459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db400be20 con 0x562a7fc46e30 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3db9600640 1 -- 172.21.15.57:0/1825152459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db40106c0 con 0x562a7fc46e30 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3dc0c00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3da80a3bb0 0x7f3da80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3dc0c00640 1 -- 172.21.15.57:0/287525735 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3db00f2e50 con 0x7f3dc41052f0 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.977+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3da80a3bb0 0x7f3da80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 -- 172.21.15.57:0/1825152459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 msgr2=0x562a7fc4b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc4b930 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3db400bd30 tx=0x7f3db400f730 comp rx=0 tx=0).stop 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 -- 172.21.15.57:0/1825152459 shutdown_connections 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc4e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc4b930 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/1825152459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc46860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 -- 172.21.15.57:0/1825152459 >> 172.21.15.57:0/1825152459 conn(0x562a7fc419b0 msgr2=0x562a7fc41db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 -- 172.21.15.57:0/1825152459 shutdown_connections 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3da80a3bb0 0x7f3da80a6070 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x562a7fc47690 tx=0x7f3dbc0083f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.978+0000 7f3dcdd94740 1 -- 172.21.15.57:0/1825152459 wait complete. 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 -- start start 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 Processor -- start 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc46440 0x562a7fc50970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc50eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562a7fc4ef50 con 0x562a7fc46e30 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562a7fb65c80 con 0x562a7fc4bfa0 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a7fc4f0d0 con 0x562a7fc46440 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.979+0000 7f3dcdd94740 10 client.0 _ll_register_callbacks cb 0x562a7fb65670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc50eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc50eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46220/0 (socket says 172.21.15.57:46220) 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 -- 172.21.15.57:0/517230087 learned_addr learned my addr 172.21.15.57:0/517230087 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc46440 0x562a7fc50970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 -- 172.21.15.57:0/517230087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc46440 msgr2=0x562a7fc50970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc46440 0x562a7fc50970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 -- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc4bfa0 msgr2=0x562a7fc57ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:24.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 -- 172.21.15.57:0/517230087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a7f9887d0 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dc3400640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc50eb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3db00bebb0 tx=0x7f3db00bed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4726a447573d211a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.980+0000 7f3dae000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dae000640 1 -- 172.21.15.57:0/517230087 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db00d8060 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db400be20 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dae000640 1 -- 172.21.15.57:0/517230087 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3db00035f0 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dae000640 1 -- 172.21.15.57:0/517230087 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3db003ce30 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 10 client.17235 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562a7fc5b260 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dae000640 1 -- 172.21.15.57:0/517230087 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db400be20 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dae000640 1 -- 172.21.15.57:0/517230087 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f3db00d8200 con 0x562a7fc46e30 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 10 client.17235 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:24.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:24.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 20 client.17235 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:24.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fc5c8f0 0x562a7fc5ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:24.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 4 client.17235 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:24.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.981+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a7fb4b750 con 0x562a7fc5c8f0 2024-04-13T05:26:24.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.982+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fc5c8f0 0x562a7fc5ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:24.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.984+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fc5c8f0 0x562a7fc5ed50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77a056cda0fd66f server_cookie=34189a47260fa642 in_seq=0 out_seq=0 2024-04-13T05:26:24.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.984+0000 7f3dae000640 10 client.17235 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:24.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.985+0000 7f3dae000640 1 -- 172.21.15.57:0/517230087 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a7fb4b750 con 0x562a7fc5c8f0 2024-04-13T05:26:24.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dae000640 10 client.17235 handle_command_reply: tid=0 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 client.17235 shutdown 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 msgr2=0x562a7fc50eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc50eb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3db00bebb0 tx=0x7f3db00bed40 comp rx=0 tx=0).stop 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 shutdown_connections 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fc5c8f0 0x562a7fc5ed50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46e30 0x562a7fc50eb0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/517230087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc46440 0x562a7fc50970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dae000640 0 client.17235 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 >> 172.21.15.57:0/517230087 conn(0x562a7fc419b0 msgr2=0x562a7fc40cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 shutdown_connections 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 1 -- 172.21.15.57:0/517230087 wait complete. 2024-04-13T05:26:24.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:24.986+0000 7f3dcdd94740 20 client.17235 trim_cache size 0 max 16384 2024-04-13T05:26:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 Processor -- start 2024-04-13T05:26:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 -- start start 2024-04-13T05:26:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc40440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc3cfc0 0x562a7fc40980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41af560 con 0x562a7fc4bfa0 2024-04-13T05:26:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41acd70 con 0x562a7fc46440 2024-04-13T05:26:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41aabc0 con 0x562a7fc3cfc0 2024-04-13T05:26:25.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:25.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46232/0 (socket says 172.21.15.57:46232) 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.002+0000 7f3dc3400640 1 -- 172.21.15.57:0/2678691251 learned_addr learned my addr 172.21.15.57:0/2678691251 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc40440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3400640 1 -- 172.21.15.57:0/2678691251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc3cfc0 msgr2=0x562a7fc40980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3400640 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc3cfc0 0x562a7fc40980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3400640 1 -- 172.21.15.57:0/2678691251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 msgr2=0x562a7fc40440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3400640 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc40440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3400640 1 -- 172.21.15.57:0/2678691251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a7fc476c0 con 0x562a7fc4bfa0 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3dc3400640 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57eb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3db00bb310 tx=0x7f3db00bedb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb7d8783eedb24d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3db8c00640 1 -- 172.21.15.57:0/2678691251 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db00d8050 con 0x562a7fc4bfa0 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3db8c00640 1 -- 172.21.15.57:0/2678691251 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db00c7690 con 0x562a7fc4bfa0 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.003+0000 7f3db8c00640 1 -- 172.21.15.57:0/2678691251 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db00beeb0 con 0x562a7fc4bfa0 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 -- 172.21.15.57:0/2678691251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 msgr2=0x562a7fc57eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57eb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3db00bb310 tx=0x7f3db00bedb0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 -- 172.21.15.57:0/2678691251 shutdown_connections 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc3cfc0 0x562a7fc40980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc4bfa0 0x562a7fc57eb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/2678691251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc46440 0x562a7fc40440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 -- 172.21.15.57:0/2678691251 >> 172.21.15.57:0/2678691251 conn(0x562a7fc419b0 msgr2=0x562a7fc50970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 -- 172.21.15.57:0/2678691251 shutdown_connections 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 -- 172.21.15.57:0/2678691251 wait complete. 2024-04-13T05:26:25.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.004+0000 7f3dcdd94740 1 -- start start 2024-04-13T05:26:25.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 Processor -- start 2024-04-13T05:26:25.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc3cfc0 0x562a7fc55a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 0x562a7fc55f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ab230 con 0x562a7fc46440 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ac6a0 con 0x562a7fc3cfc0 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc41ab900 con 0x562a7fc4bfa0 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dcdd94740 10 client.0 _ll_register_callbacks cb 0x7f3db00095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 0x562a7fc55f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 0x562a7fc55f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46238/0 (socket says 172.21.15.57:46238) 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dc3400640 1 -- 172.21.15.57:0/990447542 learned_addr learned my addr 172.21.15.57:0/990447542 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.005+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc3cfc0 0x562a7fc55a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:25.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc3400640 1 -- 172.21.15.57:0/990447542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 msgr2=0x562a7fc56f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc3400640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc56f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc3400640 1 -- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc3cfc0 msgr2=0x562a7fc55a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc3400640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc3cfc0 0x562a7fc55a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc3400640 1 -- 172.21.15.57:0/990447542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a7fc57560 con 0x562a7fc46440 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc8e00640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dc3400640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 0x562a7fc55f60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3db0004dd0 tx=0x7f3db00c32e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e9336816b76a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dae000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dae000640 1 -- 172.21.15.57:0/990447542 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db00d8040 con 0x562a7fc46440 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dae000640 1 -- 172.21.15.57:0/990447542 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f3db0005010 con 0x562a7fc46440 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db00c7690 con 0x562a7fc46440 2024-04-13T05:26:25.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dae000640 1 -- 172.21.15.57:0/990447542 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3db00beeb0 con 0x562a7fc46440 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dcdd94740 10 client.17241 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.006+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562a7fcc32a0 con 0x562a7fc46440 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dae000640 1 -- 172.21.15.57:0/990447542 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db00c7690 con 0x562a7fc46440 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dae000640 1 -- 172.21.15.57:0/990447542 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f3db00bad90 con 0x562a7fc46440 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dcdd94740 10 client.17241 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dcdd94740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dcdd94740 20 client.17241 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fcc39e0 0x562a7fcc5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dcdd94740 4 client.17241 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.007+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a7fbb9a40 con 0x562a7fcc39e0 2024-04-13T05:26:25.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.008+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fcc39e0 0x562a7fcc5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.009+0000 7f3dc3e00640 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fcc39e0 0x562a7fcc5e00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857ce2bc7e225475 server_cookie=a1585c3f5ac199e6 in_seq=0 out_seq=0 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.009+0000 7f3dae000640 10 client.17241 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.009+0000 7f3dae000640 1 -- 172.21.15.57:0/990447542 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a7fbb9a40 con 0x562a7fcc39e0 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.009+0000 7f3dae000640 10 client.17241 handle_command_reply: tid=0 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 client.17241 shutdown 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 msgr2=0x562a7fc55f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 0x562a7fc55f60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3db0004dd0 tx=0x7f3db00c32e0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 shutdown_connections 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x562a7fcc39e0 0x562a7fcc5e00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a7fc4bfa0 0x562a7fc56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562a7fc46440 0x562a7fc55f60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 --2- 172.21.15.57:0/990447542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a7fc3cfc0 0x562a7fc55a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dae000640 0 client.17241 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:25.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 >> 172.21.15.57:0/990447542 conn(0x562a7fc419b0 msgr2=0x562a7fc44c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:25.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 shutdown_connections 2024-04-13T05:26:25.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 1 -- 172.21.15.57:0/990447542 wait complete. 2024-04-13T05:26:25.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.010+0000 7f3dcdd94740 20 client.17241 trim_cache size 0 max 16384 2024-04-13T05:26:25.016 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3da80a3bb0 msgr2=0x7f3da80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3da80a3bb0 0x7f3da80a6070 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x562a7fc47690 tx=0x7f3dbc0083f0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 msgr2=0x7f3dc419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc419a860 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3db0000c00 tx=0x7f3db00042f0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 shutdown_connections 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f3da80a3bb0 0x7f3da80a6070 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dc4105cb0 0x7f3dc41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc41052f0 0x7f3dc419a860 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 --2- 172.21.15.57:0/287525735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dc4104100 0x7f3dc419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:25.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 >> 172.21.15.57:0/287525735 conn(0x7f3dc40ff670 msgr2=0x7f3dc4108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:25.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 shutdown_connections 2024-04-13T05:26:25.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:25.011+0000 7f3dc9800640 1 -- 172.21.15.57:0/287525735 wait complete. 2024-04-13T05:26:25.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:24 smithi057 ceph-mon[32331]: pgmap v720: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 361 KiB/s wr, 192 op/s 2024-04-13T05:26:25.027 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:24 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2236335969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:26:25.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:24 smithi083 ceph-mon[35133]: pgmap v720: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 361 KiB/s wr, 192 op/s 2024-04-13T05:26:25.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:24 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2236335969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:26:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:24 smithi189 ceph-mon[35531]: pgmap v720: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 361 KiB/s wr, 192 op/s 2024-04-13T05:26:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:24 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2236335969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:26:26.030 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:26:26.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:26:26.031 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:26:26.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.328+0000 7fbce9600640 1 -- 172.21.15.57:0/2903750146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce41052f0 msgr2=0x7fbce4105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.328+0000 7fbce9600640 1 --2- 172.21.15.57:0/2903750146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce41052f0 0x7fbce4105770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009900 tx=0x7fbcd002bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:26.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.328+0000 7fbce9600640 1 -- 172.21.15.57:0/2903750146 shutdown_connections 2024-04-13T05:26:26.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.328+0000 7fbce9600640 1 --2- 172.21.15.57:0/2903750146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce4105cb0 0x7fbce410c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.328+0000 7fbce9600640 1 --2- 172.21.15.57:0/2903750146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce41052f0 0x7fbce4105770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.328+0000 7fbce9600640 1 --2- 172.21.15.57:0/2903750146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4104100 0x7fbce4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.329+0000 7fbce9600640 1 -- 172.21.15.57:0/2903750146 >> 172.21.15.57:0/2903750146 conn(0x7fbce40ff670 msgr2=0x7fbce4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.329+0000 7fbce9600640 1 -- 172.21.15.57:0/2903750146 shutdown_connections 2024-04-13T05:26:26.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.330+0000 7fbce9600640 1 -- 172.21.15.57:0/2903750146 wait complete. 2024-04-13T05:26:26.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.332+0000 7fbce9600640 1 Processor -- start 2024-04-13T05:26:26.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.333+0000 7fbce9600640 1 -- start start 2024-04-13T05:26:26.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.334+0000 7fbce9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce4104100 0x7fbce419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.334+0000 7fbce9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 0x7fbce419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.334+0000 7fbce9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4105cb0 0x7fbce41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.334+0000 7fbce3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 0x7fbce419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.335+0000 7fbce3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 0x7fbce419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:48660/0 (socket says 172.21.15.57:48660) 2024-04-13T05:26:26.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.336+0000 7fbce3400640 1 -- 172.21.15.57:0/1282825553 learned_addr learned my addr 172.21.15.57:0/1282825553 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.336+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410fc90 con 0x7fbce4105cb0 2024-04-13T05:26:26.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.336+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410fb10 con 0x7fbce41052f0 2024-04-13T05:26:26.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.336+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410fe10 con 0x7fbce4104100 2024-04-13T05:26:26.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.336+0000 7fbce3e00640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce4104100 0x7fbce419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.336+0000 7fbce8c00640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4105cb0 0x7fbce41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3400640 1 -- 172.21.15.57:0/1282825553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce4104100 msgr2=0x7fbce419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3400640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce4104100 0x7fbce419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3400640 1 -- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4105cb0 msgr2=0x7fbce41a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3400640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4105cb0 0x7fbce41a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3400640 1 -- 172.21.15.57:0/1282825553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce41a1f30 con 0x7fbce41052f0 2024-04-13T05:26:26.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3e00640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce4104100 0x7fbce419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:26:26.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.337+0000 7fbce3400640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 0x7fbce419a850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0000c00 tx=0x7fbcd0041c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.339+0000 7fbce0c00640 1 -- 172.21.15.57:0/1282825553 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0004450 con 0x7fbce41052f0 2024-04-13T05:26:26.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.339+0000 7fbce0c00640 1 -- 172.21.15.57:0/1282825553 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd0004aa0 con 0x7fbce41052f0 2024-04-13T05:26:26.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.339+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce4076400 con 0x7fbce41052f0 2024-04-13T05:26:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.339+0000 7fbce0c00640 1 -- 172.21.15.57:0/1282825553 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0003c20 con 0x7fbce41052f0 2024-04-13T05:26:26.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.339+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce4076940 con 0x7fbce41052f0 2024-04-13T05:26:26.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.345+0000 7fbce0c00640 1 -- 172.21.15.57:0/1282825553 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbcd0004c50 con 0x7fbce41052f0 2024-04-13T05:26:26.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.346+0000 7fbcedc24740 1 Processor -- start 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.346+0000 7fbcedc24740 1 -- start start 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.346+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fe420 0x55deeb4fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.346+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb503910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb506410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbcedc24740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55deeb464bc0 con 0x55deeb4fee10 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbcedc24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55deeb40b660 con 0x55deeb4fe420 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbcedc24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55deeb460b80 con 0x55deeb503f80 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbce3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb506410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbce3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb506410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:47272/0 (socket says 172.21.15.57:47272) 2024-04-13T05:26:26.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.347+0000 7fbce3400640 1 -- 172.21.15.57:0/1941918433 learned_addr learned my addr 172.21.15.57:0/1941918433 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce0c00640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbcc80a3c30 0x7fbcc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce3e00640 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb503910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce3400640 1 -- 172.21.15.57:0/1941918433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fe420 msgr2=0x55deeb4fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce3400640 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fe420 0x55deeb4fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce3400640 1 -- 172.21.15.57:0/1941918433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 msgr2=0x55deeb503910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce3400640 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb503910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce0c00640 1 -- 172.21.15.57:0/1282825553 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbcd00fa740 con 0x7fbce41052f0 2024-04-13T05:26:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.348+0000 7fbce3400640 1 -- 172.21.15.57:0/1941918433 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deeb42e220 con 0x55deeb503f80 2024-04-13T05:26:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.349+0000 7fbce8c00640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbcc80a3c30 0x7fbcc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.349+0000 7fbce3e00640 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb503910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.349+0000 7fbce3400640 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb506410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00f7ef0 tx=0x7fbcd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddfd4842c75a949 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.350+0000 7fbcd9200640 1 -- 172.21.15.57:0/1941918433 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00c6500 con 0x55deeb503f80 2024-04-13T05:26:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.350+0000 7fbcd9200640 1 -- 172.21.15.57:0/1941918433 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd00bf990 con 0x55deeb503f80 2024-04-13T05:26:26.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.350+0000 7fbcd9200640 1 -- 172.21.15.57:0/1941918433 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00c6090 con 0x55deeb503f80 2024-04-13T05:26:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.351+0000 7fbcedc24740 1 -- 172.21.15.57:0/1941918433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 msgr2=0x55deeb506410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.351+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb506410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00f7ef0 tx=0x7fbcd0009260 comp rx=0 tx=0).stop 2024-04-13T05:26:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.351+0000 7fbce8c00640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbcc80a3c30 0x7fbcc80a60f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55deeb4ff670 tx=0x7fbcdc009b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.352+0000 7fbcedc24740 1 -- 172.21.15.57:0/1941918433 shutdown_connections 2024-04-13T05:26:26.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.352+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb506410 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.352+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb503910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.352+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1941918433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fe420 0x55deeb4fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.352+0000 7fbcedc24740 1 -- 172.21.15.57:0/1941918433 >> 172.21.15.57:0/1941918433 conn(0x55deeb4f9990 msgr2=0x55deeb4f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.353+0000 7fbcedc24740 1 -- 172.21.15.57:0/1941918433 shutdown_connections 2024-04-13T05:26:26.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.353+0000 7fbcedc24740 1 -- 172.21.15.57:0/1941918433 wait complete. 2024-04-13T05:26:26.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.354+0000 7fbcedc24740 1 -- start start 2024-04-13T05:26:26.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.354+0000 7fbcedc24740 1 Processor -- start 2024-04-13T05:26:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.355+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb519490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.355+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb5199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.356+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb503f80 0x55deeb520a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.356+0000 7fbcedc24740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55deeb460b80 con 0x55deeb4fee10 2024-04-13T05:26:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.356+0000 7fbcedc24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55deeb41ea30 con 0x55deeb503f80 2024-04-13T05:26:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.356+0000 7fbcedc24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55deeb517d00 con 0x55deeb4fe420 2024-04-13T05:26:26.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.356+0000 7fbce3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb519490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.357+0000 7fbce3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb519490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:47286/0 (socket says 172.21.15.57:47286) 2024-04-13T05:26:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.357+0000 7fbce3e00640 1 -- 172.21.15.57:0/4196616397 learned_addr learned my addr 172.21.15.57:0/4196616397 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.357+0000 7fbcedc24740 10 client.0 _ll_register_callbacks cb 0x55deeb176c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.357+0000 7fbce3400640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb5199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.357+0000 7fbce8c00640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb503f80 0x55deeb520a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.358+0000 7fbce3400640 1 -- 172.21.15.57:0/4196616397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 msgr2=0x55deeb519490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.358+0000 7fbce3400640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb519490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.358+0000 7fbce3400640 1 -- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb503f80 msgr2=0x55deeb520a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.358+0000 7fbce3400640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb503f80 0x55deeb520a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.358+0000 7fbce3400640 1 -- 172.21.15.57:0/4196616397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deeb23f7e0 con 0x55deeb4fee10 2024-04-13T05:26:26.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.359+0000 7fbce3400640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb5199d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00f8210 tx=0x7fbcd00c6090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9b4510b73b22fe server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.359+0000 7fbccd600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:26.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.360+0000 7fbccd600640 1 -- 172.21.15.57:0/4196616397 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00c6270 con 0x55deeb4fee10 2024-04-13T05:26:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.360+0000 7fbccd600640 1 -- 172.21.15.57:0/4196616397 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbcd00b10d0 con 0x55deeb4fee10 2024-04-13T05:26:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.360+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd00bf990 con 0x55deeb4fee10 2024-04-13T05:26:26.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.361+0000 7fbccd600640 1 -- 172.21.15.57:0/4196616397 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbcd00cf2f0 con 0x55deeb4fee10 2024-04-13T05:26:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.362+0000 7fbcedc24740 10 client.17250 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:26.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.362+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55deeb29ff70 con 0x55deeb4fee10 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbccd600640 1 -- 172.21.15.57:0/4196616397 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd00bf990 con 0x55deeb4fee10 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbccd600640 1 -- 172.21.15.57:0/4196616397 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fbcd003c020 con 0x55deeb4fee10 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbcedc24740 10 client.17250 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbcedc24740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbcedc24740 20 client.17250 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbcedc24740 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb527590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbcedc24740 4 client.17250 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.363+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deeb4725e0 con 0x55deeb525170 2024-04-13T05:26:26.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.364+0000 7fbce3e00640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb527590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.365+0000 7fbce3e00640 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb527590 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853f78467ee72c1a server_cookie=6fb2042ee19afd67 in_seq=0 out_seq=0 2024-04-13T05:26:26.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.365+0000 7fbccd600640 10 client.17250 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.367+0000 7fbccd600640 1 -- 172.21.15.57:0/4196616397 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55deeb4725e0 con 0x55deeb525170 2024-04-13T05:26:26.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.367+0000 7fbccd600640 10 client.17250 handle_command_reply: tid=0 2024-04-13T05:26:26.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.367+0000 7fbcedc24740 1 client.17250 shutdown 2024-04-13T05:26:26.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.368+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 msgr2=0x55deeb5199d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.368+0000 7fbcedc24740 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb5199d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00f8210 tx=0x7fbcd00c6090 comp rx=0 tx=0).stop 2024-04-13T05:26:26.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.368+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 shutdown_connections 2024-04-13T05:26:26.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.368+0000 7fbcedc24740 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb527590 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbcedc24740 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb503f80 0x55deeb520a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbccd600640 0 client.17250 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.371 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbcedc24740 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fee10 0x55deeb5199d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbcedc24740 1 --2- 172.21.15.57:0/4196616397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb519490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 >> 172.21.15.57:0/4196616397 conn(0x55deeb4f9990 msgr2=0x55deeb4f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 shutdown_connections 2024-04-13T05:26:26.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.369+0000 7fbcedc24740 1 -- 172.21.15.57:0/4196616397 wait complete. 2024-04-13T05:26:26.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.370+0000 7fbcedc24740 20 client.17250 trim_cache size 0 max 16384 2024-04-13T05:26:26.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.430+0000 7fbcedc24740 1 Processor -- start 2024-04-13T05:26:26.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.430+0000 7fbcedc24740 1 -- start start 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.431+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fe420 0x55deeb4fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.431+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb5209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.431+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb52be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.431+0000 7fbcedc24740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41ab0a0 con 0x55deeb4fe420 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.431+0000 7fbcedc24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a88b0 con 0x55deeb4fee10 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.431+0000 7fbcedc24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a6700 con 0x55deeb503f80 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fe420 0x55deeb4fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb52be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb52be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:47294/0 (socket says 172.21.15.57:47294) 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce8c00640 1 -- 172.21.15.57:0/1014655820 learned_addr learned my addr 172.21.15.57:0/1014655820 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb5209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce3400640 1 -- 172.21.15.57:0/1014655820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 msgr2=0x55deeb52be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce3400640 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb52be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce3400640 1 -- 172.21.15.57:0/1014655820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fe420 msgr2=0x55deeb4fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.432+0000 7fbce3400640 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fe420 0x55deeb4fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.433+0000 7fbce3400640 1 -- 172.21.15.57:0/1014655820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deeb500a30 con 0x55deeb4fee10 2024-04-13T05:26:26.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.433+0000 7fbce3400640 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb5209d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00c66c0 tx=0x7fbcd0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93f3abe4758efb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.435+0000 7fbccfe00640 1 -- 172.21.15.57:0/1014655820 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00c6270 con 0x55deeb4fee10 2024-04-13T05:26:26.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.435+0000 7fbccfe00640 1 -- 172.21.15.57:0/1014655820 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd00bfbd0 con 0x55deeb4fee10 2024-04-13T05:26:26.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.435+0000 7fbccfe00640 1 -- 172.21.15.57:0/1014655820 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00c6500 con 0x55deeb4fee10 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.436+0000 7fbcedc24740 1 -- 172.21.15.57:0/1014655820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 msgr2=0x55deeb5209d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.436+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb5209d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00c66c0 tx=0x7fbcd0009260 comp rx=0 tx=0).stop 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 -- 172.21.15.57:0/1014655820 shutdown_connections 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb5209d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb503f80 0x55deeb52be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 --2- 172.21.15.57:0/1014655820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb4fe420 0x55deeb4fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 -- 172.21.15.57:0/1014655820 >> 172.21.15.57:0/1014655820 conn(0x55deeb4f9990 msgr2=0x55deeb4fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 -- 172.21.15.57:0/1014655820 shutdown_connections 2024-04-13T05:26:26.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.437+0000 7fbcedc24740 1 -- 172.21.15.57:0/1014655820 wait complete. 2024-04-13T05:26:26.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.438+0000 7fbcedc24740 1 -- start start 2024-04-13T05:26:26.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.438+0000 7fbcedc24740 1 Processor -- start 2024-04-13T05:26:26.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.440+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb520570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.440+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb520ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.440+0000 7fbcedc24740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 0x55deeb51ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbcedc24740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a6d70 con 0x55deeb503f80 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbce3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb520570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 0x55deeb51ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbcedc24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a81e0 con 0x55deeb4fee10 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbce8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 0x55deeb51ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46288/0 (socket says 172.21.15.57:46288) 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbce8c00640 1 -- 172.21.15.57:0/450308278 learned_addr learned my addr 172.21.15.57:0/450308278 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbce3400640 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb520ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a7440 con 0x55deeb4fe420 2024-04-13T05:26:26.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.441+0000 7fbcedc24740 10 client.0 _ll_register_callbacks cb 0x7fbcd00c2a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:26.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.442+0000 7fbce8c00640 1 -- 172.21.15.57:0/450308278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 msgr2=0x55deeb520570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.442+0000 7fbce8c00640 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb520570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.443+0000 7fbce8c00640 1 -- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 msgr2=0x55deeb520ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.443+0000 7fbce8c00640 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb520ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.443+0000 7fbce8c00640 1 -- 172.21.15.57:0/450308278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deeb137170 con 0x55deeb503f80 2024-04-13T05:26:26.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.443+0000 7fbce8c00640 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 0x55deeb51ec70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc010df0 tx=0x7fbcdc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea07d103a63af7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.443+0000 7fbccd600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:26.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.444+0000 7fbccd600640 1 -- 172.21.15.57:0/450308278 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003c40 con 0x55deeb503f80 2024-04-13T05:26:26.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.444+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd00bfbd0 con 0x55deeb503f80 2024-04-13T05:26:26.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.444+0000 7fbccd600640 1 -- 172.21.15.57:0/450308278 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbcdc003de0 con 0x55deeb503f80 2024-04-13T05:26:26.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.445+0000 7fbccd600640 1 -- 172.21.15.57:0/450308278 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbcdc01e920 con 0x55deeb503f80 2024-04-13T05:26:26.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.445+0000 7fbcedc24740 10 client.17256 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:26.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.445+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55deeb41fbe0 con 0x55deeb503f80 2024-04-13T05:26:26.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.446+0000 7fbccd600640 1 -- 172.21.15.57:0/450308278 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd00bfbd0 con 0x55deeb503f80 2024-04-13T05:26:26.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbccd600640 1 -- 172.21.15.57:0/450308278 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fbcdc028420 con 0x55deeb503f80 2024-04-13T05:26:26.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbcedc24740 10 client.17256 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:26.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbcedc24740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:26.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbcedc24740 20 client.17256 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:26.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbcedc24740 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb58eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbcedc24740 4 client.17256 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:26:26.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.447+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55deeb46c450 con 0x55deeb525170 2024-04-13T05:26:26.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.448+0000 7fbce3e00640 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb58eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.449+0000 7fbce3e00640 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb58eac0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84440b648d820bc4 server_cookie=ac7f1a6c6745e6fe in_seq=0 out_seq=0 2024-04-13T05:26:26.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.449+0000 7fbccd600640 10 client.17256 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.451+0000 7fbccd600640 1 -- 172.21.15.57:0/450308278 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55deeb46c450 con 0x55deeb525170 2024-04-13T05:26:26.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.451+0000 7fbccd600640 10 client.17256 handle_command_reply: tid=0 2024-04-13T05:26:26.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.451+0000 7fbcedc24740 1 client.17256 shutdown 2024-04-13T05:26:26.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.453+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 msgr2=0x55deeb51ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 0x55deeb51ec70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc010df0 tx=0x7fbcdc009500 comp rx=0 tx=0).stop 2024-04-13T05:26:26.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 shutdown_connections 2024-04-13T05:26:26.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55deeb525170 0x55deeb58eac0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55deeb4fee10 0x55deeb520ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55deeb503f80 0x55deeb51ec70 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 --2- 172.21.15.57:0/450308278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deeb4fe420 0x55deeb520570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbccd600640 0 client.17256 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.454+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 >> 172.21.15.57:0/450308278 conn(0x55deeb4f9990 msgr2=0x55deeb4fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.455+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 shutdown_connections 2024-04-13T05:26:26.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.455+0000 7fbcedc24740 1 -- 172.21.15.57:0/450308278 wait complete. 2024-04-13T05:26:26.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.455+0000 7fbcedc24740 20 client.17256 trim_cache size 0 max 16384 2024-04-13T05:26:26.458 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:26:26.458 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:26:26.459 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "c8a2cc7e-f138-47fd-a7a3-92bc5852c40d", 2024-04-13T05:26:26.459 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:26:26.459 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:26:26.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbcc80a3c30 msgr2=0x7fbcc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbcc80a3c30 0x7fbcc80a60f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55deeb4ff670 tx=0x7fbcdc009b70 comp rx=0 tx=0).stop 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 msgr2=0x7fbce419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 0x7fbce419a850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0000c00 tx=0x7fbcd0041c90 comp rx=0 tx=0).stop 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 shutdown_connections 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbcc80a3c30 0x7fbcc80a60f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbce41052f0 0x7fbce419a850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbce4105cb0 0x7fbce41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 --2- 172.21.15.57:0/1282825553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbce4104100 0x7fbce419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 >> 172.21.15.57:0/1282825553 conn(0x7fbce40ff670 msgr2=0x7fbce41086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.457+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 shutdown_connections 2024-04-13T05:26:26.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.458+0000 7fbce9600640 1 -- 172.21.15.57:0/1282825553 wait complete. 2024-04-13T05:26:26.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:26:26.482 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 -- 172.21.15.57:0/1520814076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8076980 msgr2=0x7f8ab8076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 --2- 172.21.15.57:0/1520814076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8076980 0x7f8ab8076da0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa802bc30 comp rx=0 tx=0).stop 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 -- 172.21.15.57:0/1520814076 shutdown_connections 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 --2- 172.21.15.57:0/1520814076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ab8078560 0x7f8ab8078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 --2- 172.21.15.57:0/1520814076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ab8077bc0 0x7f8ab8078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 --2- 172.21.15.57:0/1520814076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8076980 0x7f8ab8076da0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 -- 172.21.15.57:0/1520814076 >> 172.21.15.57:0/1520814076 conn(0x7f8ab80ff7b0 msgr2=0x7f8ab8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 -- 172.21.15.57:0/1520814076 shutdown_connections 2024-04-13T05:26:26.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.699+0000 7f8abdc00640 1 -- 172.21.15.57:0/1520814076 wait complete. 2024-04-13T05:26:26.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.700+0000 7f8abdc00640 1 Processor -- start 2024-04-13T05:26:26.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 -- start start 2024-04-13T05:26:26.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ab8076980 0x7f8ab819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ab8077bc0 0x7f8ab819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 0x7f8ab81a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81141c0 con 0x7f8ab8078560 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8114040 con 0x7f8ab8077bc0 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8abdc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8114340 con 0x7f8ab8076980 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 0x7f8ab81a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.701+0000 7f8ab6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ab8076980 0x7f8ab819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 0x7f8ab81a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46300/0 (socket says 172.21.15.57:46300) 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 -- 172.21.15.57:0/3495901055 learned_addr learned my addr 172.21.15.57:0/3495901055 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab6000640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ab8077bc0 0x7f8ab819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 -- 172.21.15.57:0/3495901055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ab8076980 msgr2=0x7f8ab819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ab8076980 0x7f8ab819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 -- 172.21.15.57:0/3495901055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ab8077bc0 msgr2=0x7f8ab819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ab8077bc0 0x7f8ab819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.705 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.702+0000 7f8ab7400640 1 -- 172.21.15.57:0/3495901055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab81a64d0 con 0x7f8ab8078560 2024-04-13T05:26:26.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.703+0000 7f8ab7400640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 0x7f8ab81a5e60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000eb10 tx=0x7f8ab000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.703+0000 7f8aaf400640 1 -- 172.21.15.57:0/3495901055 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0004b20 con 0x7f8ab8078560 2024-04-13T05:26:26.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.703+0000 7f8aaf400640 1 -- 172.21.15.57:0/3495901055 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ab0004cc0 con 0x7f8ab8078560 2024-04-13T05:26:26.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.703+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab81a67c0 con 0x7f8ab8078560 2024-04-13T05:26:26.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.704+0000 7f8aaf400640 1 -- 172.21.15.57:0/3495901055 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0010690 con 0x7f8ab8078560 2024-04-13T05:26:26.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.704+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab81a6d10 con 0x7f8ab8078560 2024-04-13T05:26:26.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.708+0000 7f8ac0b9f740 1 Processor -- start 2024-04-13T05:26:26.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.708+0000 7f8aaf400640 1 -- 172.21.15.57:0/3495901055 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ab0005020 con 0x7f8ab8078560 2024-04-13T05:26:26.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.709+0000 7f8ac0b9f740 1 -- start start 2024-04-13T05:26:26.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a35668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a356b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a356e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e9a34cd020 con 0x55e9a356c030 2024-04-13T05:26:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9a34736d0 con 0x55e9a35664d0 2024-04-13T05:26:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9a34c8fe0 con 0x55e9a3566ec0 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a35668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.710+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a356e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.711+0000 7f8aaf400640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a9c0a3c30 0x7f8a9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.711+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a356e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46314/0 (socket says 172.21.15.57:46314) 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.711+0000 7f8ab6000640 1 -- 172.21.15.57:0/2140682818 learned_addr learned my addr 172.21.15.57:0/2140682818 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.711+0000 7f8aaf400640 1 -- 172.21.15.57:0/3495901055 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8ab00d0cf0 con 0x7f8ab8078560 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.711+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a356b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.711+0000 7f8ab7400640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a9c0a3c30 0x7f8a9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab6a00640 1 -- 172.21.15.57:0/2140682818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 msgr2=0x55e9a35668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a35668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab6a00640 1 -- 172.21.15.57:0/2140682818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 msgr2=0x55e9a356e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a356e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab6a00640 1 -- 172.21.15.57:0/2140682818 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a32a97d0 con 0x55e9a3566ec0 2024-04-13T05:26:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab7400640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a9c0a3c30 0x7f8a9c0a60f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0099030 tx=0x7f8ab0009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.712+0000 7f8ab6000640 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a356e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:26:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.713+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a356b9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8aa802c170 tx=0x7f8aa8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a1f7326450b146 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.713+0000 7f8aa3e00640 1 -- 172.21.15.57:0/2140682818 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8045020 con 0x55e9a3566ec0 2024-04-13T05:26:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.713+0000 7f8aa3e00640 1 -- 172.21.15.57:0/2140682818 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aa802ca20 con 0x55e9a3566ec0 2024-04-13T05:26:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.713+0000 7f8aa3e00640 1 -- 172.21.15.57:0/2140682818 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8005640 con 0x55e9a3566ec0 2024-04-13T05:26:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/2140682818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 msgr2=0x55e9a356b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a356b9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8aa802c170 tx=0x7f8aa8009560 comp rx=0 tx=0).stop 2024-04-13T05:26:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/2140682818 shutdown_connections 2024-04-13T05:26:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a356e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a356b9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/2140682818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a35668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.714+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/2140682818 >> 172.21.15.57:0/2140682818 conn(0x55e9a3561a40 msgr2=0x55e9a3561e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.715+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/2140682818 shutdown_connections 2024-04-13T05:26:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.715+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/2140682818 wait complete. 2024-04-13T05:26:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.715+0000 7f8ac0b9f740 1 -- start start 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.715+0000 7f8ac0b9f740 1 Processor -- start 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a3570800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a3570d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a3577d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e9a356ef80 con 0x55e9a356c030 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9a3486a90 con 0x55e9a35664d0 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9a356f100 con 0x55e9a3566ec0 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ac0b9f740 10 client.0 _ll_register_callbacks cb 0x55e9a3486480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a3570800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a3570800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:48732/0 (socket says 172.21.15.57:48732) 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.716+0000 7f8ab6000640 1 -- 172.21.15.57:0/216043579 learned_addr learned my addr 172.21.15.57:0/216043579 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6000640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a3577d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a3570d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6a00640 1 -- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 msgr2=0x55e9a3570800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a3570800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6a00640 1 -- 172.21.15.57:0/216043579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 msgr2=0x55e9a3577d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a3577d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.717+0000 7f8ab6a00640 1 -- 172.21.15.57:0/216043579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a31df3f0 con 0x55e9a3566ec0 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.718+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a3570d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e9a35629b0 tx=0x7f8aa8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d1d6b1a7869064 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.718+0000 7f8aa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:26:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.718+0000 7f8aa0c00640 1 -- 172.21.15.57:0/216043579 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8045030 con 0x55e9a3566ec0 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.718+0000 7f8aa0c00640 1 -- 172.21.15.57:0/216043579 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8aa8004450 con 0x55e9a3566ec0 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.718+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa802ca20 con 0x55e9a3566ec0 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8aa0c00640 1 -- 172.21.15.57:0/216043579 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8aa805b050 con 0x55e9a3566ec0 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8ac0b9f740 10 client.26401 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e9a319f0f0 con 0x55e9a3566ec0 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8aa0c00640 1 -- 172.21.15.57:0/216043579 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aa802ca20 con 0x55e9a3566ec0 2024-04-13T05:26:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8aa0c00640 1 -- 172.21.15.57:0/216043579 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f8aa80048f0 con 0x55e9a3566ec0 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8ac0b9f740 10 client.26401 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.719+0000 7f8ac0b9f740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.720+0000 7f8ac0b9f740 20 client.26401 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.720+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a357c2c0 0x55e9a357e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.720+0000 7f8ac0b9f740 4 client.26401 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.720+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a346c600 con 0x55e9a357c2c0 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.720+0000 7f8ab6000640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a357c2c0 0x55e9a357e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.722+0000 7f8ab6000640 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a357c2c0 0x55e9a357e6e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c4b9df15fc8818 server_cookie=f64359d1a671f9cc in_seq=0 out_seq=0 2024-04-13T05:26:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.722+0000 7f8aa0c00640 10 client.26401 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.725+0000 7f8aa0c00640 1 -- 172.21.15.57:0/216043579 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9a346c600 con 0x55e9a357c2c0 2024-04-13T05:26:26.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.725+0000 7f8aa0c00640 10 client.26401 handle_command_reply: tid=0 2024-04-13T05:26:26.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.725+0000 7f8ac0b9f740 1 client.26401 shutdown 2024-04-13T05:26:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.725+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 msgr2=0x55e9a3570d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.725+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a3570d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e9a35629b0 tx=0x7f8aa8005e20 comp rx=0 tx=0).stop 2024-04-13T05:26:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 shutdown_connections 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a357c2c0 0x55e9a357e6e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a356c030 0x55e9a3577d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a3566ec0 0x55e9a3570d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/216043579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a35664d0 0x55e9a3570800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 >> 172.21.15.57:0/216043579 conn(0x55e9a3561a40 msgr2=0x55e9a3560b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8aa0c00640 0 client.26401 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 shutdown_connections 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/216043579 wait complete. 2024-04-13T05:26:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.726+0000 7f8ac0b9f740 20 client.26401 trim_cache size 0 max 16384 2024-04-13T05:26:26.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 Processor -- start 2024-04-13T05:26:26.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 -- start start 2024-04-13T05:26:26.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 0x55e9a3568220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a3566ec0 0x55e9a3568760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a356c030 0x55e9a3577d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81b3aa0 con 0x55e9a35664d0 2024-04-13T05:26:26.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81b12b0 con 0x55e9a3566ec0 2024-04-13T05:26:26.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81af100 con 0x55e9a356c030 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.755+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 0x55e9a3568220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 0x55e9a3568220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46334/0 (socket says 172.21.15.57:46334) 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 -- 172.21.15.57:0/1304064759 learned_addr learned my addr 172.21.15.57:0/1304064759 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a3566ec0 0x55e9a3568760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a356c030 0x55e9a3577d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 -- 172.21.15.57:0/1304064759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a356c030 msgr2=0x55e9a3577d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a356c030 0x55e9a3577d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 -- 172.21.15.57:0/1304064759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a3566ec0 msgr2=0x55e9a3568760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a3566ec0 0x55e9a3568760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6000640 1 -- 172.21.15.57:0/1304064759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a356f230 con 0x55e9a35664d0 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.756+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a3566ec0 0x55e9a3568760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.757+0000 7f8ab6000640 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 0x55e9a3568220 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b790 tx=0x7f8aa400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08722002d799cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.757+0000 7f8aa3400640 1 -- 172.21.15.57:0/1304064759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40043c0 con 0x55e9a35664d0 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.757+0000 7f8aa3400640 1 -- 172.21.15.57:0/1304064759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aa4025e50 con 0x55e9a35664d0 2024-04-13T05:26:26.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.757+0000 7f8aa3400640 1 -- 172.21.15.57:0/1304064759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4005440 con 0x55e9a35664d0 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.758+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/1304064759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 msgr2=0x55e9a3568220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.758+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 0x55e9a3568220 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b790 tx=0x7f8aa400b390 comp rx=0 tx=0).stop 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/1304064759 shutdown_connections 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a3566ec0 0x55e9a3568760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a356c030 0x55e9a3577d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/1304064759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a35664d0 0x55e9a3568220 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/1304064759 >> 172.21.15.57:0/1304064759 conn(0x55e9a3561a40 msgr2=0x55e9a3570800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/1304064759 shutdown_connections 2024-04-13T05:26:26.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/1304064759 wait complete. 2024-04-13T05:26:26.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.759+0000 7f8ac0b9f740 1 -- start start 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 Processor -- start 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a35664d0 0x55e9a3575280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 0x55e9a35757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a356c030 0x55e9a3576710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81af770 con 0x55e9a3566ec0 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81b0be0 con 0x55e9a356c030 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81afe40 con 0x55e9a35664d0 2024-04-13T05:26:26.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ac0b9f740 10 client.0 _ll_register_callbacks cb 0x7f8aa8005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.760+0000 7f8ab6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 0x55e9a35757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.761+0000 7f8ab6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 0x55e9a35757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46350/0 (socket says 172.21.15.57:46350) 2024-04-13T05:26:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.761+0000 7f8ab6a00640 1 -- 172.21.15.57:0/3932751528 learned_addr learned my addr 172.21.15.57:0/3932751528 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.761+0000 7f8ab6000640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a356c030 0x55e9a3576710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.761+0000 7f8ab6000640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a35664d0 0x55e9a3575280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.761+0000 7f8ab6a00640 1 -- 172.21.15.57:0/3932751528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a35664d0 msgr2=0x55e9a3575280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a35664d0 0x55e9a3575280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ab6a00640 1 -- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a356c030 msgr2=0x55e9a3576710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a356c030 0x55e9a3576710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ab6a00640 1 -- 172.21.15.57:0/3932751528 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a3576d00 con 0x55e9a3566ec0 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ab6000640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a35664d0 0x55e9a3575280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ab6a00640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 0x55e9a35757c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8004eb0 tx=0x7f8aa8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a68289e8ee5881d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:26.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8aa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:26.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8aa0c00640 1 -- 172.21.15.57:0/3932751528 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8045070 con 0x55e9a3566ec0 2024-04-13T05:26:26.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa4025e50 con 0x55e9a3566ec0 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.762+0000 7f8aa0c00640 1 -- 172.21.15.57:0/3932751528 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f8aa8004f30 con 0x55e9a3566ec0 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8aa0c00640 1 -- 172.21.15.57:0/3932751528 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8aa8068050 con 0x55e9a3566ec0 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 10 client.17274 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e9a3578040 con 0x55e9a3566ec0 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8aa0c00640 1 -- 172.21.15.57:0/3932751528 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aa4025e50 con 0x55e9a3566ec0 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8aa0c00640 1 -- 172.21.15.57:0/3932751528 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f8aa8003940 con 0x55e9a3566ec0 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 10 client.17274 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 20 client.17274 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a35e2f60 0x55e9a35e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 4 client.17274 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.763+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9a35705f0 con 0x55e9a35e2f60 2024-04-13T05:26:26.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.764+0000 7f8ab6000640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a35e2f60 0x55e9a35e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.765+0000 7f8ab6000640 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a35e2f60 0x55e9a35e5380 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3723ebedd21db8a8 server_cookie=12387f77ad8f5362 in_seq=0 out_seq=0 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.765+0000 7f8aa0c00640 10 client.17274 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.765+0000 7f8aa0c00640 1 -- 172.21.15.57:0/3932751528 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e9a35705f0 con 0x55e9a35e2f60 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.765+0000 7f8aa0c00640 10 client.17274 handle_command_reply: tid=0 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.765+0000 7f8ac0b9f740 1 client.17274 shutdown 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 msgr2=0x55e9a35757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 0x55e9a35757c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8004eb0 tx=0x7f8aa8009520 comp rx=0 tx=0).stop 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 shutdown_connections 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55e9a35e2f60 0x55e9a35e5380 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a356c030 0x55e9a3576710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9a3566ec0 0x55e9a35757c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 --2- 172.21.15.57:0/3932751528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a35664d0 0x55e9a3575280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8aa0c00640 0 client.17274 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 >> 172.21.15.57:0/3932751528 conn(0x55e9a3561a40 msgr2=0x55e9a35707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 shutdown_connections 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 1 -- 172.21.15.57:0/3932751528 wait complete. 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.766+0000 7f8ac0b9f740 20 client.17274 trim_cache size 0 max 16384 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:26:26.770 INFO:teuthology.orchestra.run.smithi057.stdout: "c8a2cc7e-f138-47fd-a7a3-92bc5852c40d": { 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "c8a2cc7e-f138-47fd-a7a3-92bc5852c40d", 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:26:26.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.767+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a9c0a3c30 msgr2=0x7f8a9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.767+0000 7f8abdc00640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a9c0a3c30 0x7f8a9c0a60f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0099030 tx=0x7f8ab0009a70 comp rx=0 tx=0).stop 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.767+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 msgr2=0x7f8ab81a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.767+0000 7f8abdc00640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 0x7f8ab81a5e60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000eb10 tx=0x7f8ab000c2d0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 shutdown_connections 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f8a9c0a3c30 0x7f8a9c0a60f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ab8077bc0 0x7f8ab819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ab8078560 0x7f8ab81a5e60 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 --2- 172.21.15.57:0/3495901055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ab8076980 0x7f8ab819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 >> 172.21.15.57:0/3495901055 conn(0x7f8ab80ff7b0 msgr2=0x7f8ab81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 shutdown_connections 2024-04-13T05:26:26.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:26.768+0000 7f8abdc00640 1 -- 172.21.15.57:0/3495901055 wait complete. 2024-04-13T05:26:26.791 INFO:tasks.cephfs.filesystem:scrub status for tag:c8a2cc7e-f138-47fd-a7a3-92bc5852c40d - {'path': '/', 'tag': 'c8a2cc7e-f138-47fd-a7a3-92bc5852c40d', 'options': 'recursive,force'} 2024-04-13T05:26:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:26 smithi083 ceph-mon[35133]: pgmap v721: 129 pgs: 129 active+clean; 401 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 494 KiB/s wr, 212 op/s 2024-04-13T05:26:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:26 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:26:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:26 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:26 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:26:27.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:26 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:26:27.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:26 smithi057 ceph-mon[32331]: pgmap v721: 129 pgs: 129 active+clean; 401 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 494 KiB/s wr, 212 op/s 2024-04-13T05:26:27.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:26 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:26:27.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:26 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:27.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:26 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:26:27.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:26 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:26:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:26 smithi189 ceph-mon[35531]: pgmap v721: 129 pgs: 129 active+clean; 401 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 494 KiB/s wr, 212 op/s 2024-04-13T05:26:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:26 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:26:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:26 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:26 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:26:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:26 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:26:28.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:27 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:26:28.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:27 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:28.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:27 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:28.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:27 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:26:28.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:27 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:28.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:27 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:28.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:27 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:26:28.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:27 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:28.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:27 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:29.234 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:28 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:29.234 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:28 smithi189 ceph-mon[35531]: pgmap v722: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 341 KiB/s wr, 188 op/s 2024-04-13T05:26:29.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:28 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:29.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:28 smithi083 ceph-mon[35133]: pgmap v722: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 341 KiB/s wr, 188 op/s 2024-04-13T05:26:29.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:28 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:26:29.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:28 smithi057 ceph-mon[32331]: pgmap v722: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 341 KiB/s wr, 188 op/s 2024-04-13T05:26:31.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:30 smithi083 ceph-mon[35133]: pgmap v723: 129 pgs: 129 active+clean; 394 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 453 KiB/s wr, 287 op/s 2024-04-13T05:26:31.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:30 smithi057 ceph-mon[32331]: pgmap v723: 129 pgs: 129 active+clean; 394 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 453 KiB/s wr, 287 op/s 2024-04-13T05:26:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:30 smithi189 ceph-mon[35531]: pgmap v723: 129 pgs: 129 active+clean; 394 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 453 KiB/s wr, 287 op/s 2024-04-13T05:26:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:32 smithi083 ceph-mon[35133]: pgmap v724: 129 pgs: 129 active+clean; 393 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 333 KiB/s wr, 256 op/s 2024-04-13T05:26:33.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:32 smithi057 ceph-mon[32331]: pgmap v724: 129 pgs: 129 active+clean; 393 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 333 KiB/s wr, 256 op/s 2024-04-13T05:26:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:32 smithi189 ceph-mon[35531]: pgmap v724: 129 pgs: 129 active+clean; 393 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 333 KiB/s wr, 256 op/s 2024-04-13T05:26:35.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:34 smithi083 ceph-mon[35133]: pgmap v725: 129 pgs: 129 active+clean; 391 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 400 KiB/s wr, 270 op/s 2024-04-13T05:26:35.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:34 smithi057 ceph-mon[32331]: pgmap v725: 129 pgs: 129 active+clean; 391 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 400 KiB/s wr, 270 op/s 2024-04-13T05:26:35.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:34 smithi189 ceph-mon[35531]: pgmap v725: 129 pgs: 129 active+clean; 391 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 400 KiB/s wr, 270 op/s 2024-04-13T05:26:37.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:36 smithi083 ceph-mon[35133]: pgmap v726: 129 pgs: 129 active+clean; 389 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 379 KiB/s wr, 259 op/s 2024-04-13T05:26:37.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:36 smithi057 ceph-mon[32331]: pgmap v726: 129 pgs: 129 active+clean; 389 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 379 KiB/s wr, 259 op/s 2024-04-13T05:26:37.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:36 smithi189 ceph-mon[35531]: pgmap v726: 129 pgs: 129 active+clean; 389 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 379 KiB/s wr, 259 op/s 2024-04-13T05:26:38.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:37 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:26:38.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:37 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:26:38.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:37 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:26:38.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:37 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:26:38.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:37 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:26:38.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:37 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:26:38.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:37 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:26:38.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:37 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:26:38.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:37 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:26:39.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:38 smithi083 ceph-mon[35133]: pgmap v727: 129 pgs: 129 active+clean; 389 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 245 KiB/s wr, 215 op/s 2024-04-13T05:26:39.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:38 smithi057 ceph-mon[32331]: pgmap v727: 129 pgs: 129 active+clean; 389 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 245 KiB/s wr, 215 op/s 2024-04-13T05:26:39.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:38 smithi189 ceph-mon[35531]: pgmap v727: 129 pgs: 129 active+clean; 389 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 245 KiB/s wr, 215 op/s 2024-04-13T05:26:41.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:40 smithi057 ceph-mon[32331]: pgmap v728: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 299 KiB/s wr, 236 op/s 2024-04-13T05:26:41.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:41 smithi189 ceph-mon[35531]: pgmap v728: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 299 KiB/s wr, 236 op/s 2024-04-13T05:26:41.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:41 smithi083 ceph-mon[35133]: pgmap v728: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 299 KiB/s wr, 236 op/s 2024-04-13T05:26:43.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:43 smithi057 ceph-mon[32331]: pgmap v729: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 187 KiB/s wr, 135 op/s 2024-04-13T05:26:43.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:43 smithi189 ceph-mon[35531]: pgmap v729: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 187 KiB/s wr, 135 op/s 2024-04-13T05:26:43.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:43 smithi083 ceph-mon[35133]: pgmap v729: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 187 KiB/s wr, 135 op/s 2024-04-13T05:26:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:45 smithi057 ceph-mon[32331]: pgmap v730: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 187 KiB/s wr, 105 op/s 2024-04-13T05:26:45.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:45 smithi189 ceph-mon[35531]: pgmap v730: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 187 KiB/s wr, 105 op/s 2024-04-13T05:26:45.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:45 smithi083 ceph-mon[35133]: pgmap v730: 129 pgs: 129 active+clean; 383 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 187 KiB/s wr, 105 op/s 2024-04-13T05:26:47.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:47 smithi057 ceph-mon[32331]: pgmap v731: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 102 KiB/s wr, 78 op/s 2024-04-13T05:26:47.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:47 smithi189 ceph-mon[35531]: pgmap v731: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 102 KiB/s wr, 78 op/s 2024-04-13T05:26:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:47 smithi083 ceph-mon[35133]: pgmap v731: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 102 KiB/s wr, 78 op/s 2024-04-13T05:26:49.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:49 smithi057 ceph-mon[32331]: pgmap v732: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-04-13T05:26:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:49 smithi189 ceph-mon[35531]: pgmap v732: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-04-13T05:26:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:49 smithi083 ceph-mon[35133]: pgmap v732: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-04-13T05:26:51.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:51 smithi189 ceph-mon[35531]: pgmap v733: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 156 KiB/s wr, 64 op/s 2024-04-13T05:26:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:51 smithi083 ceph-mon[35133]: pgmap v733: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 156 KiB/s wr, 64 op/s 2024-04-13T05:26:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:51 smithi057 ceph-mon[32331]: pgmap v733: 129 pgs: 129 active+clean; 382 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 156 KiB/s wr, 64 op/s 2024-04-13T05:26:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:53 smithi189 ceph-mon[35531]: pgmap v734: 129 pgs: 129 active+clean; 381 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 103 KiB/s wr, 42 op/s 2024-04-13T05:26:53.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:53 smithi057 ceph-mon[32331]: pgmap v734: 129 pgs: 129 active+clean; 381 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 103 KiB/s wr, 42 op/s 2024-04-13T05:26:53.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:53 smithi083 ceph-mon[35133]: pgmap v734: 129 pgs: 129 active+clean; 381 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 103 KiB/s wr, 42 op/s 2024-04-13T05:26:55.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:55 smithi189 ceph-mon[35531]: pgmap v735: 129 pgs: 129 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 225 KiB/s wr, 70 op/s 2024-04-13T05:26:55.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:55 smithi083 ceph-mon[35133]: pgmap v735: 129 pgs: 129 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 225 KiB/s wr, 70 op/s 2024-04-13T05:26:55.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:55 smithi057 ceph-mon[32331]: pgmap v735: 129 pgs: 129 active+clean; 378 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 225 KiB/s wr, 70 op/s 2024-04-13T05:26:56.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:26:56.792 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:26:57.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 -- 172.21.15.57:0/4198129607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70102950 msgr2=0x7f0d70104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/4198129607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70102950 0x7f0d70104d40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009900 tx=0x7f0d6002bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 -- 172.21.15.57:0/4198129607 shutdown_connections 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/4198129607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70107bb0 0x7f0d7010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/4198129607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d70105280 0x7f0d70107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/4198129607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70102950 0x7f0d70104d40 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.033+0000 7f0d6fe00640 1 -- 172.21.15.57:0/4198129607 >> 172.21.15.57:0/4198129607 conn(0x7f0d700fc360 msgr2=0x7f0d700fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.034+0000 7f0d6fe00640 1 -- 172.21.15.57:0/4198129607 shutdown_connections 2024-04-13T05:26:57.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.034+0000 7f0d6fe00640 1 -- 172.21.15.57:0/4198129607 wait complete. 2024-04-13T05:26:57.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 Processor -- start 2024-04-13T05:26:57.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 -- start start 2024-04-13T05:26:57.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d70102950 0x7f0d701022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70105280 0x7f0d70100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 0x7f0d70100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010da40 con 0x7f0d70107bb0 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010d8c0 con 0x7f0d70105280 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010dbc0 con 0x7f0d70102950 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.035+0000 7f0d6f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 0x7f0d70100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 0x7f0d70100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45714/0 (socket says 172.21.15.57:45714) 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 -- 172.21.15.57:0/1578977530 learned_addr learned my addr 172.21.15.57:0/1578977530 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d66000640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70105280 0x7f0d70100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d70102950 0x7f0d701022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 -- 172.21.15.57:0/1578977530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d70102950 msgr2=0x7f0d701022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d70102950 0x7f0d701022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 -- 172.21.15.57:0/1578977530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70105280 msgr2=0x7f0d70100900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70105280 0x7f0d70100900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 -- 172.21.15.57:0/1578977530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701016d0 con 0x7f0d70107bb0 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d66000640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70105280 0x7f0d70100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.036+0000 7f0d6f400640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 0x7f0d70100e40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800cce0 tx=0x7f0d6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.037+0000 7f0d74a00640 1 -- 172.21.15.57:0/1578977530 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800f450 con 0x7f0d70107bb0 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.037+0000 7f0d74a00640 1 -- 172.21.15.57:0/1578977530 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d6800faa0 con 0x7f0d70107bb0 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.037+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701a20f0 con 0x7f0d70107bb0 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.037+0000 7f0d74a00640 1 -- 172.21.15.57:0/1578977530 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68018b70 con 0x7f0d70107bb0 2024-04-13T05:26:57.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.037+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a25f0 con 0x7f0d70107bb0 2024-04-13T05:26:57.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 Processor -- start 2024-04-13T05:26:57.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d74a00640 1 -- 172.21.15.57:0/1578977530 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0d68018d10 con 0x7f0d70107bb0 2024-04-13T05:26:57.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 -- start start 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3f1250 0x5580ad3f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5580ad352020 con 0x5580ad3eb6f0 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5580ad2f86d0 con 0x5580ad3f1250 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d78de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580ad34dfe0 con 0x5580ad3ec0e0 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.041+0000 7f0d66000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ebb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45720/0 (socket says 172.21.15.57:45720) 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 -- 172.21.15.57:0/3376214647 learned_addr learned my addr 172.21.15.57:0/3376214647 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 -- 172.21.15.57:0/3376214647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 msgr2=0x5580ad3f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d74a00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0d500a3bb0 0x7f0d500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 -- 172.21.15.57:0/3376214647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3f1250 msgr2=0x5580ad3f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3f1250 0x5580ad3f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 -- 172.21.15.57:0/3376214647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ad12e7d0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d74a00640 1 -- 172.21.15.57:0/1578977530 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0d680d0d60 con 0x7f0d70107bb0 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d6f400640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0d500a3bb0 0x7f0d500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.042+0000 7f0d66000640 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ebb10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800bc60 tx=0x7f0d5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e92ee8206861a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.043+0000 7f0d5fe00640 1 -- 172.21.15.57:0/3376214647 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58005220 con 0x5580ad3eb6f0 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.043+0000 7f0d5fe00640 1 -- 172.21.15.57:0/3376214647 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d58004c50 con 0x5580ad3eb6f0 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.043+0000 7f0d5fe00640 1 -- 172.21.15.57:0/3376214647 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58011760 con 0x5580ad3eb6f0 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 -- 172.21.15.57:0/3376214647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 msgr2=0x5580ad3ebb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d6f400640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0d500a3bb0 0x7f0d500a6070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0d6809c940 tx=0x7f0d6800af30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ebb10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800bc60 tx=0x7f0d5800f730 comp rx=0 tx=0).stop 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 -- 172.21.15.57:0/3376214647 shutdown_connections 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3f1250 0x5580ad3f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3376214647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ebb10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 -- 172.21.15.57:0/3376214647 >> 172.21.15.57:0/3376214647 conn(0x5580ad3e6c60 msgr2=0x5580ad3e7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 -- 172.21.15.57:0/3376214647 shutdown_connections 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 -- 172.21.15.57:0/3376214647 wait complete. 2024-04-13T05:26:57.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.044+0000 7f0d78de2740 1 -- start start 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 Processor -- start 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3eb6f0 0x5580ad3f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 0x5580ad3fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5580ad3f4200 con 0x5580ad3f1250 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5580ad30ba90 con 0x5580ad3ec0e0 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580ad3f4380 con 0x5580ad3eb6f0 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d78de2740 10 client.0 _ll_register_callbacks cb 0x5580ad30b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.045+0000 7f0d66000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 0x5580ad3fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 0x5580ad3fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45736/0 (socket says 172.21.15.57:45736) 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 -- 172.21.15.57:0/1363254250 learned_addr learned my addr 172.21.15.57:0/1363254250 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3eb6f0 0x5580ad3f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 -- 172.21.15.57:0/1363254250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3eb6f0 msgr2=0x5580ad3f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3eb6f0 0x5580ad3f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 -- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 msgr2=0x5580ad3f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 -- 172.21.15.57:0/1363254250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ad0643f0 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d66000640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 0x5580ad3fcf20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004ab0 tx=0x7f0d5800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a9aba641e39084 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.046+0000 7f0d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1363254250 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5801d020 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d58004c50 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1363254250 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0d580052b0 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1363254250 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d5802e050 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d78de2740 10 client.17286 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580ad0240f0 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.047+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1363254250 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d58004c50 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1363254250 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f0d58019840 con 0x5580ad3f1250 2024-04-13T05:26:57.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d78de2740 10 client.17286 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:57.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d78de2740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:57.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d78de2740 20 client.17286 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:57.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad401650 0x5580ad403a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d78de2740 4 client.17286 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:57.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580ad2f1600 con 0x5580ad401650 2024-04-13T05:26:57.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.048+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad401650 0x5580ad403a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.049+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad401650 0x5580ad403a80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68da8c2e251abe0 server_cookie=141531b2a68f0b59 in_seq=0 out_seq=0 2024-04-13T05:26:57.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.049+0000 7f0d5cc00640 10 client.17286 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:57.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1363254250 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580ad2f1600 con 0x5580ad401650 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d5cc00640 10 client.17286 handle_command_reply: tid=0 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 client.17286 shutdown 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 msgr2=0x5580ad3fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 0x5580ad3fcf20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004ab0 tx=0x7f0d5800bf60 comp rx=0 tx=0).stop 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 shutdown_connections 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad401650 0x5580ad403a80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3f1250 0x5580ad3fcf20 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1363254250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3eb6f0 0x5580ad3f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d5cc00640 0 client.17286 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 >> 172.21.15.57:0/1363254250 conn(0x5580ad3e6c60 msgr2=0x5580ad3e5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 shutdown_connections 2024-04-13T05:26:57.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.052+0000 7f0d78de2740 1 -- 172.21.15.57:0/1363254250 wait complete. 2024-04-13T05:26:57.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.053+0000 7f0d78de2740 20 client.17286 trim_cache size 0 max 16384 2024-04-13T05:26:57.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.079+0000 7f0d78de2740 1 Processor -- start 2024-04-13T05:26:57.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.079+0000 7f0d78de2740 1 -- start start 2024-04-13T05:26:57.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3e6c60 0x5580ad3ed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d78de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701af3d0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d78de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701acbe0 con 0x5580ad3ec0e0 2024-04-13T05:26:57.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d78de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701aaa30 con 0x5580ad3e6c60 2024-04-13T05:26:57.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ed8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45748/0 (socket says 172.21.15.57:45748) 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d6ea00640 1 -- 172.21.15.57:0/3303221586 learned_addr learned my addr 172.21.15.57:0/3303221586 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.080+0000 7f0d66000640 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3e6c60 0x5580ad3ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.081+0000 7f0d66000640 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.081+0000 7f0d6ea00640 1 -- 172.21.15.57:0/3303221586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3e6c60 msgr2=0x5580ad3ed360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.081+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3e6c60 0x5580ad3ed360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.081+0000 7f0d6ea00640 1 -- 172.21.15.57:0/3303221586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 msgr2=0x5580ad3fcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.081+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.081+0000 7f0d6ea00640 1 -- 172.21.15.57:0/3303221586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ad3f40e0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.082+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ed8a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009230 tx=0x7f0d60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728c5dc7f092318e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.082+0000 7f0d5f400640 1 -- 172.21.15.57:0/3303221586 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60047070 con 0x5580ad3eb6f0 2024-04-13T05:26:57.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.082+0000 7f0d5f400640 1 -- 172.21.15.57:0/3303221586 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d60049dd0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.082+0000 7f0d5f400640 1 -- 172.21.15.57:0/3303221586 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60049310 con 0x5580ad3eb6f0 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 -- 172.21.15.57:0/3303221586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 msgr2=0x5580ad3ed8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ed8a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009230 tx=0x7f0d60000bc0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 -- 172.21.15.57:0/3303221586 shutdown_connections 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3ed8a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 --2- 172.21.15.57:0/3303221586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3e6c60 0x5580ad3ed360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 -- 172.21.15.57:0/3303221586 >> 172.21.15.57:0/3303221586 conn(0x5580ad3e73b0 msgr2=0x5580ad3f4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 -- 172.21.15.57:0/3303221586 shutdown_connections 2024-04-13T05:26:57.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.083+0000 7f0d78de2740 1 -- 172.21.15.57:0/3303221586 wait complete. 2024-04-13T05:26:57.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 -- start start 2024-04-13T05:26:57.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 Processor -- start 2024-04-13T05:26:57.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3e6c60 0x5580ad3fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ab0a0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ac510 con 0x5580ad3e6c60 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ab770 con 0x5580ad3ec0e0 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.084+0000 7f0d78de2740 10 client.0 _ll_register_callbacks cb 0x7f0d580046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3fa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45750/0 (socket says 172.21.15.57:45750) 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 -- 172.21.15.57:0/1463367318 learned_addr learned my addr 172.21.15.57:0/1463367318 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d66000640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3e6c60 0x5580ad3fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d66000640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 -- 172.21.15.57:0/1463367318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 msgr2=0x5580ad3fb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 -- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3e6c60 msgr2=0x5580ad3fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3e6c60 0x5580ad3fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 -- 172.21.15.57:0/1463367318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ad3fbcc0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d66000640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3e6c60 0x5580ad3fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d6ea00640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3fa6c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f0d60036040 tx=0x7f0d60049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9178f47bf214809b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.085+0000 7f0d5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1463367318 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60057020 con 0x5580ad3eb6f0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1463367318 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0d60004040 con 0x5580ad3eb6f0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d60049dd0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1463367318 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d60004240 con 0x5580ad3eb6f0 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d78de2740 10 client.17292 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:57.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580ad3fd000 con 0x5580ad3eb6f0 2024-04-13T05:26:57.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.086+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1463367318 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d60049dd0 con 0x5580ad3eb6f0 2024-04-13T05:26:57.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1463367318 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f0d60052070 con 0x5580ad3eb6f0 2024-04-13T05:26:57.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d78de2740 10 client.17292 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:57.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d78de2740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:57.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d78de2740 20 client.17292 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:57.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad3f1250 0x5580ad46a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d78de2740 4 client.17292 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:26:57.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580ad35fa40 con 0x5580ad3f1250 2024-04-13T05:26:57.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.087+0000 7f0d66000640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad3f1250 0x5580ad46a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.088+0000 7f0d66000640 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad3f1250 0x5580ad46a1c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340555394427c7c8 server_cookie=33404133b1f0eb41 in_seq=0 out_seq=0 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.088+0000 7f0d5cc00640 10 client.17292 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d5cc00640 1 -- 172.21.15.57:0/1463367318 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5580ad35fa40 con 0x5580ad3f1250 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d5cc00640 10 client.17292 handle_command_reply: tid=0 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 client.17292 shutdown 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 msgr2=0x5580ad3fa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3fa6c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f0d60036040 tx=0x7f0d60049bd0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 shutdown_connections 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5580ad3f1250 0x5580ad46a1c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580ad3ec0e0 0x5580ad3fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580ad3eb6f0 0x5580ad3fa6c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 --2- 172.21.15.57:0/1463367318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580ad3e6c60 0x5580ad3fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 >> 172.21.15.57:0/1463367318 conn(0x5580ad3e73b0 msgr2=0x5580ad3f5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d5cc00640 0 client.17292 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 shutdown_connections 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 1 -- 172.21.15.57:0/1463367318 wait complete. 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.089+0000 7f0d78de2740 20 client.17292 trim_cache size 0 max 16384 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:26:57.095 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0d500a3bb0 msgr2=0x7f0d500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0d500a3bb0 0x7f0d500a6070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0d6809c940 tx=0x7f0d6800af30 comp rx=0 tx=0).stop 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 msgr2=0x7f0d70100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 0x7f0d70100e40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800cce0 tx=0x7f0d6800a620 comp rx=0 tx=0).stop 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 shutdown_connections 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0d500a3bb0 0x7f0d500a6070 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d70105280 0x7f0d70100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d70107bb0 0x7f0d70100e40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 --2- 172.21.15.57:0/1578977530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d70102950 0x7f0d701022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 >> 172.21.15.57:0/1578977530 conn(0x7f0d700fc360 msgr2=0x7f0d70103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.090+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 shutdown_connections 2024-04-13T05:26:57.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.091+0000 7f0d6fe00640 1 -- 172.21.15.57:0/1578977530 wait complete. 2024-04-13T05:26:57.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:26:57.114 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:26:57.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:57 smithi057 ceph-mon[32331]: pgmap v736: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 205 KiB/s wr, 92 op/s 2024-04-13T05:26:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:57 smithi189 ceph-mon[35531]: pgmap v736: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 205 KiB/s wr, 92 op/s 2024-04-13T05:26:57.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 -- 172.21.15.57:0/259083357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 msgr2=0x7f4a10104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 --2- 172.21.15.57:0/259083357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a10104640 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009900 tx=0x7f4a0402bc30 comp rx=0 tx=0).stop 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 -- 172.21.15.57:0/259083357 shutdown_connections 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 --2- 172.21.15.57:0/259083357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10105410 0x7f4a10105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 --2- 172.21.15.57:0/259083357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a10105dd0 0x7f4a1010c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 --2- 172.21.15.57:0/259083357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a10104640 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 -- 172.21.15.57:0/259083357 >> 172.21.15.57:0/259083357 conn(0x7f4a100ff7b0 msgr2=0x7f4a10101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 -- 172.21.15.57:0/259083357 shutdown_connections 2024-04-13T05:26:57.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.378+0000 7f4a18000640 1 -- 172.21.15.57:0/259083357 wait complete. 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 Processor -- start 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 -- start start 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a1019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10105410 0x7f4a1019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a10105dd0 0x7f4a101a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010fdb0 con 0x7f4a10104220 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010fc30 con 0x7f4a10105410 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.379+0000 7f4a18000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010ff30 con 0x7f4a10105dd0 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a1019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a1019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45764/0 (socket says 172.21.15.57:45764) 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 -- 172.21.15.57:0/4096805852 learned_addr learned my addr 172.21.15.57:0/4096805852 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a16000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a10105dd0 0x7f4a101a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a14c00640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10105410 0x7f4a1019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 -- 172.21.15.57:0/4096805852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a10105dd0 msgr2=0x7f4a101a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a10105dd0 0x7f4a101a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 -- 172.21.15.57:0/4096805852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10105410 msgr2=0x7f4a1019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10105410 0x7f4a1019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.380+0000 7f4a15600640 1 -- 172.21.15.57:0/4096805852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a101a20b0 con 0x7f4a10104220 2024-04-13T05:26:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.381+0000 7f4a15600640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a1019a490 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000c00 tx=0x7f4a040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.381+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04043020 con 0x7f4a10104220 2024-04-13T05:26:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.381+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a101a2340 con 0x7f4a10104220 2024-04-13T05:26:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.381+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a040044c0 con 0x7f4a10104220 2024-04-13T05:26:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.381+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04003c90 con 0x7f4a10104220 2024-04-13T05:26:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.381+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a101a28e0 con 0x7f4a10104220 2024-04-13T05:26:57.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.382+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1010b610 con 0x7f4a10104220 2024-04-13T05:26:57.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.386+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4a04046020 con 0x7f4a10104220 2024-04-13T05:26:57.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.387+0000 7f4a0e000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49f80a3c30 0x7f49f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.387+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4a040f2c70 con 0x7f4a10104220 2024-04-13T05:26:57.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.388+0000 7f4a14c00640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49f80a3c30 0x7f49f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.388+0000 7f4a14c00640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49f80a3c30 0x7f49f80a60f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0095f0 tx=0x7f49fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.390+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a0404d020 con 0x7f4a10104220 2024-04-13T05:26:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:57 smithi083 ceph-mon[35133]: pgmap v736: 129 pgs: 129 active+clean; 370 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 205 KiB/s wr, 92 op/s 2024-04-13T05:26:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.653+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a1002d5e0 con 0x7f4a10104220 2024-04-13T05:26:57.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.655+0000 7f4a0e000640 1 -- 172.21.15.57:0/4096805852 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11864 (secure 0 0 0) 0x7f4a040bf250 con 0x7f4a10104220 2024-04-13T05:26:57.658 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:26:57.658 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:53.740918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:26:57.658 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-04-13T05:26:57.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.657+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49f80a3c30 msgr2=0x7f49f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.657+0000 7f4a18000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49f80a3c30 0x7f49f80a60f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0095f0 tx=0x7f49fc005f90 comp rx=0 tx=0).stop 2024-04-13T05:26:57.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.657+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 msgr2=0x7f4a1019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.657+0000 7f4a18000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a1019a490 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000c00 tx=0x7f4a040040c0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 shutdown_connections 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49f80a3c30 0x7f49f80a60f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10105410 0x7f4a1019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a10105dd0 0x7f4a101a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 --2- 172.21.15.57:0/4096805852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a10104220 0x7f4a1019a490 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 >> 172.21.15.57:0/4096805852 conn(0x7f4a100ff7b0 msgr2=0x7f4a101087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 shutdown_connections 2024-04-13T05:26:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.658+0000 7f4a18000640 1 -- 172.21.15.57:0/4096805852 wait complete. 2024-04-13T05:26:57.682 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:26:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 -- 172.21.15.57:0/1095794944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 msgr2=0x7f874410c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 --2- 172.21.15.57:0/1095794944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f874410c2a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f873c009900 tx=0x7f873c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:26:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 -- 172.21.15.57:0/1095794944 shutdown_connections 2024-04-13T05:26:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 --2- 172.21.15.57:0/1095794944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f874410c2a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 --2- 172.21.15.57:0/1095794944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8744074a20 0x7f8744074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 --2- 172.21.15.57:0/1095794944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8744076000 0x7f8744074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 -- 172.21.15.57:0/1095794944 >> 172.21.15.57:0/1095794944 conn(0x7f87440fd4c0 msgr2=0x7f87440ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 -- 172.21.15.57:0/1095794944 shutdown_connections 2024-04-13T05:26:57.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.906+0000 7f8748e00640 1 -- 172.21.15.57:0/1095794944 wait complete. 2024-04-13T05:26:57.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.907+0000 7f8748e00640 1 Processor -- start 2024-04-13T05:26:57.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.907+0000 7f8748e00640 1 -- start start 2024-04-13T05:26:57.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8748e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8744074a20 0x7f874419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8748e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8744076000 0x7f874419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8748e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f87441a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8748e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f874410fc60 con 0x7f8744109e10 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8748e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f874410fae0 con 0x7f8744074a20 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8748e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f874410fde0 con 0x7f8744076000 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8743e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f87441a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8743e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f87441a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45792/0 (socket says 172.21.15.57:45792) 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.908+0000 7f8743e00640 1 -- 172.21.15.57:0/401958390 learned_addr learned my addr 172.21.15.57:0/401958390 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743400640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8744074a20 0x7f874419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f873aa00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8744076000 0x7f874419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743e00640 1 -- 172.21.15.57:0/401958390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8744076000 msgr2=0x7f874419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8744076000 0x7f874419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743e00640 1 -- 172.21.15.57:0/401958390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8744074a20 msgr2=0x7f874419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8744074a20 0x7f874419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743e00640 1 -- 172.21.15.57:0/401958390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87441a1e20 con 0x7f8744109e10 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8743e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f87441a17b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f873c036040 tx=0x7f873c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8740c00640 1 -- 172.21.15.57:0/401958390 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c043020 con 0x7f8744109e10 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87441a20b0 con 0x7f8744109e10 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8740c00640 1 -- 172.21.15.57:0/401958390 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f873c004490 con 0x7f8744109e10 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.909+0000 7f8740c00640 1 -- 172.21.15.57:0/401958390 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c003d10 con 0x7f8744109e10 2024-04-13T05:26:57.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.910+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441a2600 con 0x7f8744109e10 2024-04-13T05:26:57.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.913+0000 7f874d431740 1 Processor -- start 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.913+0000 7f8740c00640 1 -- 172.21.15.57:0/401958390 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f873c046020 con 0x7f8744109e10 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.913+0000 7f874d431740 1 -- start start 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f874d431740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fa7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f874d431740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7e30 0x55e519fac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f874d431740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519faf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f874d431740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e519f0d020 con 0x55e519facfa0 2024-04-13T05:26:57.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f874d431740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e519eb3950 con 0x55e519fa7e30 2024-04-13T05:26:57.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f874d431740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e519f08fe0 con 0x55e519fa7440 2024-04-13T05:26:57.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f873aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519faf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f873aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519faf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45794/0 (socket says 172.21.15.57:45794) 2024-04-13T05:26:57.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f873aa00640 1 -- 172.21.15.57:0/704768225 learned_addr learned my addr 172.21.15.57:0/704768225 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f8743400640 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7e30 0x55e519fac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.914+0000 7f873aa00640 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fa7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f873aa00640 1 -- 172.21.15.57:0/704768225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 msgr2=0x55e519fa7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f873aa00640 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fa7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f873aa00640 1 -- 172.21.15.57:0/704768225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7e30 msgr2=0x55e519fac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f873aa00640 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7e30 0x55e519fac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f873aa00640 1 -- 172.21.15.57:0/704768225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e519c1f3f0 con 0x55e519facfa0 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f8740c00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87280a3bb0 0x7f87280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f873aa00640 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519faf430 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f872c00bf70 tx=0x7f872c00b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af757c9dd6d224c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f8740c00640 1 -- 172.21.15.57:0/401958390 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f873c0f2c80 con 0x7f8744109e10 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f8743400640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87280a3bb0 0x7f87280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f8738c00640 1 -- 172.21.15.57:0/704768225 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c018030 con 0x55e519facfa0 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f8738c00640 1 -- 172.21.15.57:0/704768225 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c010bd0 con 0x55e519facfa0 2024-04-13T05:26:57.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.915+0000 7f8738c00640 1 -- 172.21.15.57:0/704768225 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c00eb70 con 0x55e519facfa0 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.916+0000 7f874d431740 1 -- 172.21.15.57:0/704768225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 msgr2=0x55e519faf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.916+0000 7f8743400640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87280a3bb0 0x7f87280a6070 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55e519fa8230 tx=0x7f8734009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.916+0000 7f874d431740 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519faf430 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f872c00bf70 tx=0x7f872c00b640 comp rx=0 tx=0).stop 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 -- 172.21.15.57:0/704768225 shutdown_connections 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7e30 0x55e519fac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519faf430 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 --2- 172.21.15.57:0/704768225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fa7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 -- 172.21.15.57:0/704768225 >> 172.21.15.57:0/704768225 conn(0x55e519fa29b0 msgr2=0x55e519fa2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 -- 172.21.15.57:0/704768225 shutdown_connections 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 -- 172.21.15.57:0/704768225 wait complete. 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 -- start start 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.917+0000 7f874d431740 1 Processor -- start 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 0x55e519fb19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e519faff50 con 0x55e519fa7440 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e519ec6c80 con 0x55e519facfa0 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e519fb00d0 con 0x55e519fa7e30 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f874d431740 10 client.0 _ll_register_callbacks cb 0x55e519ec6670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f8743400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 0x55e519fb19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f8743400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 0x55e519fb19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45800/0 (socket says 172.21.15.57:45800) 2024-04-13T05:26:57.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f8743400640 1 -- 172.21.15.57:0/394264299 learned_addr learned my addr 172.21.15.57:0/394264299 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f8743e00640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.918+0000 7f873aa00640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743400640 1 -- 172.21.15.57:0/394264299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 msgr2=0x55e519fb1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743400640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743400640 1 -- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 msgr2=0x55e519fb8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743400640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743400640 1 -- 172.21.15.57:0/394264299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e519ce97d0 con 0x55e519fa7440 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743e00640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8743400640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 0x55e519fb19f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8734009760 tx=0x7f8734004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58c0f9e267e2bad server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8731600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8731600640 1 -- 172.21.15.57:0/394264299 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873400ec60 con 0x55e519fa7440 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f8731600640 1 -- 172.21.15.57:0/394264299 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f873400ee00 con 0x55e519fa7440 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.919+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872c010bd0 con 0x55e519fa7440 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f8731600640 1 -- 172.21.15.57:0/394264299 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f873401f420 con 0x55e519fa7440 2024-04-13T05:26:57.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 10 client.17310 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:57.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e519fbc3b0 con 0x55e519fa7440 2024-04-13T05:26:57.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f8731600640 1 -- 172.21.15.57:0/394264299 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c010bd0 con 0x55e519fa7440 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f8731600640 1 -- 172.21.15.57:0/394264299 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f873400f040 con 0x55e519fa7440 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 10 client.17310 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 10 client.17310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 20 client.17310 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e519fbdb60 0x55e519fc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 4 client.17310 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.920+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e519eac750 con 0x55e519fbdb60 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.921+0000 7f873aa00640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e519fbdb60 0x55e519fc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.922+0000 7f873aa00640 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e519fbdb60 0x55e519fc0020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=923686372a73526b server_cookie=4e4c057508c5c9bb in_seq=0 out_seq=0 2024-04-13T05:26:57.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.922+0000 7f8731600640 10 client.17310 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f8731600640 1 -- 172.21.15.57:0/394264299 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e519eac750 con 0x55e519fbdb60 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f8731600640 10 client.17310 handle_command_reply: tid=0 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 client.17310 shutdown 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 msgr2=0x55e519fb19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 0x55e519fb19f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8734009760 tx=0x7f8734004a40 comp rx=0 tx=0).stop 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 shutdown_connections 2024-04-13T05:26:57.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e519fbdb60 0x55e519fc0020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 --2- 172.21.15.57:0/394264299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7440 0x55e519fb19f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f8731600640 0 client.17310 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.925+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 >> 172.21.15.57:0/394264299 conn(0x55e519fa29b0 msgr2=0x55e519fa1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.926+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 shutdown_connections 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.926+0000 7f874d431740 1 -- 172.21.15.57:0/394264299 wait complete. 2024-04-13T05:26:57.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.926+0000 7f874d431740 20 client.17310 trim_cache size 0 max 16384 2024-04-13T05:26:57.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 Processor -- start 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 -- start start 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 0x55e519fb8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87441af490 con 0x55e519fa7e30 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87441acca0 con 0x55e519facfa0 2024-04-13T05:26:57.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.942+0000 7f874d431740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87441aaaf0 con 0x55e519fa7440 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 0x55e519fb8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 0x55e519fb8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45814/0 (socket says 172.21.15.57:45814) 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 -- 172.21.15.57:0/2501138292 learned_addr learned my addr 172.21.15.57:0/2501138292 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f8743400640 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f8743e00640 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 -- 172.21.15.57:0/2501138292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 msgr2=0x55e519fb1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fb1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 -- 172.21.15.57:0/2501138292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 msgr2=0x55e519fb1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.943+0000 7f873aa00640 1 -- 172.21.15.57:0/2501138292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e519fbc1a0 con 0x55e519fa7e30 2024-04-13T05:26:57.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.944+0000 7f873aa00640 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 0x55e519fb8f30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f872c012e90 tx=0x7f872c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14497087f1d075c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.944+0000 7f8733e00640 1 -- 172.21.15.57:0/2501138292 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c018040 con 0x55e519fa7e30 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.944+0000 7f8733e00640 1 -- 172.21.15.57:0/2501138292 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c010bd0 con 0x55e519fa7e30 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.944+0000 7f8733e00640 1 -- 172.21.15.57:0/2501138292 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c0052a0 con 0x55e519fa7e30 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 -- 172.21.15.57:0/2501138292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 msgr2=0x55e519fb8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 0x55e519fb8f30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f872c012e90 tx=0x7f872c005850 comp rx=0 tx=0).stop 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 -- 172.21.15.57:0/2501138292 shutdown_connections 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519facfa0 0x55e519fb1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519fa7e30 0x55e519fb8f30 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 --2- 172.21.15.57:0/2501138292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7440 0x55e519fb1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 -- 172.21.15.57:0/2501138292 >> 172.21.15.57:0/2501138292 conn(0x55e519fa29b0 msgr2=0x55e519fa2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 -- 172.21.15.57:0/2501138292 shutdown_connections 2024-04-13T05:26:57.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 -- 172.21.15.57:0/2501138292 wait complete. 2024-04-13T05:26:57.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.945+0000 7f874d431740 1 -- start start 2024-04-13T05:26:57.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 Processor -- start 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7440 0x55e519fb6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519fb7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87441ab160 con 0x55e519facfa0 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87441ac5d0 con 0x55e519fa7440 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87441ab830 con 0x55e519fa7e30 2024-04-13T05:26:57.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f874d431740 10 client.0 _ll_register_callbacks cb 0x7f87340125f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f8743e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519fb7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f8743e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519fb7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45826/0 (socket says 172.21.15.57:45826) 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.946+0000 7f8743e00640 1 -- 172.21.15.57:0/3283512284 learned_addr learned my addr 172.21.15.57:0/3283512284 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743400640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7440 0x55e519fb6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f873aa00640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743e00640 1 -- 172.21.15.57:0/3283512284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 msgr2=0x55e519fb6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743e00640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743e00640 1 -- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7440 msgr2=0x55e519fb6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743e00640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7440 0x55e519fb6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743e00640 1 -- 172.21.15.57:0/3283512284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e519fb7cf0 con 0x55e519facfa0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f873aa00640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8743e00640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519fb7700 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f873c0be860 tx=0x7f873c0be6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44b8c00c67ec803 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8731600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8731600640 1 -- 172.21.15.57:0/3283512284 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0bf260 con 0x55e519facfa0 2024-04-13T05:26:57.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f8731600640 1 -- 172.21.15.57:0/3283512284 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f873c0df0b0 con 0x55e519facfa0 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.947+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872c010bd0 con 0x55e519facfa0 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f8731600640 1 -- 172.21.15.57:0/3283512284 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f873c0beb60 con 0x55e519facfa0 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 10 client.17316 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e519fb9060 con 0x55e519facfa0 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f8731600640 1 -- 172.21.15.57:0/3283512284 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c010bd0 con 0x55e519facfa0 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f8731600640 1 -- 172.21.15.57:0/3283512284 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f873c0c7340 con 0x55e519facfa0 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 10 client.17316 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 10 client.17316 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 20 client.17316 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e51a024800 0x55e51a026c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 4 client.17316 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.948+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e519f1aa40 con 0x55e51a024800 2024-04-13T05:26:57.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.949+0000 7f8743400640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e51a024800 0x55e51a026c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.950+0000 7f8743400640 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e51a024800 0x55e51a026c20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e773df8532f7ad4 server_cookie=ac00a3aa8bf24c68 in_seq=0 out_seq=0 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.950+0000 7f8731600640 10 client.17316 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.950+0000 7f8731600640 1 -- 172.21.15.57:0/3283512284 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e519f1aa40 con 0x55e51a024800 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.950+0000 7f8731600640 10 client.17316 handle_command_reply: tid=0 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.950+0000 7f874d431740 1 client.17316 shutdown 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 msgr2=0x55e519fb7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519fb7700 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f873c0be860 tx=0x7f873c0be6a0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 shutdown_connections 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55e51a024800 0x55e51a026c20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e519facfa0 0x55e519fb7700 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e519fa7e30 0x55e519fb6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f8731600640 0 client.17316 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:26:57.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 --2- 172.21.15.57:0/3283512284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e519fa7440 0x55e519fb6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 >> 172.21.15.57:0/3283512284 conn(0x55e519fa29b0 msgr2=0x55e519fa5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 shutdown_connections 2024-04-13T05:26:57.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 1 -- 172.21.15.57:0/3283512284 wait complete. 2024-04-13T05:26:57.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.951+0000 7f874d431740 20 client.17316 trim_cache size 0 max 16384 2024-04-13T05:26:57.956 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87280a3bb0 msgr2=0x7f87280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87280a3bb0 0x7f87280a6070 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55e519fa8230 tx=0x7f8734009580 comp rx=0 tx=0).stop 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 msgr2=0x7f87441a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f87441a17b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f873c036040 tx=0x7f873c004270 comp rx=0 tx=0).stop 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 shutdown_connections 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f87280a3bb0 0x7f87280a6070 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8744109e10 0x7f87441a17b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8744076000 0x7f874419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 --2- 172.21.15.57:0/401958390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8744074a20 0x7f874419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 >> 172.21.15.57:0/401958390 conn(0x7f87440fd4c0 msgr2=0x7f8744105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 shutdown_connections 2024-04-13T05:26:57.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:57.952+0000 7f8748e00640 1 -- 172.21.15.57:0/401958390 wait complete. 2024-04-13T05:26:57.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 -- 172.21.15.57:0/1576077448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28105280 msgr2=0x7fae28107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 --2- 172.21.15.57:0/1576077448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28105280 0x7fae28107670 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fae14009900 tx=0x7fae1402bc30 comp rx=0 tx=0).stop 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 -- 172.21.15.57:0/1576077448 shutdown_connections 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 --2- 172.21.15.57:0/1576077448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae28107bb0 0x7fae2810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 --2- 172.21.15.57:0/1576077448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28105280 0x7fae28107670 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 --2- 172.21.15.57:0/1576077448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae28102950 0x7fae28104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 -- 172.21.15.57:0/1576077448 >> 172.21.15.57:0/1576077448 conn(0x7fae280fc360 msgr2=0x7fae280fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 -- 172.21.15.57:0/1576077448 shutdown_connections 2024-04-13T05:26:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.200+0000 7fae2d000640 1 -- 172.21.15.57:0/1576077448 wait complete. 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.201+0000 7fae2d000640 1 Processor -- start 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.201+0000 7fae2d000640 1 -- start start 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae2d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 0x7fae281022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae2d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae28105280 0x7fae28100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae2d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae28107bb0 0x7fae28100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae2d000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810da00 con 0x7fae28102950 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae2d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810d880 con 0x7fae28105280 2024-04-13T05:26:58.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae2d000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810db80 con 0x7fae28107bb0 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae27400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 0x7fae281022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae27400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 0x7fae281022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45834/0 (socket says 172.21.15.57:45834) 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae27400640 1 -- 172.21.15.57:0/3508021405 learned_addr learned my addr 172.21.15.57:0/3508021405 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae26a00640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae28105280 0x7fae28100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.202+0000 7fae27e00640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae28107bb0 0x7fae28100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae27400640 1 -- 172.21.15.57:0/3508021405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae28107bb0 msgr2=0x7fae28100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae27400640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae28107bb0 0x7fae28100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae27400640 1 -- 172.21.15.57:0/3508021405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae28105280 msgr2=0x7fae28100900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae27400640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae28105280 0x7fae28100900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae27400640 1 -- 172.21.15.57:0/3508021405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae281016d0 con 0x7fae28102950 2024-04-13T05:26:58.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae27400640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 0x7fae281022c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fae1800d770 tx=0x7fae1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae1fe00640 1 -- 172.21.15.57:0/3508021405 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18017070 con 0x7fae28102950 2024-04-13T05:26:58.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae1fe00640 1 -- 172.21.15.57:0/3508021405 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae180054d0 con 0x7fae28102950 2024-04-13T05:26:58.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae1fe00640 1 -- 172.21.15.57:0/3508021405 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18004cf0 con 0x7fae28102950 2024-04-13T05:26:58.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.203+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae281a2260 con 0x7fae28102950 2024-04-13T05:26:58.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.204+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a2730 con 0x7fae28102950 2024-04-13T05:26:58.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.207+0000 7fae315cd740 1 Processor -- start 2024-04-13T05:26:58.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.207+0000 7fae1fe00640 1 -- 172.21.15.57:0/3508021405 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fae18004e90 con 0x7fae28102950 2024-04-13T05:26:58.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.207+0000 7fae315cd740 1 -- start start 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.207+0000 7fae315cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.207+0000 7fae315cd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.207+0000 7fae315cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae315cd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e40b435020 con 0x55e40b4cfe30 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae315cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e40b3db950 con 0x55e40b4d4fa0 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae315cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e40b430fe0 con 0x55e40b4cf440 2024-04-13T05:26:58.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45838/0 (socket says 172.21.15.57:45838) 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 -- 172.21.15.57:0/1124466482 learned_addr learned my addr 172.21.15.57:0/1124466482 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27e00640 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae26a00640 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 -- 172.21.15.57:0/1124466482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 msgr2=0x55e40b4cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 -- 172.21.15.57:0/1124466482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 msgr2=0x55e40b4d7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27400640 1 -- 172.21.15.57:0/1124466482 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40b1473f0 con 0x55e40b4cfe30 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.208+0000 7fae27e00640 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:58.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.209+0000 7fae27400640 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d4930 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fae180d84b0 tx=0x7fae180d8970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fa9adf19b22cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.209+0000 7fae1fe00640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fae0c0a3bb0 0x7fae0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.209+0000 7fae1cc00640 1 -- 172.21.15.57:0/1124466482 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180e3070 con 0x55e40b4cfe30 2024-04-13T05:26:58.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.209+0000 7fae1fe00640 1 -- 172.21.15.57:0/3508021405 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fae180d0330 con 0x7fae28102950 2024-04-13T05:26:58.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.209+0000 7fae1cc00640 1 -- 172.21.15.57:0/1124466482 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18009c30 con 0x55e40b4cfe30 2024-04-13T05:26:58.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.209+0000 7fae1cc00640 1 -- 172.21.15.57:0/1124466482 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180984b0 con 0x55e40b4cfe30 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae27e00640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fae0c0a3bb0 0x7fae0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 -- 172.21.15.57:0/1124466482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 msgr2=0x55e40b4d4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d4930 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fae180d84b0 tx=0x7fae180d8970 comp rx=0 tx=0).stop 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae27e00640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fae0c0a3bb0 0x7fae0c0a6070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55e40b4d0690 tx=0x7fae20009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 -- 172.21.15.57:0/1124466482 shutdown_connections 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d4930 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 --2- 172.21.15.57:0/1124466482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.210+0000 7fae315cd740 1 -- 172.21.15.57:0/1124466482 >> 172.21.15.57:0/1124466482 conn(0x55e40b4ca9b0 msgr2=0x55e40b4cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.211+0000 7fae315cd740 1 -- 172.21.15.57:0/1124466482 shutdown_connections 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.211+0000 7fae315cd740 1 -- 172.21.15.57:0/1124466482 wait complete. 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.211+0000 7fae315cd740 1 -- start start 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.211+0000 7fae315cd740 1 Processor -- start 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e40b4d7f50 con 0x55e40b4cfe30 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e40b3eec80 con 0x55e40b4d4fa0 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e40b4d80d0 con 0x55e40b4cf440 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae315cd740 10 client.0 _ll_register_callbacks cb 0x55e40b3ee670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae26a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae26a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45852/0 (socket says 172.21.15.57:45852) 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae26a00640 1 -- 172.21.15.57:0/4022392157 learned_addr learned my addr 172.21.15.57:0/4022392157 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.212+0000 7fae27400640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae27e00640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae26a00640 1 -- 172.21.15.57:0/4022392157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 msgr2=0x55e40b4d99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae26a00640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4d99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae26a00640 1 -- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 msgr2=0x55e40b4e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae26a00640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4e0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae26a00640 1 -- 172.21.15.57:0/4022392157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40b2117d0 con 0x55e40b4cfe30 2024-04-13T05:26:58.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae27e00640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:58.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae27400640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:58.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae26a00640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d9ef0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fae14000c00 tx=0x7fae14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba39acb658101c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae11600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:58.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae11600640 1 -- 172.21.15.57:0/4022392157 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14005d50 con 0x55e40b4cfe30 2024-04-13T05:26:58.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae11600640 1 -- 172.21.15.57:0/4022392157 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fae14005230 con 0x55e40b4cfe30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.213+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e40b3d4750 con 0x55e40b4cfe30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae11600640 1 -- 172.21.15.57:0/4022392157 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fae14050050 con 0x55e40b4cfe30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 10 client.17328 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e40b4e4310 con 0x55e40b4cfe30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae11600640 1 -- 172.21.15.57:0/4022392157 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e40b3d4750 con 0x55e40b4cfe30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae11600640 1 -- 172.21.15.57:0/4022392157 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fae140039c0 con 0x55e40b4cfe30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 10 client.17328 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 10 client.17328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 20 client.17328 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b4e5a30 0x55e40b4e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 4 client.17328 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.214+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40b442a40 con 0x55e40b4e5a30 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.215+0000 7fae27400640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b4e5a30 0x55e40b4e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.216+0000 7fae27400640 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b4e5a30 0x55e40b4e7eb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afef85986c9c693d server_cookie=bbddf290e14b1ecf in_seq=0 out_seq=0 2024-04-13T05:26:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.216+0000 7fae11600640 10 client.17328 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:58.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae11600640 1 -- 172.21.15.57:0/4022392157 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e40b442a40 con 0x55e40b4e5a30 2024-04-13T05:26:58.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae11600640 10 client.17328 handle_command_reply: tid=0 2024-04-13T05:26:58.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 client.17328 shutdown 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 msgr2=0x55e40b4d9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d9ef0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fae14000c00 tx=0x7fae14004270 comp rx=0 tx=0).stop 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 shutdown_connections 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b4e5a30 0x55e40b4e7eb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4d4fa0 0x55e40b4e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4cfe30 0x55e40b4d9ef0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 --2- 172.21.15.57:0/4022392157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cf440 0x55e40b4d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae11600640 0 client.17328 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 >> 172.21.15.57:0/4022392157 conn(0x55e40b4ca9b0 msgr2=0x55e40b4c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.218+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 shutdown_connections 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.219+0000 7fae315cd740 1 -- 172.21.15.57:0/4022392157 wait complete. 2024-04-13T05:26:58.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.219+0000 7fae315cd740 20 client.17328 trim_cache size 0 max 16384 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.236+0000 7fae315cd740 1 Processor -- start 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.236+0000 7fae315cd740 1 -- start start 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.236+0000 7fae315cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.236+0000 7fae315cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.236+0000 7fae315cd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae315cd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fae281af4e0 con 0x55e40b4d4fa0 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae315cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae281accf0 con 0x55e40b4cf440 2024-04-13T05:26:58.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae315cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae281aab40 con 0x55e40b4cfe30 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45864/0 (socket says 172.21.15.57:45864) 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 -- 172.21.15.57:0/1637109225 learned_addr learned my addr 172.21.15.57:0/1637109225 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae26a00640 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27400640 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 -- 172.21.15.57:0/1637109225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 msgr2=0x55e40b4e0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4e0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 -- 172.21.15.57:0/1637109225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 msgr2=0x55e40b4cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.237+0000 7fae27e00640 1 -- 172.21.15.57:0/1637109225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40b4d4650 con 0x55e40b4d4fa0 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.238+0000 7fae27e00640 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d8470 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fae200127b0 tx=0x7fae20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02c13925ac40fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.238+0000 7fae13e00640 1 -- 172.21.15.57:0/1637109225 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2001e930 con 0x55e40b4d4fa0 2024-04-13T05:26:58.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.238+0000 7fae13e00640 1 -- 172.21.15.57:0/1637109225 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae2001f420 con 0x55e40b4d4fa0 2024-04-13T05:26:58.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.238+0000 7fae13e00640 1 -- 172.21.15.57:0/1637109225 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2001c650 con 0x55e40b4d4fa0 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 -- 172.21.15.57:0/1637109225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 msgr2=0x55e40b4d8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d8470 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fae200127b0 tx=0x7fae20014000 comp rx=0 tx=0).stop 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 -- 172.21.15.57:0/1637109225 shutdown_connections 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b4d8470 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4e0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 --2- 172.21.15.57:0/1637109225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 -- 172.21.15.57:0/1637109225 >> 172.21.15.57:0/1637109225 conn(0x55e40b4ca9b0 msgr2=0x55e40b4d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 -- 172.21.15.57:0/1637109225 shutdown_connections 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.239+0000 7fae315cd740 1 -- 172.21.15.57:0/1637109225 wait complete. 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.240+0000 7fae315cd740 1 -- start start 2024-04-13T05:26:58.243 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.240+0000 7fae315cd740 1 Processor -- start 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4de1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b552830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ab1b0 con 0x55e40b4d4fa0 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ac620 con 0x55e40b4cf440 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ab880 con 0x55e40b4cfe30 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae315cd740 10 client.0 _ll_register_callbacks cb 0x7fae14009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae27e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b552830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae27e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b552830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45880/0 (socket says 172.21.15.57:45880) 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae27e00640 1 -- 172.21.15.57:0/4108942602 learned_addr learned my addr 172.21.15.57:0/4108942602 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae27400640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.241+0000 7fae26a00640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4de1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27e00640 1 -- 172.21.15.57:0/4108942602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 msgr2=0x55e40b4de1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27e00640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4de1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27e00640 1 -- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 msgr2=0x55e40b4ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27e00640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27e00640 1 -- 172.21.15.57:0/4108942602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40b4de7e0 con 0x55e40b4d4fa0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27400640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae27e00640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b552830 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fae20012780 tx=0x7fae20009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb3414ddc3c7aac server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae11600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae11600640 1 -- 172.21.15.57:0/4108942602 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20036030 con 0x55e40b4d4fa0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae11600640 1 -- 172.21.15.57:0/4108942602 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fae2002d040 con 0x55e40b4d4fa0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.242+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2001f420 con 0x55e40b4d4fa0 2024-04-13T05:26:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae11600640 1 -- 172.21.15.57:0/4108942602 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fae20040050 con 0x55e40b4d4fa0 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 10 client.17334 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e40b553e00 con 0x55e40b4d4fa0 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae11600640 1 -- 172.21.15.57:0/4108942602 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae2001f420 con 0x55e40b4d4fa0 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae11600640 1 -- 172.21.15.57:0/4108942602 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fae2000f040 con 0x55e40b4d4fa0 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 10 client.17334 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 10 client.17334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 20 client.17334 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b554890 0x55e40b556cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 4 client.17334 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:26:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.243+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e40b43c8b0 con 0x55e40b554890 2024-04-13T05:26:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.244+0000 7fae27400640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b554890 0x55e40b556cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.245+0000 7fae27400640 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b554890 0x55e40b556cb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52d4cdde56ddb961 server_cookie=7495c73bd8410b2d in_seq=0 out_seq=0 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.245+0000 7fae11600640 10 client.17334 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae11600640 1 -- 172.21.15.57:0/4108942602 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e40b43c8b0 con 0x55e40b554890 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae11600640 10 client.17334 handle_command_reply: tid=0 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 client.17334 shutdown 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 msgr2=0x55e40b552830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b552830 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fae20012780 tx=0x7fae20009d20 comp rx=0 tx=0).stop 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 shutdown_connections 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55e40b554890 0x55e40b556cb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40b4d4fa0 0x55e40b552830 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e40b4cfe30 0x55e40b4de1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 --2- 172.21.15.57:0/4108942602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e40b4cf440 0x55e40b4ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae11600640 0 client.17334 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:26:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 >> 172.21.15.57:0/4108942602 conn(0x55e40b4ca9b0 msgr2=0x55e40b4cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 shutdown_connections 2024-04-13T05:26:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 1 -- 172.21.15.57:0/4108942602 wait complete. 2024-04-13T05:26:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.246+0000 7fae315cd740 20 client.17334 trim_cache size 0 max 16384 2024-04-13T05:26:58.251 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fae0c0a3bb0 msgr2=0x7fae0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fae0c0a3bb0 0x7fae0c0a6070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55e40b4d0690 tx=0x7fae20009580 comp rx=0 tx=0).stop 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 msgr2=0x7fae281022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 0x7fae281022c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fae1800d770 tx=0x7fae1800dc30 comp rx=0 tx=0).stop 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 shutdown_connections 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fae0c0a3bb0 0x7fae0c0a6070 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae28105280 0x7fae28100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae28107bb0 0x7fae28100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 --2- 172.21.15.57:0/3508021405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae28102950 0x7fae281022c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 >> 172.21.15.57:0/3508021405 conn(0x7fae280fc360 msgr2=0x7fae281033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 shutdown_connections 2024-04-13T05:26:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.247+0000 7fae2d000640 1 -- 172.21.15.57:0/3508021405 wait complete. 2024-04-13T05:26:58.266 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4096805852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:26:58.269 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:26:58.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4096805852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 -- 172.21.15.57:0/156608600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88101e80 msgr2=0x7f1c881022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/156608600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88101e80 0x7f1c881022a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009900 tx=0x7f1c7802bc40 comp rx=0 tx=0).stop 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 -- 172.21.15.57:0/156608600 shutdown_connections 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/156608600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c88103a20 0x7f1c8810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/156608600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c88103060 0x7f1c881034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/156608600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88101e80 0x7f1c881022a0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 -- 172.21.15.57:0/156608600 >> 172.21.15.57:0/156608600 conn(0x7f1c880fd460 msgr2=0x7f1c880ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 -- 172.21.15.57:0/156608600 shutdown_connections 2024-04-13T05:26:58.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.494+0000 7f1c8ce00640 1 -- 172.21.15.57:0/156608600 wait complete. 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.495+0000 7f1c8ce00640 1 Processor -- start 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.495+0000 7f1c8ce00640 1 -- start start 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c8ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c88101e80 0x7f1c88107d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c8ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c88103060 0x7f1c88108290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c8ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 0x7f1c881048c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c8ce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810da50 con 0x7f1c88103a20 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c8ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810d8d0 con 0x7f1c88103060 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c8ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810dbd0 con 0x7f1c88101e80 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 0x7f1c881048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 0x7f1c881048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45904/0 (socket says 172.21.15.57:45904) 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c87e00640 1 -- 172.21.15.57:0/3948457706 learned_addr learned my addr 172.21.15.57:0/3948457706 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c86a00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c88103060 0x7f1c88108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.496+0000 7f1c87400640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c88101e80 0x7f1c88107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c87e00640 1 -- 172.21.15.57:0/3948457706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c88101e80 msgr2=0x7f1c88107d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c87e00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c88101e80 0x7f1c88107d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c87e00640 1 -- 172.21.15.57:0/3948457706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c88103060 msgr2=0x7f1c88108290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c87e00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c88103060 0x7f1c88108290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c87e00640 1 -- 172.21.15.57:0/3948457706 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c88105180 con 0x7f1c88103a20 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c87e00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 0x7f1c881048c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000cad0 tx=0x7f1c8000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c7fe00640 1 -- 172.21.15.57:0/3948457706 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000ae20 con 0x7f1c88103a20 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c7fe00640 1 -- 172.21.15.57:0/3948457706 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c80005020 con 0x7f1c88103a20 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c881a6720 con 0x7f1c88103a20 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c7fe00640 1 -- 172.21.15.57:0/3948457706 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000f6b0 con 0x7f1c88103a20 2024-04-13T05:26:58.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.497+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881a6b30 con 0x7f1c88103a20 2024-04-13T05:26:58.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4096805852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:26:58.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.501+0000 7f1c7fe00640 1 -- 172.21.15.57:0/3948457706 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c800051d0 con 0x7f1c88103a20 2024-04-13T05:26:58.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.501+0000 7f1c913b4740 1 Processor -- start 2024-04-13T05:26:58.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.501+0000 7f1c913b4740 1 -- start start 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa33991860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa33996930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa33999430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c913b4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55fa338f7020 con 0x55fa33996fa0 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c913b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa3389d950 con 0x55fa33991e30 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c913b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa338f2fe0 con 0x55fa33991440 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c86a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa33999430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c86a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa33999430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45916/0 (socket says 172.21.15.57:45916) 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c86a00640 1 -- 172.21.15.57:0/250821527 learned_addr learned my addr 172.21.15.57:0/250821527 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c87400640 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa33996930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.502+0000 7f1c86a00640 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa33991860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c86a00640 1 -- 172.21.15.57:0/250821527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 msgr2=0x55fa33991860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c86a00640 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa33991860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c86a00640 1 -- 172.21.15.57:0/250821527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 msgr2=0x55fa33996930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c86a00640 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa33996930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c86a00640 1 -- 172.21.15.57:0/250821527 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa336093f0 con 0x55fa33996fa0 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c86a00640 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa33999430 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400bfd0 tx=0x7f1c7400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cabd0a5958c07a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c7fe00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1c6c0a3bb0 0x7f1c6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.503+0000 7f1c7cc00640 1 -- 172.21.15.57:0/250821527 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400e950 con 0x55fa33996fa0 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.504+0000 7f1c7cc00640 1 -- 172.21.15.57:0/250821527 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c74011440 con 0x55fa33996fa0 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.504+0000 7f1c7cc00640 1 -- 172.21.15.57:0/250821527 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74010670 con 0x55fa33996fa0 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.504+0000 7f1c7fe00640 1 -- 172.21.15.57:0/3948457706 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1c800d0930 con 0x7f1c88103a20 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.504+0000 7f1c87400640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1c6c0a3bb0 0x7f1c6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.504+0000 7f1c913b4740 1 -- 172.21.15.57:0/250821527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 msgr2=0x55fa33999430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.504+0000 7f1c913b4740 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa33999430 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400bfd0 tx=0x7f1c7400b460 comp rx=0 tx=0).stop 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 -- 172.21.15.57:0/250821527 shutdown_connections 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa33996930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa33999430 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 --2- 172.21.15.57:0/250821527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa33991860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 -- 172.21.15.57:0/250821527 >> 172.21.15.57:0/250821527 conn(0x55fa3398c9b0 msgr2=0x55fa3398cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 -- 172.21.15.57:0/250821527 shutdown_connections 2024-04-13T05:26:58.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 -- 172.21.15.57:0/250821527 wait complete. 2024-04-13T05:26:58.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c87400640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1c6c0a3bb0 0x7f1c6c0a6070 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1c7802c250 tx=0x7f1c7802c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.505+0000 7f1c913b4740 1 -- start start 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 Processor -- start 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3399b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa3399bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55fa33999f50 con 0x55fa33996fa0 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa338b0c80 con 0x55fa33991e30 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa3399a0d0 con 0x55fa33991440 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c913b4740 10 client.0 _ll_register_callbacks cb 0x55fa338b0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:58.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45930/0 (socket says 172.21.15.57:45930) 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.506+0000 7f1c87e00640 1 -- 172.21.15.57:0/3902883483 learned_addr learned my addr 172.21.15.57:0/3902883483 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c86a00640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa3399bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87400640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3399b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87e00640 1 -- 172.21.15.57:0/3902883483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 msgr2=0x55fa3399b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87e00640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3399b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87e00640 1 -- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 msgr2=0x55fa3399bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87e00640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa3399bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87e00640 1 -- 172.21.15.57:0/3902883483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa336d37d0 con 0x55fa33996fa0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c87e00640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a2e90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1c800991e0 tx=0x7f1c8009d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef61d71e438478d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c71600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c71600640 1 -- 172.21.15.57:0/3902883483 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001d040 con 0x55fa33996fa0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c71600640 1 -- 172.21.15.57:0/3902883483 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1c800870c0 con 0x55fa33996fa0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.507+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c74011440 con 0x55fa33996fa0 2024-04-13T05:26:58.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c71600640 1 -- 172.21.15.57:0/3902883483 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c8009db40 con 0x55fa33996fa0 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 10 client.17346 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fa339a62d0 con 0x55fa33996fa0 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c71600640 1 -- 172.21.15.57:0/3902883483 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c74011440 con 0x55fa33996fa0 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c71600640 1 -- 172.21.15.57:0/3902883483 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f1c80098660 con 0x55fa33996fa0 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 10 client.17346 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 20 client.17346 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa339a7a90 0x55fa339a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 4 client.17346 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.508+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa33896750 con 0x55fa339a7a90 2024-04-13T05:26:58.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.509+0000 7f1c87400640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa339a7a90 0x55fa339a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.510+0000 7f1c87400640 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa339a7a90 0x55fa339a9ee0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beafa4d280202d22 server_cookie=9087c775c25369e8 in_seq=0 out_seq=0 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.510+0000 7f1c71600640 10 client.17346 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c71600640 1 -- 172.21.15.57:0/3902883483 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa33896750 con 0x55fa339a7a90 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c71600640 10 client.17346 handle_command_reply: tid=0 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 client.17346 shutdown 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 msgr2=0x55fa339a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a2e90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1c800991e0 tx=0x7f1c8009d050 comp rx=0 tx=0).stop 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 shutdown_connections 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa339a7a90 0x55fa339a9ee0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa3399bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a2e90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 --2- 172.21.15.57:0/3902883483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3399b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c71600640 0 client.17346 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 >> 172.21.15.57:0/3902883483 conn(0x55fa3398c9b0 msgr2=0x55fa3398bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.512+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 shutdown_connections 2024-04-13T05:26:58.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.513+0000 7f1c913b4740 1 -- 172.21.15.57:0/3902883483 wait complete. 2024-04-13T05:26:58.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.513+0000 7f1c913b4740 20 client.17346 trim_cache size 0 max 16384 2024-04-13T05:26:58.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.528+0000 7f1c913b4740 1 Processor -- start 2024-04-13T05:26:58.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.528+0000 7f1c913b4740 1 -- start start 2024-04-13T05:26:58.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.528+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3398b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.528+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa339a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa3398b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c913b4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b3930 con 0x55fa33996fa0 2024-04-13T05:26:58.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c913b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b1140 con 0x55fa33991e30 2024-04-13T05:26:58.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c913b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881aef90 con 0x55fa33991440 2024-04-13T05:26:58.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa3398b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa3398b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45938/0 (socket says 172.21.15.57:45938) 2024-04-13T05:26:58.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 -- 172.21.15.57:0/2899596732 learned_addr learned my addr 172.21.15.57:0/2899596732 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87400640 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3398b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c86a00640 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa339a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 -- 172.21.15.57:0/2899596732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 msgr2=0x55fa3398b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3398b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 -- 172.21.15.57:0/2899596732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 msgr2=0x55fa339a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa339a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.529+0000 7f1c87e00640 1 -- 172.21.15.57:0/2899596732 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa33992210 con 0x55fa33996fa0 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.530+0000 7f1c87e00640 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa3398b980 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1c80098c70 tx=0x7f1c8009ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad732632100481b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.530+0000 7f1c73e00640 1 -- 172.21.15.57:0/2899596732 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001d070 con 0x55fa33996fa0 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.530+0000 7f1c73e00640 1 -- 172.21.15.57:0/2899596732 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c800870b0 con 0x55fa33996fa0 2024-04-13T05:26:58.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.530+0000 7f1c73e00640 1 -- 172.21.15.57:0/2899596732 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800a2110 con 0x55fa33996fa0 2024-04-13T05:26:58.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 -- 172.21.15.57:0/2899596732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 msgr2=0x55fa3398b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa3398b980 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1c80098c70 tx=0x7f1c8009ad20 comp rx=0 tx=0).stop 2024-04-13T05:26:58.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 -- 172.21.15.57:0/2899596732 shutdown_connections 2024-04-13T05:26:58.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991e30 0x55fa339a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa3398b980 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2899596732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991440 0x55fa3398b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 -- 172.21.15.57:0/2899596732 >> 172.21.15.57:0/2899596732 conn(0x55fa3398c9b0 msgr2=0x55fa33993360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 -- 172.21.15.57:0/2899596732 shutdown_connections 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 -- 172.21.15.57:0/2899596732 wait complete. 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.531+0000 7f1c913b4740 1 -- start start 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 Processor -- start 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991440 0x55fa339a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991e30 0x55fa339a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881af600 con 0x55fa33996fa0 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b0a70 con 0x55fa33991440 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881afcd0 con 0x55fa33991e30 2024-04-13T05:26:58.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c913b4740 10 client.0 _ll_register_callbacks cb 0x7f1c8009c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.532+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45944/0 (socket says 172.21.15.57:45944) 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 -- 172.21.15.57:0/2925886758 learned_addr learned my addr 172.21.15.57:0/2925886758 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87400640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991440 0x55fa339a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c86a00640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991e30 0x55fa339a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 -- 172.21.15.57:0/2925886758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991e30 msgr2=0x55fa339a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991e30 0x55fa339a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 -- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991440 msgr2=0x55fa339a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991440 0x55fa339a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 -- 172.21.15.57:0/2925886758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa339a1ea0 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c87e00640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a18b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1c8009c6c0 tx=0x7f1c800a2240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f541fdafaa74fc server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c71600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.533+0000 7f1c71600640 1 -- 172.21.15.57:0/2925886758 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80066070 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c800870b0 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c71600640 1 -- 172.21.15.57:0/2925886758 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1c8009bc60 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c71600640 1 -- 172.21.15.57:0/2925886758 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c8001d070 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 10 client.17352 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fa339a31a0 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c71600640 1 -- 172.21.15.57:0/2925886758 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c800870b0 con 0x55fa33996fa0 2024-04-13T05:26:58.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c71600640 1 -- 172.21.15.57:0/2925886758 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f1c8009dce0 con 0x55fa33996fa0 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 10 client.17352 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 20 client.17352 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa33a0e750 0x55fa33a10b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 4 client.17352 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.534+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa33904a40 con 0x55fa33a0e750 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.535+0000 7f1c87400640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa33a0e750 0x55fa33a10b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.536+0000 7f1c87400640 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa33a0e750 0x55fa33a10b70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0db40b951afc88 server_cookie=b5cceb55e9abd088 in_seq=0 out_seq=0 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.536+0000 7f1c71600640 10 client.17352 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c71600640 1 -- 172.21.15.57:0/2925886758 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa33904a40 con 0x55fa33a0e750 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c71600640 10 client.17352 handle_command_reply: tid=0 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 client.17352 shutdown 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 msgr2=0x55fa339a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a18b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1c8009c6c0 tx=0x7f1c800a2240 comp rx=0 tx=0).stop 2024-04-13T05:26:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 shutdown_connections 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55fa33a0e750 0x55fa33a10b70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fa33996fa0 0x55fa339a18b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa33991e30 0x55fa339a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 --2- 172.21.15.57:0/2925886758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa33991440 0x55fa339a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c71600640 0 client.17352 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 >> 172.21.15.57:0/2925886758 conn(0x55fa3398c9b0 msgr2=0x55fa3398fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 shutdown_connections 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 1 -- 172.21.15.57:0/2925886758 wait complete. 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.537+0000 7f1c913b4740 20 client.17352 trim_cache size 0 max 16384 2024-04-13T05:26:58.542 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:26:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1c6c0a3bb0 msgr2=0x7f1c6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1c6c0a3bb0 0x7f1c6c0a6070 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1c7802c250 tx=0x7f1c7802c7f0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 msgr2=0x7f1c881048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 0x7f1c881048c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000cad0 tx=0x7f1c8000cf90 comp rx=0 tx=0).stop 2024-04-13T05:26:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 shutdown_connections 2024-04-13T05:26:58.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1c6c0a3bb0 0x7f1c6c0a6070 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c88103060 0x7f1c88108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c88103a20 0x7f1c881048c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 --2- 172.21.15.57:0/3948457706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c88101e80 0x7f1c88107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 >> 172.21.15.57:0/3948457706 conn(0x7f1c880fd460 msgr2=0x7f1c880741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 shutdown_connections 2024-04-13T05:26:58.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:58.538+0000 7f1c8ce00640 1 -- 172.21.15.57:0/3948457706 wait complete. 2024-04-13T05:26:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:26:59 smithi083 ceph-mon[35133]: pgmap v737: 129 pgs: 129 active+clean; 369 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 204 KiB/s wr, 88 op/s 2024-04-13T05:26:59.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:26:59 smithi057 ceph-mon[32331]: pgmap v737: 129 pgs: 129 active+clean; 369 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 204 KiB/s wr, 88 op/s 2024-04-13T05:26:59.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:26:59.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:26:59.560 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:26:59.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:26:59 smithi189 ceph-mon[35531]: pgmap v737: 129 pgs: 129 active+clean; 369 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 204 KiB/s wr, 88 op/s 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 -- 172.21.15.57:0/2965992247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 msgr2=0x7f4698074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 --2- 172.21.15.57:0/2965992247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f4698074e20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4680009900 tx=0x7f468002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 -- 172.21.15.57:0/2965992247 shutdown_connections 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 --2- 172.21.15.57:0/2965992247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 0x7f469810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 --2- 172.21.15.57:0/2965992247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f4698074e20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 --2- 172.21.15.57:0/2965992247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4698075fa0 0x7f46980743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 -- 172.21.15.57:0/2965992247 >> 172.21.15.57:0/2965992247 conn(0x7f46980fd5a0 msgr2=0x7f46980ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 -- 172.21.15.57:0/2965992247 shutdown_connections 2024-04-13T05:26:59.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.792+0000 7f469f000640 1 -- 172.21.15.57:0/2965992247 wait complete. 2024-04-13T05:26:59.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.793+0000 7f469f000640 1 Processor -- start 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.793+0000 7f469f000640 1 -- start start 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.793+0000 7f469f000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f469819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4698075fa0 0x7f469819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 0x7f46981a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469f000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f469810fda0 con 0x7f46980749c0 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f469810fc20 con 0x7f4698075fa0 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f469810ff20 con 0x7f4698109ea0 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f469819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f469819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45952/0 (socket says 172.21.15.57:45952) 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 -- 172.21.15.57:0/2481609791 learned_addr learned my addr 172.21.15.57:0/2481609791 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469d000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 0x7f46981a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f468fe00640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4698075fa0 0x7f469819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 -- 172.21.15.57:0/2481609791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 msgr2=0x7f46981a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 0x7f46981a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 -- 172.21.15.57:0/2481609791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4698075fa0 msgr2=0x7f469819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4698075fa0 0x7f469819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f4697e00640 1 -- 172.21.15.57:0/2481609791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46981a2080 con 0x7f46980749c0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.794+0000 7f469d000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 0x7f46981a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.795+0000 7f4697e00640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f469819a460 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f468800d950 tx=0x7f468800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.795+0000 7f4695600640 1 -- 172.21.15.57:0/2481609791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688017070 con 0x7f46980749c0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.795+0000 7f4695600640 1 -- 172.21.15.57:0/2481609791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4688009c30 con 0x7f46980749c0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.795+0000 7f4695600640 1 -- 172.21.15.57:0/2481609791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468800f620 con 0x7f46980749c0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.795+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46981a2370 con 0x7f46980749c0 2024-04-13T05:26:59.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.795+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46981a2840 con 0x7f46980749c0 2024-04-13T05:26:59.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 Processor -- start 2024-04-13T05:26:59.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f4695600640 1 -- 172.21.15.57:0/2481609791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f46880109a0 con 0x7f46980749c0 2024-04-13T05:26:59.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 -- start start 2024-04-13T05:26:59.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1bc70 0x5628fde20770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 0x5628fde23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5628fdd81bc0 con 0x5628fde20de0 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628fdd28660 con 0x5628fde1bc70 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5628fdd7db80 con 0x5628fde1b280 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.799+0000 7f468fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 0x5628fde23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 0x5628fde23270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45960/0 (socket says 172.21.15.57:45960) 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 -- 172.21.15.57:0/620639953 learned_addr learned my addr 172.21.15.57:0/620639953 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f4697e00640 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1bc70 0x5628fde20770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f469d000640 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 -- 172.21.15.57:0/620639953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 msgr2=0x5628fde1b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde1b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 -- 172.21.15.57:0/620639953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1bc70 msgr2=0x5628fde20770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1bc70 0x5628fde20770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.800+0000 7f468fe00640 1 -- 172.21.15.57:0/620639953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628fdd4b220 con 0x5628fde20de0 2024-04-13T05:26:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f4695600640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46780a3c30 0x7f46780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f468fe00640 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 0x5628fde23270 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f468002c8e0 tx=0x7f4680005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df550f2d1577847 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f4695600640 1 -- 172.21.15.57:0/2481609791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f46880d0400 con 0x7f46980749c0 2024-04-13T05:26:59.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f469d000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46780a3c30 0x7f46780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f468d600640 1 -- 172.21.15.57:0/620639953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680004340 con 0x5628fde20de0 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f468d600640 1 -- 172.21.15.57:0/620639953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468002ce30 con 0x5628fde20de0 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.801+0000 7f468d600640 1 -- 172.21.15.57:0/620639953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800048e0 con 0x5628fde20de0 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 -- 172.21.15.57:0/620639953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 msgr2=0x5628fde23270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f469d000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46780a3c30 0x7f46780a60f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5628fde1c4d0 tx=0x7f4690009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 0x5628fde23270 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f468002c8e0 tx=0x7f4680005dd0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 -- 172.21.15.57:0/620639953 shutdown_connections 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1bc70 0x5628fde20770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde20de0 0x5628fde23270 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/620639953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 -- 172.21.15.57:0/620639953 >> 172.21.15.57:0/620639953 conn(0x5628fde167f0 msgr2=0x5628fde16bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 -- 172.21.15.57:0/620639953 shutdown_connections 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.802+0000 7f46a1e4b740 1 -- 172.21.15.57:0/620639953 wait complete. 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 -- start start 2024-04-13T05:26:59.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 Processor -- start 2024-04-13T05:26:59.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde25c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde20de0 0x5628fde2cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5628fde23d30 con 0x5628fde1bc70 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.803+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628fdd3ba30 con 0x5628fde20de0 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5628fde23eb0 con 0x5628fde1b280 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f46a1e4b740 10 client.0 _ll_register_callbacks cb 0x5628fda93c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde25c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde25c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45970/0 (socket says 172.21.15.57:45970) 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 -- 172.21.15.57:0/1880441197 learned_addr learned my addr 172.21.15.57:0/1880441197 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f4697e00640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f469d000640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde20de0 0x5628fde2cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 -- 172.21.15.57:0/1880441197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 msgr2=0x5628fde25710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde25710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 -- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde20de0 msgr2=0x5628fde2cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde20de0 0x5628fde2cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f468fe00640 1 -- 172.21.15.57:0/1880441197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628fdb5c7e0 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.804+0000 7f469d000640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde20de0 0x5628fde2cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f468fe00640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde25c50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f46800096c0 tx=0x7f46800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66cdbee500514b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f4686000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f4686000640 1 -- 172.21.15.57:0/1880441197 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680054060 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f4686000640 1 -- 172.21.15.57:0/1880441197 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f46800035f0 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468002ce30 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f4686000640 1 -- 172.21.15.57:0/1880441197 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f468005b050 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f46a1e4b740 10 client.17364 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5628fdbbcf70 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.805+0000 7f4686000640 1 -- 172.21.15.57:0/1880441197 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468002ce30 con 0x5628fde1bc70 2024-04-13T05:26:59.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f4686000640 1 -- 172.21.15.57:0/1880441197 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f4680034a20 con 0x5628fde1bc70 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f46a1e4b740 10 client.17364 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f46a1e4b740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f46a1e4b740 20 client.17364 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde31180 0x5628fde33570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f46a1e4b740 4 client.17364 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628fdd8f5e0 con 0x5628fde31180 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.806+0000 7f4697e00640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde31180 0x5628fde33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.807+0000 7f4697e00640 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde31180 0x5628fde33570 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbbbbabed4d685f8 server_cookie=f888845db3387d2 in_seq=0 out_seq=0 2024-04-13T05:26:59.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.807+0000 7f4686000640 10 client.17364 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f4686000640 1 -- 172.21.15.57:0/1880441197 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628fdd8f5e0 con 0x5628fde31180 2024-04-13T05:26:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f4686000640 10 client.17364 handle_command_reply: tid=0 2024-04-13T05:26:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 client.17364 shutdown 2024-04-13T05:26:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 msgr2=0x5628fde25c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde25c50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f46800096c0 tx=0x7f46800042f0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 shutdown_connections 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde31180 0x5628fde33570 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde20de0 0x5628fde2cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde25c50 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/1880441197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1b280 0x5628fde25710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f4686000640 0 client.17364 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.809+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 >> 172.21.15.57:0/1880441197 conn(0x5628fde167f0 msgr2=0x5628fde159f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.810+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 shutdown_connections 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.810+0000 7f46a1e4b740 1 -- 172.21.15.57:0/1880441197 wait complete. 2024-04-13T05:26:59.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.810+0000 7f46a1e4b740 20 client.17364 trim_cache size 0 max 16384 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 Processor -- start 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 -- start start 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 0x5628fde1a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1bc70 0x5628fde37dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f46981af720 con 0x5628fde167f0 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46981acf30 con 0x5628fde1b280 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46981aad80 con 0x5628fde1bc70 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f4697e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 0x5628fde1a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f4697e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 0x5628fde1a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45978/0 (socket says 172.21.15.57:45978) 2024-04-13T05:26:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.830+0000 7f4697e00640 1 -- 172.21.15.57:0/3384838388 learned_addr learned my addr 172.21.15.57:0/3384838388 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f4697e00640 1 -- 172.21.15.57:0/3384838388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1bc70 msgr2=0x5628fde37dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f4697e00640 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1bc70 0x5628fde37dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f468fe00640 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f4697e00640 1 -- 172.21.15.57:0/3384838388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 msgr2=0x5628fde2cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f4697e00640 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f4697e00640 1 -- 172.21.15.57:0/3384838388 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628fda54170 con 0x5628fde167f0 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f4697e00640 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 0x5628fde1a160 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f46880cded0 tx=0x7f46880d2910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb5367885421c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f468cc00640 1 -- 172.21.15.57:0/3384838388 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688020040 con 0x5628fde167f0 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f468cc00640 1 -- 172.21.15.57:0/3384838388 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4688009de0 con 0x5628fde167f0 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.831+0000 7f468cc00640 1 -- 172.21.15.57:0/3384838388 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468809a640 con 0x5628fde167f0 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3384838388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 msgr2=0x5628fde1a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 0x5628fde1a160 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f46880cded0 tx=0x7f46880d2910 comp rx=0 tx=0).stop 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3384838388 shutdown_connections 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde1bc70 0x5628fde37dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3384838388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde167f0 0x5628fde1a160 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3384838388 >> 172.21.15.57:0/3384838388 conn(0x5628fde16f40 msgr2=0x5628fde19a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3384838388 shutdown_connections 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3384838388 wait complete. 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.832+0000 7f46a1e4b740 1 -- start start 2024-04-13T05:26:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 Processor -- start 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde167f0 0x5628fde2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f46981ab3f0 con 0x5628fde1bc70 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46981ac860 con 0x5628fde1b280 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46981abac0 con 0x5628fde167f0 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.833+0000 7f46a1e4b740 10 client.0 _ll_register_callbacks cb 0x7f46880d3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde2b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45992/0 (socket says 172.21.15.57:45992) 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 -- 172.21.15.57:0/3073475052 learned_addr learned my addr 172.21.15.57:0/3073475052 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f468fe00640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f4697e00640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde167f0 0x5628fde2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 -- 172.21.15.57:0/3073475052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde167f0 msgr2=0x5628fde2c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde167f0 0x5628fde2c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 -- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 msgr2=0x5628fde2ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 -- 172.21.15.57:0/3073475052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628fdd3cbe0 con 0x5628fde1bc70 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f469d000640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde2b360 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f46900095d0 tx=0x7f4690004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a82b951ae9ef4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.834+0000 7f4686000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f4686000640 1 -- 172.21.15.57:0/3073475052 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690020b20 con 0x5628fde1bc70 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4688009de0 con 0x5628fde1bc70 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f4686000640 1 -- 172.21.15.57:0/3073475052 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4690020cc0 con 0x5628fde1bc70 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f4686000640 1 -- 172.21.15.57:0/3073475052 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f469001e480 con 0x5628fde1bc70 2024-04-13T05:26:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f46a1e4b740 10 client.17370 fetch_fsmap learned FSMap version 33 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5628fde982f0 con 0x5628fde1bc70 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.835+0000 7f4686000640 1 -- 172.21.15.57:0/3073475052 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4688009de0 con 0x5628fde1bc70 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f4686000640 1 -- 172.21.15.57:0/3073475052 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f4690027070 con 0x5628fde1bc70 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f46a1e4b740 10 client.17370 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f46a1e4b740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f46a1e4b740 20 client.17370 populate_metadata read hostname 'smithi057' 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde20de0 0x5628fde9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f46a1e4b740 4 client.17370 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5628fdd89450 con 0x5628fde20de0 2024-04-13T05:26:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.836+0000 7f4697e00640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde20de0 0x5628fde9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:26:59.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.837+0000 7f4697e00640 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde20de0 0x5628fde9ace0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961bad548db4bd71 server_cookie=8c6d121f67cfb731 in_seq=0 out_seq=0 2024-04-13T05:26:59.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.837+0000 7f4686000640 10 client.17370 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:59.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.840+0000 7f4686000640 1 -- 172.21.15.57:0/3073475052 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5628fdd89450 con 0x5628fde20de0 2024-04-13T05:26:59.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.840+0000 7f4686000640 10 client.17370 handle_command_reply: tid=0 2024-04-13T05:26:59.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.840+0000 7f46a1e4b740 1 client.17370 shutdown 2024-04-13T05:26:59.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.840+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 msgr2=0x5628fde2b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.840+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde2b360 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f46900095d0 tx=0x7f4690004ab0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 shutdown_connections 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5628fde20de0 0x5628fde9ace0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628fde1b280 0x5628fde2ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5628fde1bc70 0x5628fde2b360 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 --2- 172.21.15.57:0/3073475052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fde167f0 0x5628fde2c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 >> 172.21.15.57:0/3073475052 conn(0x5628fde16f40 msgr2=0x5628fde1a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f4686000640 0 client.17370 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 shutdown_connections 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 1 -- 172.21.15.57:0/3073475052 wait complete. 2024-04-13T05:26:59.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.841+0000 7f46a1e4b740 20 client.17370 trim_cache size 0 max 16384 2024-04-13T05:26:59.845 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:26:59.845 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:26:59.845 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "ad3214a3-f869-4254-b082-d3ef3496b4ad", 2024-04-13T05:26:59.845 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:26:59.845 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46780a3c30 msgr2=0x7f46780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46780a3c30 0x7f46780a60f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5628fde1c4d0 tx=0x7f4690009800 comp rx=0 tx=0).stop 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 msgr2=0x7f469819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f469819a460 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f468800d950 tx=0x7f468800de10 comp rx=0 tx=0).stop 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 shutdown_connections 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f46780a3c30 0x7f46780a60f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4698075fa0 0x7f469819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4698109ea0 0x7f46981a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 --2- 172.21.15.57:0/2481609791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46980749c0 0x7f469819a460 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 >> 172.21.15.57:0/2481609791 conn(0x7f46980fd5a0 msgr2=0x7f4698105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:26:59.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 shutdown_connections 2024-04-13T05:26:59.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:26:59.842+0000 7f469f000640 1 -- 172.21.15.57:0/2481609791 wait complete. 2024-04-13T05:26:59.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:26:59.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:27:00.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 -- 172.21.15.57:0/3162165596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c102a80 msgr2=0x7ff09c104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/3162165596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c102a80 0x7ff09c104e70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009900 tx=0x7ff08c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:27:00.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 -- 172.21.15.57:0/3162165596 shutdown_connections 2024-04-13T05:27:00.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/3162165596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff09c107ce0 0x7ff09c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/3162165596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c1053b0 0x7ff09c1077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/3162165596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c102a80 0x7ff09c104e70 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.115+0000 7ff0a2a00640 1 -- 172.21.15.57:0/3162165596 >> 172.21.15.57:0/3162165596 conn(0x7ff09c0fc470 msgr2=0x7ff09c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:00.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.116+0000 7ff0a2a00640 1 -- 172.21.15.57:0/3162165596 shutdown_connections 2024-04-13T05:27:00.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.116+0000 7ff0a2a00640 1 -- 172.21.15.57:0/3162165596 wait complete. 2024-04-13T05:27:00.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.117+0000 7ff0a2a00640 1 Processor -- start 2024-04-13T05:27:00.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.117+0000 7ff0a2a00640 1 -- start start 2024-04-13T05:27:00.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.117+0000 7ff0a2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 0x7ff09c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.117+0000 7ff0a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff09c1053b0 0x7ff09c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.117+0000 7ff0a2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c107ce0 0x7ff09c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff0a2a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c10db40 con 0x7ff09c107ce0 2024-04-13T05:27:00.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff0a2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c10d9c0 con 0x7ff09c1053b0 2024-04-13T05:27:00.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff0a2a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c10dcc0 con 0x7ff09c102a80 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff09be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 0x7ff09c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff09b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff09c1053b0 0x7ff09c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff09be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 0x7ff09c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:36170/0 (socket says 172.21.15.57:36170) 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff09be00640 1 -- 172.21.15.57:0/1477405893 learned_addr learned my addr 172.21.15.57:0/1477405893 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.118+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c107ce0 0x7ff09c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.119+0000 7ff09be00640 1 -- 172.21.15.57:0/1477405893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff09c1053b0 msgr2=0x7ff09c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.119+0000 7ff09be00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff09c1053b0 0x7ff09c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.119+0000 7ff09be00640 1 -- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c107ce0 msgr2=0x7ff09c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.119+0000 7ff09be00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c107ce0 0x7ff09c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.119+0000 7ff09be00640 1 -- 172.21.15.57:0/1477405893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c19fe70 con 0x7ff09c102a80 2024-04-13T05:27:00.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.120+0000 7ff09be00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 0x7ff09c198250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff08c006010 tx=0x7ff08c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:00.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.121+0000 7ff098c00640 1 -- 172.21.15.57:0/1477405893 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c043020 con 0x7ff09c102a80 2024-04-13T05:27:00.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.121+0000 7ff098c00640 1 -- 172.21.15.57:0/1477405893 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff08c004aa0 con 0x7ff09c102a80 2024-04-13T05:27:00.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.121+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c100870 con 0x7ff09c102a80 2024-04-13T05:27:00.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.121+0000 7ff098c00640 1 -- 172.21.15.57:0/1477405893 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003c60 con 0x7ff09c102a80 2024-04-13T05:27:00.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.121+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c100e10 con 0x7ff09c102a80 2024-04-13T05:27:00.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.125+0000 7ff0a59ea740 1 Processor -- start 2024-04-13T05:27:00.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 -- start start 2024-04-13T05:27:00.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec44d0 0x55b675ec48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ec99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ecc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b675e2b020 con 0x55b675ec4ec0 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b675dd16d0 con 0x55b675eca030 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.126+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b675e26fe0 con 0x55b675ec44d0 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.127+0000 7ff098c00640 1 -- 172.21.15.57:0/1477405893 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff08c046020 con 0x7ff09c102a80 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.127+0000 7ff09b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ecc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.127+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec44d0 0x55b675ec48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.127+0000 7ff09b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ecc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:39108/0 (socket says 172.21.15.57:39108) 2024-04-13T05:27:00.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.127+0000 7ff09b400640 1 -- 172.21.15.57:0/2184838396 learned_addr learned my addr 172.21.15.57:0/2184838396 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.127+0000 7ff09be00640 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ec99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2184838396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 msgr2=0x55b675ecc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ecc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2184838396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 msgr2=0x55b675ec99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ec99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2184838396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b675c077d0 con 0x55b675ec44d0 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff09be00640 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ec99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.128+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec44d0 0x55b675ec48f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff094009eb0 tx=0x7ff094010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c424571f509ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.129+0000 7ff091200640 1 -- 172.21.15.57:0/2184838396 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094016070 con 0x55b675ec44d0 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.129+0000 7ff091200640 1 -- 172.21.15.57:0/2184838396 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff094004d20 con 0x55b675ec44d0 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.129+0000 7ff091200640 1 -- 172.21.15.57:0/2184838396 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0940126b0 con 0x55b675ec44d0 2024-04-13T05:27:00.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.129+0000 7ff098c00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff0800a3c30 0x7ff0800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.129+0000 7ff098c00640 1 -- 172.21.15.57:0/1477405893 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff08c0f2a90 con 0x7ff09c102a80 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff09be00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff0800a3c30 0x7ff0800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2184838396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec44d0 msgr2=0x55b675ec48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec44d0 0x55b675ec48f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff094009eb0 tx=0x7ff094010740 comp rx=0 tx=0).stop 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2184838396 shutdown_connections 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ecc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ec99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2184838396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec44d0 0x55b675ec48f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2184838396 >> 172.21.15.57:0/2184838396 conn(0x55b675ebfa40 msgr2=0x55b675ebfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2184838396 shutdown_connections 2024-04-13T05:27:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.130+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2184838396 wait complete. 2024-04-13T05:27:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.131+0000 7ff0a59ea740 1 -- start start 2024-04-13T05:27:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.131+0000 7ff0a59ea740 1 Processor -- start 2024-04-13T05:27:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.131+0000 7ff09be00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff0800a3c30 0x7ff0800a60f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0f0190 tx=0x7ff08c02bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ece830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675eced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ed5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b675eccfe0 con 0x55b675ec44d0 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b675de4a90 con 0x55b675eca030 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b675ecd160 con 0x55b675ec4ec0 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a59ea740 10 client.0 _ll_register_callbacks cb 0x55b675de4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ece830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ece830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46020/0 (socket says 172.21.15.57:46020) 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff0a0a00640 1 -- 172.21.15.57:0/1379648622 learned_addr learned my addr 172.21.15.57:0/1379648622 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.132+0000 7ff09b400640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675eced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ed5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 -- 172.21.15.57:0/1379648622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 msgr2=0x55b675eced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675eced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 -- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 msgr2=0x55b675ed5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ed5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 -- 172.21.15.57:0/1379648622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b675b3d3f0 con 0x55b675ec44d0 2024-04-13T05:27:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ece830 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff094005900 tx=0x7ff094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fed39623f1b027 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.133+0000 7ff085600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.134+0000 7ff085600640 1 -- 172.21.15.57:0/1379648622 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094016070 con 0x55b675ec44d0 2024-04-13T05:27:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.134+0000 7ff085600640 1 -- 172.21.15.57:0/1379648622 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff094022040 con 0x55b675ec44d0 2024-04-13T05:27:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.134+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff094004d20 con 0x55b675ec44d0 2024-04-13T05:27:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.134+0000 7ff085600640 1 -- 172.21.15.57:0/1379648622 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff094032050 con 0x55b675ec44d0 2024-04-13T05:27:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.134+0000 7ff0a59ea740 10 client.17379 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.134+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b675afd0f0 con 0x55b675ec44d0 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff085600640 1 -- 172.21.15.57:0/1379648622 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff094004d20 con 0x55b675ec44d0 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff085600640 1 -- 172.21.15.57:0/1379648622 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff094025070 con 0x55b675ec44d0 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff0a59ea740 10 client.17379 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff0a59ea740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff0a59ea740 20 client.17379 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675eda2c0 0x55b675edc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff0a59ea740 4 client.17379 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.135+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b675dca600 con 0x55b675eda2c0 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.136+0000 7ff09b400640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675eda2c0 0x55b675edc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.137+0000 7ff09b400640 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675eda2c0 0x55b675edc6b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8017a740a16ea914 server_cookie=51c432d408ba87c0 in_seq=0 out_seq=0 2024-04-13T05:27:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.137+0000 7ff085600640 10 client.17379 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.139+0000 7ff085600640 1 -- 172.21.15.57:0/1379648622 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b675dca600 con 0x55b675eda2c0 2024-04-13T05:27:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.139+0000 7ff085600640 10 client.17379 handle_command_reply: tid=0 2024-04-13T05:27:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.139+0000 7ff0a59ea740 1 client.17379 shutdown 2024-04-13T05:27:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 msgr2=0x55b675ece830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ece830 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff094005900 tx=0x7ff094005830 comp rx=0 tx=0).stop 2024-04-13T05:27:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 shutdown_connections 2024-04-13T05:27:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675eda2c0 0x55b675edc6b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ed5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675eced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/1379648622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ece830 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 >> 172.21.15.57:0/1379648622 conn(0x55b675ebfa40 msgr2=0x55b675ebeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff085600640 0 client.17379 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 shutdown_connections 2024-04-13T05:27:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.140+0000 7ff0a59ea740 1 -- 172.21.15.57:0/1379648622 wait complete. 2024-04-13T05:27:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.141+0000 7ff0a59ea740 20 client.17379 trim_cache size 0 max 16384 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 Processor -- start 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 -- start start 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ec3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675ed5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ee0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1ab280 con 0x55b675ec44d0 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1a8a90 con 0x55b675eca030 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1a68e0 con 0x55b675ec4ec0 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ec3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ec3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46022/0 (socket says 172.21.15.57:46022) 2024-04-13T05:27:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.168+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2165977083 learned_addr learned my addr 172.21.15.57:0/2165977083 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff09b400640 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675ed5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2165977083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 msgr2=0x55b675ed5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675ed5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2165977083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 msgr2=0x55b675ee0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ee0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff0a0a00640 1 -- 172.21.15.57:0/2165977083 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b675ecd2a0 con 0x55b675ec44d0 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ec3310 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff0940098b0 tx=0x7ff094009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c13c87b1d503930 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff087e00640 1 -- 172.21.15.57:0/2165977083 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094016070 con 0x55b675ec44d0 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff087e00640 1 -- 172.21.15.57:0/2165977083 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff094010c40 con 0x55b675ec44d0 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.169+0000 7ff087e00640 1 -- 172.21.15.57:0/2165977083 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094005930 con 0x55b675ec44d0 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.170+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2165977083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 msgr2=0x55b675ec3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.170+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ec3310 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff0940098b0 tx=0x7ff094009d30 comp rx=0 tx=0).stop 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.170+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2165977083 shutdown_connections 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.170+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675eca030 0x55b675ee0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.171+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675ec4ec0 0x55b675ed5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.171+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/2165977083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec44d0 0x55b675ec3310 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.171+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2165977083 >> 172.21.15.57:0/2165977083 conn(0x55b675ebfa40 msgr2=0x55b675ebeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.171+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2165977083 shutdown_connections 2024-04-13T05:27:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.171+0000 7ff0a59ea740 1 -- 172.21.15.57:0/2165977083 wait complete. 2024-04-13T05:27:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.171+0000 7ff0a59ea740 1 -- start start 2024-04-13T05:27:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 Processor -- start 2024-04-13T05:27:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675ec44d0 0x55b675ed3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ed3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675eca030 0x55b675ed4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1a6f50 con 0x55b675ec4ec0 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1a83c0 con 0x55b675ec44d0 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c1a7620 con 0x55b675eca030 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.172+0000 7ff0a59ea740 10 client.0 _ll_register_callbacks cb 0x7ff094004f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ed3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ed3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46032/0 (socket says 172.21.15.57:46032) 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 -- 172.21.15.57:0/811826072 learned_addr learned my addr 172.21.15.57:0/811826072 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675eca030 0x55b675ed4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675ec44d0 0x55b675ed3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 -- 172.21.15.57:0/811826072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675eca030 msgr2=0x55b675ed4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675eca030 0x55b675ed4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 -- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675ec44d0 msgr2=0x55b675ed3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675ec44d0 0x55b675ed3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.173+0000 7ff09b400640 1 -- 172.21.15.57:0/811826072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b675ed5440 con 0x55b675ec4ec0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.174+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675eca030 0x55b675ed4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.174+0000 7ff09b400640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ed3e40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x55b675ecfd10 tx=0x7ff0880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8e9dbfeff49326 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.174+0000 7ff085600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.174+0000 7ff085600640 1 -- 172.21.15.57:0/811826072 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08802a070 con 0x55b675ec4ec0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.174+0000 7ff085600640 1 -- 172.21.15.57:0/811826072 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff08800eb70 con 0x55b675ec4ec0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.174+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff094010c40 con 0x55b675ec4ec0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.175+0000 7ff085600640 1 -- 172.21.15.57:0/811826072 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff088035050 con 0x55b675ec4ec0 2024-04-13T05:27:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.175+0000 7ff0a59ea740 10 client.17385 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.175+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b675f40c70 con 0x55b675ec4ec0 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.175+0000 7ff085600640 1 -- 172.21.15.57:0/811826072 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff094010c40 con 0x55b675ec4ec0 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.175+0000 7ff085600640 1 -- 172.21.15.57:0/811826072 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff088024680 con 0x55b675ec4ec0 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a59ea740 10 client.17385 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a59ea740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a59ea740 20 client.17385 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675f416b0 0x55b675f43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a59ea740 4 client.17385 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b675e38a40 con 0x55b675f416b0 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.176+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675f416b0 0x55b675f43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.177+0000 7ff0a0a00640 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675f416b0 0x55b675f43ad0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324fef55f294e7c4 server_cookie=76395a6e85dc70f4 in_seq=0 out_seq=0 2024-04-13T05:27:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.177+0000 7ff085600640 10 client.17385 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff085600640 1 -- 172.21.15.57:0/811826072 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b675e38a40 con 0x55b675f416b0 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff085600640 10 client.17385 handle_command_reply: tid=0 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 client.17385 shutdown 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 msgr2=0x55b675ed3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ed3e40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x55b675ecfd10 tx=0x7ff0880046c0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 shutdown_connections 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55b675f416b0 0x55b675f43ad0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b675eca030 0x55b675ed4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b675ec4ec0 0x55b675ed3e40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 --2- 172.21.15.57:0/811826072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b675ec44d0 0x55b675ed3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff085600640 0 client.17385 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 >> 172.21.15.57:0/811826072 conn(0x55b675ebfa40 msgr2=0x55b675ecddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 shutdown_connections 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.179+0000 7ff0a59ea740 1 -- 172.21.15.57:0/811826072 wait complete. 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.180+0000 7ff0a59ea740 20 client.17385 trim_cache size 0 max 16384 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (267 inodes in the stack)", 2024-04-13T05:27:00.184 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout: "ad3214a3-f869-4254-b082-d3ef3496b4ad": { 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "ad3214a3-f869-4254-b082-d3ef3496b4ad", 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:27:00.185 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.180+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff0800a3c30 msgr2=0x7ff0800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.180+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff0800a3c30 0x7ff0800a60f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0f0190 tx=0x7ff08c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.180+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 msgr2=0x7ff09c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.180+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 0x7ff09c198250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff08c006010 tx=0x7ff08c0042f0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 shutdown_connections 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff0800a3c30 0x7ff0800a60f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff09c1053b0 0x7ff09c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff09c107ce0 0x7ff09c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 --2- 172.21.15.57:0/1477405893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09c102a80 0x7ff09c198250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 >> 172.21.15.57:0/1477405893 conn(0x7ff09c0fc470 msgr2=0x7ff09c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 shutdown_connections 2024-04-13T05:27:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:00.181+0000 7ff0a2a00640 1 -- 172.21.15.57:0/1477405893 wait complete. 2024-04-13T05:27:00.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:00 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:27:00.204 INFO:tasks.cephfs.filesystem:scrub status for tag:ad3214a3-f869-4254-b082-d3ef3496b4ad - {'path': '/', 'tag': 'ad3214a3-f869-4254-b082-d3ef3496b4ad', 'options': 'recursive,force'} 2024-04-13T05:27:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:00 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:27:00.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:00 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:27:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: pgmap v738: 129 pgs: 129 active+clean; 353 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 137 op/s 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:01 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:27:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:27:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:27:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: pgmap v738: 129 pgs: 129 active+clean; 353 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 137 op/s 2024-04-13T05:27:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:01 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: pgmap v738: 129 pgs: 129 active+clean; 353 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 137 op/s 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:01 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:03.503 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:03 smithi083 ceph-mon[35133]: pgmap v739: 129 pgs: 129 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 243 KiB/s wr, 142 op/s 2024-04-13T05:27:03.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:03 smithi057 ceph-mon[32331]: pgmap v739: 129 pgs: 129 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 243 KiB/s wr, 142 op/s 2024-04-13T05:27:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:03 smithi189 ceph-mon[35531]: pgmap v739: 129 pgs: 129 active+clean; 351 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 243 KiB/s wr, 142 op/s 2024-04-13T05:27:05.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:05 smithi083 ceph-mon[35133]: pgmap v740: 129 pgs: 129 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 391 KiB/s wr, 203 op/s 2024-04-13T05:27:05.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:05 smithi057 ceph-mon[32331]: pgmap v740: 129 pgs: 129 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 391 KiB/s wr, 203 op/s 2024-04-13T05:27:05.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:05 smithi189 ceph-mon[35531]: pgmap v740: 129 pgs: 129 active+clean; 348 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 391 KiB/s wr, 203 op/s 2024-04-13T05:27:07.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:07 smithi083 ceph-mon[35133]: pgmap v741: 129 pgs: 129 active+clean; 344 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 273 KiB/s wr, 212 op/s 2024-04-13T05:27:07.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:07 smithi057 ceph-mon[32331]: pgmap v741: 129 pgs: 129 active+clean; 344 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 273 KiB/s wr, 212 op/s 2024-04-13T05:27:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:07 smithi189 ceph-mon[35531]: pgmap v741: 129 pgs: 129 active+clean; 344 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 273 KiB/s wr, 212 op/s 2024-04-13T05:27:08.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:08 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:27:08.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:08 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:27:08.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:08 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:27:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:09 smithi083 ceph-mon[35133]: pgmap v742: 129 pgs: 129 active+clean; 344 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 267 KiB/s wr, 187 op/s 2024-04-13T05:27:09.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:09 smithi057 ceph-mon[32331]: pgmap v742: 129 pgs: 129 active+clean; 344 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 267 KiB/s wr, 187 op/s 2024-04-13T05:27:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:09 smithi189 ceph-mon[35531]: pgmap v742: 129 pgs: 129 active+clean; 344 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 267 KiB/s wr, 187 op/s 2024-04-13T05:27:10.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:27:10.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:27:10.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:27:10.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:27:10.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:27:10.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:27:10.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:27:10.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:27:10.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:27:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:11 smithi189 ceph-mon[35531]: pgmap v743: 129 pgs: 129 active+clean; 340 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 454 KiB/s wr, 232 op/s 2024-04-13T05:27:11.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:11 smithi083 ceph-mon[35133]: pgmap v743: 129 pgs: 129 active+clean; 340 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 454 KiB/s wr, 232 op/s 2024-04-13T05:27:11.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:11 smithi057 ceph-mon[32331]: pgmap v743: 129 pgs: 129 active+clean; 340 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 454 KiB/s wr, 232 op/s 2024-04-13T05:27:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:13 smithi189 ceph-mon[35531]: pgmap v744: 129 pgs: 129 active+clean; 338 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 343 KiB/s wr, 194 op/s 2024-04-13T05:27:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:13 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:27:13.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:13 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:27:13.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:13 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:27:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:13 smithi083 ceph-mon[35133]: pgmap v744: 129 pgs: 129 active+clean; 338 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 343 KiB/s wr, 194 op/s 2024-04-13T05:27:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:13 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:27:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:13 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:27:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:13 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:27:13.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:13 smithi057 ceph-mon[32331]: pgmap v744: 129 pgs: 129 active+clean; 338 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 343 KiB/s wr, 194 op/s 2024-04-13T05:27:13.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:13 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:27:13.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:13 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:27:13.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:13 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:27:15.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:15 smithi189 ceph-mon[35531]: pgmap v745: 129 pgs: 129 active+clean; 334 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 459 KiB/s wr, 212 op/s 2024-04-13T05:27:15.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:15 smithi083 ceph-mon[35133]: pgmap v745: 129 pgs: 129 active+clean; 334 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 459 KiB/s wr, 212 op/s 2024-04-13T05:27:15.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:15 smithi057 ceph-mon[32331]: pgmap v745: 129 pgs: 129 active+clean; 334 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 459 KiB/s wr, 212 op/s 2024-04-13T05:27:17.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:17 smithi189 ceph-mon[35531]: pgmap v746: 129 pgs: 129 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 316 KiB/s wr, 176 op/s 2024-04-13T05:27:17.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:17 smithi083 ceph-mon[35133]: pgmap v746: 129 pgs: 129 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 316 KiB/s wr, 176 op/s 2024-04-13T05:27:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:17 smithi057 ceph-mon[32331]: pgmap v746: 129 pgs: 129 active+clean; 328 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 316 KiB/s wr, 176 op/s 2024-04-13T05:27:19.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:19 smithi083 ceph-mon[35133]: pgmap v747: 129 pgs: 129 active+clean; 327 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 311 KiB/s wr, 140 op/s 2024-04-13T05:27:19.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:19 smithi057 ceph-mon[32331]: pgmap v747: 129 pgs: 129 active+clean; 327 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 311 KiB/s wr, 140 op/s 2024-04-13T05:27:19.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:19 smithi189 ceph-mon[35531]: pgmap v747: 129 pgs: 129 active+clean; 327 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 311 KiB/s wr, 140 op/s 2024-04-13T05:27:21.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:21 smithi083 ceph-mon[35133]: pgmap v748: 129 pgs: 129 active+clean; 322 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 466 KiB/s wr, 193 op/s 2024-04-13T05:27:21.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:21 smithi057 ceph-mon[32331]: pgmap v748: 129 pgs: 129 active+clean; 322 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 466 KiB/s wr, 193 op/s 2024-04-13T05:27:21.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:21 smithi189 ceph-mon[35531]: pgmap v748: 129 pgs: 129 active+clean; 322 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 466 KiB/s wr, 193 op/s 2024-04-13T05:27:23.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:23 smithi083 ceph-mon[35133]: pgmap v749: 129 pgs: 129 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 280 KiB/s wr, 159 op/s 2024-04-13T05:27:23.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:23 smithi057 ceph-mon[32331]: pgmap v749: 129 pgs: 129 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 280 KiB/s wr, 159 op/s 2024-04-13T05:27:23.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:23 smithi189 ceph-mon[35531]: pgmap v749: 129 pgs: 129 active+clean; 320 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 280 KiB/s wr, 159 op/s 2024-04-13T05:27:25.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:25 smithi083 ceph-mon[35133]: pgmap v750: 129 pgs: 129 active+clean; 318 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 435 KiB/s wr, 186 op/s 2024-04-13T05:27:25.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:25 smithi057 ceph-mon[32331]: pgmap v750: 129 pgs: 129 active+clean; 318 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 435 KiB/s wr, 186 op/s 2024-04-13T05:27:25.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:25 smithi189 ceph-mon[35531]: pgmap v750: 129 pgs: 129 active+clean; 318 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 435 KiB/s wr, 186 op/s 2024-04-13T05:27:27.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:27 smithi083 ceph-mon[35133]: pgmap v751: 129 pgs: 129 active+clean; 314 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 324 KiB/s wr, 171 op/s 2024-04-13T05:27:27.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:27 smithi057 ceph-mon[32331]: pgmap v751: 129 pgs: 129 active+clean; 314 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 324 KiB/s wr, 171 op/s 2024-04-13T05:27:27.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:27 smithi189 ceph-mon[35531]: pgmap v751: 129 pgs: 129 active+clean; 314 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 324 KiB/s wr, 171 op/s 2024-04-13T05:27:29.637 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:29 smithi189 ceph-mon[35531]: pgmap v752: 129 pgs: 129 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 318 KiB/s wr, 140 op/s 2024-04-13T05:27:29.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:29 smithi083 ceph-mon[35133]: pgmap v752: 129 pgs: 129 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 318 KiB/s wr, 140 op/s 2024-04-13T05:27:29.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:29 smithi057 ceph-mon[32331]: pgmap v752: 129 pgs: 129 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 318 KiB/s wr, 140 op/s 2024-04-13T05:27:30.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:27:30.206 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:27:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 -- 172.21.15.57:0/110174287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c109810 msgr2=0x7f077c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 --2- 172.21.15.57:0/110174287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c109810 0x7f077c109c90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0764009900 tx=0x7f076402bc30 comp rx=0 tx=0).stop 2024-04-13T05:27:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 -- 172.21.15.57:0/110174287 shutdown_connections 2024-04-13T05:27:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 --2- 172.21.15.57:0/110174287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f077c10a1d0 0x7f077c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 --2- 172.21.15.57:0/110174287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c109810 0x7f077c109c90 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 --2- 172.21.15.57:0/110174287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c108620 0x7f077c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.445+0000 7f0780c00640 1 -- 172.21.15.57:0/110174287 >> 172.21.15.57:0/110174287 conn(0x7f077c0775c0 msgr2=0x7f077c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.446+0000 7f0780c00640 1 -- 172.21.15.57:0/110174287 shutdown_connections 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.446+0000 7f0780c00640 1 -- 172.21.15.57:0/110174287 wait complete. 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.447+0000 7f0780c00640 1 Processor -- start 2024-04-13T05:27:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.447+0000 7f0780c00640 1 -- start start 2024-04-13T05:27:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0780c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 0x7f077c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0780c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c109810 0x7f077c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0780c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f077c10a1d0 0x7f077c1a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0780c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1141b0 con 0x7f077c108620 2024-04-13T05:27:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0780c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f077c114030 con 0x7f077c109810 2024-04-13T05:27:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0780c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f077c114330 con 0x7f077c10a1d0 2024-04-13T05:27:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0779e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 0x7f077c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0779400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c109810 0x7f077c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0779e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 0x7f077c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42718/0 (socket says 172.21.15.57:42718) 2024-04-13T05:27:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.448+0000 7f0779e00640 1 -- 172.21.15.57:0/2250065834 learned_addr learned my addr 172.21.15.57:0/2250065834 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.449+0000 7f0779e00640 1 -- 172.21.15.57:0/2250065834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f077c10a1d0 msgr2=0x7f077c1a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.449+0000 7f0779e00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f077c10a1d0 0x7f077c1a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.449+0000 7f0779e00640 1 -- 172.21.15.57:0/2250065834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c109810 msgr2=0x7f077c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.449+0000 7f0779e00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c109810 0x7f077c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.449+0000 7f0779e00640 1 -- 172.21.15.57:0/2250065834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c1a6500 con 0x7f077c108620 2024-04-13T05:27:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.450+0000 7f0779e00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 0x7f077c19e8e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d770 tx=0x7f076c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.451+0000 7f0772a00640 1 -- 172.21.15.57:0/2250065834 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c017070 con 0x7f077c108620 2024-04-13T05:27:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.451+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c1a67f0 con 0x7f077c108620 2024-04-13T05:27:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.451+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c1a6de0 con 0x7f077c108620 2024-04-13T05:27:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.452+0000 7f0772a00640 1 -- 172.21.15.57:0/2250065834 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f076c009c30 con 0x7f077c108620 2024-04-13T05:27:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.452+0000 7f0772a00640 1 -- 172.21.15.57:0/2250065834 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c020060 con 0x7f077c108620 2024-04-13T05:27:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.457+0000 7f0783bd3740 1 Processor -- start 2024-04-13T05:27:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 -- start start 2024-04-13T05:27:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a64d0 0x55dbf20a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a6ec0 0x55dbf20ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55dbf200d020 con 0x55dbf20ac030 2024-04-13T05:27:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dbf1fb36d0 con 0x55dbf20a64d0 2024-04-13T05:27:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.458+0000 7f0783bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2008fe0 con 0x55dbf20a6ec0 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.459+0000 7f0772a00640 1 -- 172.21.15.57:0/2250065834 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f076c003bf0 con 0x7f077c108620 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.459+0000 7f0779400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.459+0000 7f0779400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42734/0 (socket says 172.21.15.57:42734) 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.459+0000 7f0779400640 1 -- 172.21.15.57:0/81929166 learned_addr learned my addr 172.21.15.57:0/81929166 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.459+0000 7f077a800640 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a64d0 0x55dbf20a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.459+0000 7f0779e00640 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a6ec0 0x55dbf20ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.460+0000 7f0779400640 1 -- 172.21.15.57:0/81929166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a6ec0 msgr2=0x55dbf20ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.460+0000 7f0779400640 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a6ec0 0x55dbf20ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.460+0000 7f0779400640 1 -- 172.21.15.57:0/81929166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a64d0 msgr2=0x55dbf20a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.460+0000 7f0779400640 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a64d0 0x55dbf20a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.460+0000 7f0779400640 1 -- 172.21.15.57:0/81929166 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf1de97d0 con 0x55dbf20ac030 2024-04-13T05:27:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.461+0000 7f0779400640 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20ae4c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0764009900 tx=0x7f0764005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211707b2d9993656 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.461+0000 7f076b000640 1 -- 172.21.15.57:0/81929166 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764045020 con 0x55dbf20ac030 2024-04-13T05:27:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.461+0000 7f076b000640 1 -- 172.21.15.57:0/81929166 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f076402caa0 con 0x55dbf20ac030 2024-04-13T05:27:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.461+0000 7f076b000640 1 -- 172.21.15.57:0/81929166 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764004920 con 0x55dbf20ac030 2024-04-13T05:27:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.462+0000 7f0772a00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f07600a3bb0 0x7f07600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.462+0000 7f0772a00640 1 -- 172.21.15.57:0/2250065834 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f076c0d0100 con 0x7f077c108620 2024-04-13T05:27:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.462+0000 7f0783bd3740 1 -- 172.21.15.57:0/81929166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 msgr2=0x55dbf20ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.462+0000 7f0783bd3740 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20ae4c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0764009900 tx=0x7f0764005e00 comp rx=0 tx=0).stop 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f077a800640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f07600a3bb0 0x7f07600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 -- 172.21.15.57:0/81929166 shutdown_connections 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20ae4c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a6ec0 0x55dbf20ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 --2- 172.21.15.57:0/81929166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a64d0 0x55dbf20a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 -- 172.21.15.57:0/81929166 >> 172.21.15.57:0/81929166 conn(0x55dbf20a1a40 msgr2=0x55dbf20a1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 -- 172.21.15.57:0/81929166 shutdown_connections 2024-04-13T05:27:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.463+0000 7f0783bd3740 1 -- 172.21.15.57:0/81929166 wait complete. 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.464+0000 7f0783bd3740 1 -- start start 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.464+0000 7f077a800640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f07600a3bb0 0x7f07600a6070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55dbf20a7720 tx=0x7f077400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.464+0000 7f0783bd3740 1 Processor -- start 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.464+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0783bd3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55dbf20aef80 con 0x55dbf20ac030 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0783bd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dbf1fc6a90 con 0x55dbf20a6ec0 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0783bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dbf20af100 con 0x55dbf20a64d0 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0783bd3740 10 client.0 _ll_register_callbacks cb 0x55dbf1fc6480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f077a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0779400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0779e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0779400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:56058/0 (socket says 172.21.15.57:56058) 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0779400640 1 -- 172.21.15.57:0/106796890 learned_addr learned my addr 172.21.15.57:0/106796890 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f0779e00640 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:46240/0 (socket says 172.21.15.57:46240) 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f077a800640 1 -- 172.21.15.57:0/106796890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 msgr2=0x55dbf20b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f077a800640 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f077a800640 1 -- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 msgr2=0x55dbf20b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f077a800640 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.465+0000 7f077a800640 1 -- 172.21.15.57:0/106796890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf1d1f3f0 con 0x55dbf20ac030 2024-04-13T05:27:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f077a800640 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b7cf0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0774016de0 tx=0x7f0774004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c347f2d444120614 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f075f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f075f400640 1 -- 172.21.15.57:0/106796890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774025060 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f075f400640 1 -- 172.21.15.57:0/106796890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f07740205f0 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f076402caa0 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f075f400640 1 -- 172.21.15.57:0/106796890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f077402c050 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f0783bd3740 10 client.17397 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.466+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dbf1cdf0f0 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f075f400640 1 -- 172.21.15.57:0/106796890 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f076402caa0 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f075f400640 1 -- 172.21.15.57:0/106796890 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f077402e2e0 con 0x55dbf20ac030 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0783bd3740 10 client.17397 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0783bd3740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0783bd3740 20 client.17397 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0783bd3740 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf20bc750 0x55dbf20beb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0783bd3740 4 client.17397 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf1fac600 con 0x55dbf20bc750 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.467+0000 7f0779e00640 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf20bc750 0x55dbf20beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.468+0000 7f0779e00640 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf20bc750 0x55dbf20beb90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896c55bb2931d18f server_cookie=1f7d3f423193b2ad in_seq=0 out_seq=0 2024-04-13T05:27:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.468+0000 7f075f400640 10 client.17397 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f075f400640 1 -- 172.21.15.57:0/106796890 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f076c07ae50 con 0x55dbf20bc750 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f075f400640 10 client.17397 handle_command_reply: tid=0 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 client.17397 shutdown 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 msgr2=0x55dbf20b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b7cf0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0774016de0 tx=0x7f0774004bc0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 shutdown_connections 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf20bc750 0x55dbf20beb90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b7cf0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 --2- 172.21.15.57:0/106796890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 >> 172.21.15.57:0/106796890 conn(0x55dbf20a1a40 msgr2=0x55dbf20a0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f075f400640 0 client.17397 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.469+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 shutdown_connections 2024-04-13T05:27:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.470+0000 7f0783bd3740 1 -- 172.21.15.57:0/106796890 wait complete. 2024-04-13T05:27:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.470+0000 7f0783bd3740 20 client.17397 trim_cache size 0 max 16384 2024-04-13T05:27:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.489+0000 7f0783bd3740 1 Processor -- start 2024-04-13T05:27:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 -- start start 2024-04-13T05:27:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20ac030 0x55dbf20afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0758012550 con 0x55dbf20a64d0 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f075800fd60 con 0x55dbf20a6ec0 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0783bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f075800dbb0 con 0x55dbf20ac030 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0779e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0779e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42746/0 (socket says 172.21.15.57:42746) 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0779e00640 1 -- 172.21.15.57:0/1454769051 learned_addr learned my addr 172.21.15.57:0/1454769051 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f0779400640 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.490+0000 7f077a800640 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20ac030 0x55dbf20afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f0779e00640 1 -- 172.21.15.57:0/1454769051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20ac030 msgr2=0x55dbf20afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f0779e00640 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20ac030 0x55dbf20afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f0779e00640 1 -- 172.21.15.57:0/1454769051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 msgr2=0x55dbf20b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f0779e00640 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f0779e00640 1 -- 172.21.15.57:0/1454769051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf20a62c0 con 0x55dbf20a64d0 2024-04-13T05:27:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f0779e00640 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f076c000c00 tx=0x7f076c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1435448c997a0c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f076a600640 1 -- 172.21.15.57:0/1454769051 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0b4070 con 0x55dbf20a64d0 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f076a600640 1 -- 172.21.15.57:0/1454769051 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55dbf1fac600 con 0x55dbf20a64d0 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.491+0000 7f076a600640 1 -- 172.21.15.57:0/1454769051 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c098650 con 0x55dbf20a64d0 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 -- 172.21.15.57:0/1454769051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 msgr2=0x55dbf20b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f076c000c00 tx=0x7f076c0046c0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 -- 172.21.15.57:0/1454769051 shutdown_connections 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20ac030 0x55dbf20afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 --2- 172.21.15.57:0/1454769051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20a64d0 0x55dbf20b0770 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 -- 172.21.15.57:0/1454769051 >> 172.21.15.57:0/1454769051 conn(0x55dbf20a1a40 msgr2=0x55dbf20a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 -- 172.21.15.57:0/1454769051 shutdown_connections 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.492+0000 7f0783bd3740 1 -- 172.21.15.57:0/1454769051 wait complete. 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 -- start start 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 Processor -- start 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f075800e220 con 0x55dbf20ac030 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f075800f690 con 0x55dbf20a6ec0 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f075800e8f0 con 0x55dbf20a64d0 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.493+0000 7f0783bd3740 10 client.0 _ll_register_callbacks cb 0x7f0774016a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f0779400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42760/0 (socket says 172.21.15.57:42760) 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f0779400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:56084/0 (socket says 172.21.15.57:56084) 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 -- 172.21.15.57:0/356778407 learned_addr learned my addr 172.21.15.57:0/356778407 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f0779e00640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 -- 172.21.15.57:0/356778407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 msgr2=0x55dbf20b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 -- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 msgr2=0x55dbf20b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f077a800640 1 -- 172.21.15.57:0/356778407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf20b6b50 con 0x55dbf20ac030 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.494+0000 7f0779400640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f077a800640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b6560 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0774004b90 tx=0x7f0774009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdab5c96241673e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f075f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f075f400640 1 -- 172.21.15.57:0/356778407 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774035020 con 0x55dbf20ac030 2024-04-13T05:27:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f075f400640 1 -- 172.21.15.57:0/356778407 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f0774015f10 con 0x55dbf20ac030 2024-04-13T05:27:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dbf1fac600 con 0x55dbf20ac030 2024-04-13T05:27:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f075f400640 1 -- 172.21.15.57:0/356778407 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f077402eed0 con 0x55dbf20ac030 2024-04-13T05:27:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f0783bd3740 10 client.17403 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dbf21233e0 con 0x55dbf20ac030 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.495+0000 7f075f400640 1 -- 172.21.15.57:0/356778407 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55dbf1fac600 con 0x55dbf20ac030 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f075f400640 1 -- 172.21.15.57:0/356778407 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f0774030be0 con 0x55dbf20ac030 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0783bd3740 10 client.17403 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0783bd3740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0783bd3740 20 client.17403 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0783bd3740 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf2123ec0 0x55dbf21262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0783bd3740 4 client.17403 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbf201aa40 con 0x55dbf2123ec0 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.496+0000 7f0779e00640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf2123ec0 0x55dbf21262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f0779e00640 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf2123ec0 0x55dbf21262e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34cec0609035b6dc server_cookie=885bc2cedf389440 in_seq=0 out_seq=0 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f075f400640 10 client.17403 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f075f400640 1 -- 172.21.15.57:0/356778407 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dbf201aa40 con 0x55dbf2123ec0 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f075f400640 10 client.17403 handle_command_reply: tid=0 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f0783bd3740 1 client.17403 shutdown 2024-04-13T05:27:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 msgr2=0x55dbf20b6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.497+0000 7f0783bd3740 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b6560 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0774004b90 tx=0x7f0774009c90 comp rx=0 tx=0).stop 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 shutdown_connections 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55dbf2123ec0 0x55dbf21262e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbf20a6ec0 0x55dbf20b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dbf20ac030 0x55dbf20b6560 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 --2- 172.21.15.57:0/356778407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbf20a64d0 0x55dbf20b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 >> 172.21.15.57:0/356778407 conn(0x55dbf20a1a40 msgr2=0x55dbf20b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f075f400640 0 client.17403 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 shutdown_connections 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 1 -- 172.21.15.57:0/356778407 wait complete. 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.498+0000 7f0783bd3740 20 client.17403 trim_cache size 0 max 16384 2024-04-13T05:27:30.504 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:27:30.505 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:27:30.505 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:27:30.505 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:27:30.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f07600a3bb0 msgr2=0x7f07600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f07600a3bb0 0x7f07600a6070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55dbf20a7720 tx=0x7f077400d040 comp rx=0 tx=0).stop 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 msgr2=0x7f077c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 0x7f077c19e8e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d770 tx=0x7f076c00dc30 comp rx=0 tx=0).stop 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 shutdown_connections 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f07600a3bb0 0x7f07600a6070 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c109810 0x7f077c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f077c10a1d0 0x7f077c1a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 --2- 172.21.15.57:0/2250065834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f077c108620 0x7f077c19e8e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 >> 172.21.15.57:0/2250065834 conn(0x7f077c0775c0 msgr2=0x7f077c10cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 shutdown_connections 2024-04-13T05:27:30.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.499+0000 7f0780c00640 1 -- 172.21.15.57:0/2250065834 wait complete. 2024-04-13T05:27:30.519 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:27:30.520 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:27:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.795+0000 7f750fe00640 1 -- 172.21.15.57:0/1545774017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508100240 msgr2=0x7f7508100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.795+0000 7f750fe00640 1 --2- 172.21.15.57:0/1545774017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508100240 0x7f7508100660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009900 tx=0x7f74fc02bc40 comp rx=0 tx=0).stop 2024-04-13T05:27:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.796+0000 7f750fe00640 1 -- 172.21.15.57:0/1545774017 shutdown_connections 2024-04-13T05:27:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.796+0000 7f750fe00640 1 --2- 172.21.15.57:0/1545774017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7508068550 0x7f7508101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.796+0000 7f750fe00640 1 --2- 172.21.15.57:0/1545774017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7508101430 0x7f75081018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.796+0000 7f750fe00640 1 --2- 172.21.15.57:0/1545774017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508100240 0x7f7508100660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.796+0000 7f750fe00640 1 -- 172.21.15.57:0/1545774017 >> 172.21.15.57:0/1545774017 conn(0x7f75080fb7b0 msgr2=0x7f75080fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.796+0000 7f750fe00640 1 -- 172.21.15.57:0/1545774017 shutdown_connections 2024-04-13T05:27:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.797+0000 7f750fe00640 1 -- 172.21.15.57:0/1545774017 wait complete. 2024-04-13T05:27:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.798+0000 7f750fe00640 1 Processor -- start 2024-04-13T05:27:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.799+0000 7f750fe00640 1 -- start start 2024-04-13T05:27:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.800+0000 7f750fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7508068550 0x7f750810ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.800+0000 7f750fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7508100240 0x7f75081104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.800+0000 7f750fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 0x7f7508111510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f750810a620 con 0x7f7508100240 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f750810a4a0 con 0x7f7508101430 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f750810a7a0 con 0x7f7508068550 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 0x7f7508111510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7508100240 0x7f75081104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 0x7f7508111510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:56104/0 (socket says 172.21.15.57:56104) 2024-04-13T05:27:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.801+0000 7f750de00640 1 -- 172.21.15.57:0/4008490357 learned_addr learned my addr 172.21.15.57:0/4008490357 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.802+0000 7f750d400640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7508068550 0x7f750810ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.802+0000 7f750de00640 1 -- 172.21.15.57:0/4008490357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7508068550 msgr2=0x7f750810ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.802+0000 7f750de00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7508068550 0x7f750810ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.802+0000 7f750de00640 1 -- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7508100240 msgr2=0x7f75081104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.802+0000 7f750de00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7508100240 0x7f75081104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.802+0000 7f750de00640 1 -- 172.21.15.57:0/4008490357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7508111b80 con 0x7f7508101430 2024-04-13T05:27:30.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.803+0000 7f750de00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 0x7f7508111510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f750000eb10 tx=0x7f750000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.804+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500004b20 con 0x7f7508101430 2024-04-13T05:27:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.804+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7500004cc0 con 0x7f7508101430 2024-04-13T05:27:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.804+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500010690 con 0x7f7508101430 2024-04-13T05:27:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.804+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7508111e70 con 0x7f7508101430 2024-04-13T05:27:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.805+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7508112370 con 0x7f7508101430 2024-04-13T05:27:30.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.807+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7508105480 con 0x7f7508101430 2024-04-13T05:27:30.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.814+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7500005000 con 0x7f7508101430 2024-04-13T05:27:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.817+0000 7f7506000640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74f00a3d60 0x7f74f00a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.817+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f75000d12b0 con 0x7f7508101430 2024-04-13T05:27:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.818+0000 7f750d400640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74f00a3d60 0x7f74f00a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.820+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f750009d910 con 0x7f7508101430 2024-04-13T05:27:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:30.821+0000 7f750d400640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74f00a3d60 0x7f74f00a6220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74fc02c250 tx=0x7f74fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.078+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f750802d5e0 con 0x7f7508101430 2024-04-13T05:27:31.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.080+0000 7f7506000640 1 -- 172.21.15.57:0/4008490357 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11864 (secure 0 0 0) 0x7f750009d210 con 0x7f7508101430 2024-04-13T05:27:31.083 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:27:31.083 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:53.740918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:27:31.083 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74f00a3d60 msgr2=0x7f74f00a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74f00a3d60 0x7f74f00a6220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74fc02c250 tx=0x7f74fc005fd0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 msgr2=0x7f7508111510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 0x7f7508111510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f750000eb10 tx=0x7f750000c2d0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 shutdown_connections 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f74f00a3d60 0x7f74f00a6220 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7508101430 0x7f7508111510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7508100240 0x7f75081104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 --2- 172.21.15.57:0/4008490357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7508068550 0x7f750810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.082+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 >> 172.21.15.57:0/4008490357 conn(0x7f75080fb7b0 msgr2=0x7f7508102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.083+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 shutdown_connections 2024-04-13T05:27:31.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.083+0000 7f750fe00640 1 -- 172.21.15.57:0/4008490357 wait complete. 2024-04-13T05:27:31.102 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:27:31.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 -- 172.21.15.57:0/2520109363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a010a0b0 msgr2=0x7ff2a0110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 --2- 172.21.15.57:0/2520109363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a010a0b0 0x7ff2a0110940 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff298009ca0 tx=0x7ff29802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 -- 172.21.15.57:0/2520109363 shutdown_connections 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 --2- 172.21.15.57:0/2520109363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a01096f0 0x7ff2a0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 --2- 172.21.15.57:0/2520109363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a010a0b0 0x7ff2a0110940 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 --2- 172.21.15.57:0/2520109363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0108500 0x7ff2a0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 -- 172.21.15.57:0/2520109363 >> 172.21.15.57:0/2520109363 conn(0x7ff2a0077490 msgr2=0x7ff2a00798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 -- 172.21.15.57:0/2520109363 shutdown_connections 2024-04-13T05:27:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.315+0000 7ff29f200640 1 -- 172.21.15.57:0/2520109363 wait complete. 2024-04-13T05:27:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 Processor -- start 2024-04-13T05:27:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 -- start start 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 0x7ff2a019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a01096f0 0x7ff2a019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a010a0b0 0x7ff2a01a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0114040 con 0x7ff2a0108500 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0113ec0 con 0x7ff2a010a0b0 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.316+0000 7ff29f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01141c0 con 0x7ff2a01096f0 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 0x7ff2a019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 0x7ff2a019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42796/0 (socket says 172.21.15.57:42796) 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 -- 172.21.15.57:0/1899828125 learned_addr learned my addr 172.21.15.57:0/1899828125 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29d400640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a01096f0 0x7ff2a019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29e800640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a010a0b0 0x7ff2a01a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 -- 172.21.15.57:0/1899828125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a01096f0 msgr2=0x7ff2a019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a01096f0 0x7ff2a019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 -- 172.21.15.57:0/1899828125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a010a0b0 msgr2=0x7ff2a01a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a010a0b0 0x7ff2a01a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 -- 172.21.15.57:0/1899828125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a01a62b0 con 0x7ff2a0108500 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29d400640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a01096f0 0x7ff2a019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29e800640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a010a0b0 0x7ff2a01a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.317+0000 7ff29de00640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 0x7ff2a019e690 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c970 tx=0x7ff29000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.318+0000 7ff296a00640 1 -- 172.21.15.57:0/1899828125 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290004220 con 0x7ff2a0108500 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.318+0000 7ff296a00640 1 -- 172.21.15.57:0/1899828125 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2900043c0 con 0x7ff2a0108500 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.318+0000 7ff296a00640 1 -- 172.21.15.57:0/1899828125 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290004a00 con 0x7ff2a0108500 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.318+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a01a65a0 con 0x7ff2a0108500 2024-04-13T05:27:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.318+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a01a6af0 con 0x7ff2a0108500 2024-04-13T05:27:31.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 Processor -- start 2024-04-13T05:27:31.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 -- start start 2024-04-13T05:27:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886274860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886279930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x55588627c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5558861da020 con 0x555886274e30 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555886180950 con 0x555886274440 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558861d5fe0 con 0x555886279fa0 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.322+0000 7ff296a00640 1 -- 172.21.15.57:0/1899828125 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff2900079b0 con 0x7ff2a0108500 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886279930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886279930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42810/0 (socket says 172.21.15.57:42810) 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886274860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 -- 172.21.15.57:0/2118092623 learned_addr learned my addr 172.21.15.57:0/2118092623 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29d400640 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x55588627c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 -- 172.21.15.57:0/2118092623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 msgr2=0x55588627c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x55588627c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 -- 172.21.15.57:0/2118092623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 msgr2=0x555886274860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886274860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.323+0000 7ff29de00640 1 -- 172.21.15.57:0/2118092623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555885eec3f0 con 0x555886274e30 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff29de00640 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886279930 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff2900d67c0 tx=0x7ff2900d6c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03f20599fcfd7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff28b000640 1 -- 172.21.15.57:0/2118092623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900e3070 con 0x555886274e30 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff28b000640 1 -- 172.21.15.57:0/2118092623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2900d0e20 con 0x555886274e30 2024-04-13T05:27:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff28b000640 1 -- 172.21.15.57:0/2118092623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900d8bf0 con 0x555886274e30 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff296a00640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2840a3c30 0x7ff2840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff296a00640 1 -- 172.21.15.57:0/1899828125 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff2900d0560 con 0x7ff2a0108500 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.324+0000 7ff29e800640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2840a3c30 0x7ff2840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2118092623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 msgr2=0x555886279930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886279930 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff2900d67c0 tx=0x7ff2900d6c80 comp rx=0 tx=0).stop 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2118092623 shutdown_connections 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x55588627c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886279930 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2118092623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886274860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2118092623 >> 172.21.15.57:0/2118092623 conn(0x55588626f9b0 msgr2=0x55588626fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2118092623 shutdown_connections 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff29e800640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2840a3c30 0x7ff2840a60f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x555886275690 tx=0x7ff298005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.325+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2118092623 wait complete. 2024-04-13T05:27:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 -- start start 2024-04-13T05:27:31.330 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 Processor -- start 2024-04-13T05:27:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x55588627e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x55588627eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x555886285f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55588627cf50 con 0x555886274e30 2024-04-13T05:27:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555886193c80 con 0x555886274440 2024-04-13T05:27:31.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55588627d0d0 con 0x555886279fa0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff2a79ef740 10 client.0 _ll_register_callbacks cb 0x555886193670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff29d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x55588627eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff29d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x55588627eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42812/0 (socket says 172.21.15.57:42812) 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.326+0000 7ff29d400640 1 -- 172.21.15.57:0/659720104 learned_addr learned my addr 172.21.15.57:0/659720104 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29e800640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x555886285f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29de00640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x55588627e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29d400640 1 -- 172.21.15.57:0/659720104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 msgr2=0x555886285f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29d400640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x555886285f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29d400640 1 -- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 msgr2=0x55588627e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29d400640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x55588627e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29d400640 1 -- 172.21.15.57:0/659720104 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555885fb67d0 con 0x555886274e30 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff29d400640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x55588627eef0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff28c010670 tx=0x7ff28c010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7c4388087f18e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff27f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:31.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.327+0000 7ff27f400640 1 -- 172.21.15.57:0/659720104 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004220 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff27f400640 1 -- 172.21.15.57:0/659720104 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff28c0043c0 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2900d0e20 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff27f400640 1 -- 172.21.15.57:0/659720104 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff28c023050 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 10 client.17421 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555886289370 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff27f400640 1 -- 172.21.15.57:0/659720104 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2900d0e20 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff27f400640 1 -- 172.21.15.57:0/659720104 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff28c00f040 con 0x555886274e30 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 10 client.17421 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 20 client.17421 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55588628aad0 0x55588628cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 4 client.17421 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.328+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555886179750 con 0x55588628aad0 2024-04-13T05:27:31.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.329+0000 7ff29de00640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55588628aad0 0x55588628cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.330+0000 7ff29de00640 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55588628aad0 0x55588628cf30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab2d0c280840b856 server_cookie=ae38c8d76b3314b1 in_seq=0 out_seq=0 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.330+0000 7ff27f400640 10 client.17421 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff27f400640 1 -- 172.21.15.57:0/659720104 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555886179750 con 0x55588628aad0 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff27f400640 10 client.17421 handle_command_reply: tid=0 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 client.17421 shutdown 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 msgr2=0x55588627eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x55588627eef0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff28c010670 tx=0x7ff28c010b30 comp rx=0 tx=0).stop 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 shutdown_connections 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55588628aad0 0x55588628cf30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886279fa0 0x555886285f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x55588627eef0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/659720104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x55588627e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff27f400640 0 client.17421 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.332+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 >> 172.21.15.57:0/659720104 conn(0x55588626f9b0 msgr2=0x55588626ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.333+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 shutdown_connections 2024-04-13T05:27:31.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.333+0000 7ff2a79ef740 1 -- 172.21.15.57:0/659720104 wait complete. 2024-04-13T05:27:31.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.333+0000 7ff2a79ef740 20 client.17421 trim_cache size 0 max 16384 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 Processor -- start 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 -- start start 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886276330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274e30 0x555886276870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 0x555886285ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01b3920 con 0x555886279fa0 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01b1130 con 0x555886274440 2024-04-13T05:27:31.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.347+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01aef80 con 0x555886274e30 2024-04-13T05:27:31.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 0x555886285ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 0x555886285ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42820/0 (socket says 172.21.15.57:42820) 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 -- 172.21.15.57:0/2117108581 learned_addr learned my addr 172.21.15.57:0/2117108581 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29d400640 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274e30 0x555886276870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29de00640 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886276330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 -- 172.21.15.57:0/2117108581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274e30 msgr2=0x555886276870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274e30 0x555886276870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 -- 172.21.15.57:0/2117108581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 msgr2=0x555886276330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886276330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.348+0000 7ff29e800640 1 -- 172.21.15.57:0/2117108581 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555886270720 con 0x555886279fa0 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.349+0000 7ff29e800640 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 0x555886285ef0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff298036000 tx=0x7ff29803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d591c5635699da server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.349+0000 7ff28a600640 1 -- 172.21.15.57:0/2117108581 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298039040 con 0x555886279fa0 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.349+0000 7ff28a600640 1 -- 172.21.15.57:0/2117108581 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff298046070 con 0x555886279fa0 2024-04-13T05:27:31.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.349+0000 7ff28a600640 1 -- 172.21.15.57:0/2117108581 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2980346a0 con 0x555886279fa0 2024-04-13T05:27:31.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.349+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2117108581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 msgr2=0x555886285ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.349+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 0x555886285ef0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff298036000 tx=0x7ff29803f000 comp rx=0 tx=0).stop 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2117108581 shutdown_connections 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886279fa0 0x555886285ef0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274e30 0x555886276870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2117108581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886274440 0x555886276330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2117108581 >> 172.21.15.57:0/2117108581 conn(0x55588626f9b0 msgr2=0x55588627d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2117108581 shutdown_connections 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2117108581 wait complete. 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 -- start start 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.350+0000 7ff2a79ef740 1 Processor -- start 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274440 0x555886283600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886283b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886279fa0 0x555886284b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01af5f0 con 0x555886274e30 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01b0a60 con 0x555886279fa0 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01afcc0 con 0x555886274440 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff2a79ef740 10 client.0 _ll_register_callbacks cb 0x7ff28c009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886283b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886283b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42824/0 (socket says 172.21.15.57:42824) 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 -- 172.21.15.57:0/2008827548 learned_addr learned my addr 172.21.15.57:0/2008827548 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29de00640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274440 0x555886283600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29e800640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886279fa0 0x555886284b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 -- 172.21.15.57:0/2008827548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274440 msgr2=0x555886283600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274440 0x555886283600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 -- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886279fa0 msgr2=0x555886284b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886279fa0 0x555886284b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29d400640 1 -- 172.21.15.57:0/2008827548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555886285140 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29de00640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274440 0x555886283600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.351+0000 7ff29e800640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886279fa0 0x555886284b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff29d400640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886283b40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x5558862708f0 tx=0x7ff28c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d00c581a74bb184 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff27f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff27f400640 1 -- 172.21.15.57:0/2008827548 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c005240 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff298046070 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff27f400640 1 -- 172.21.15.57:0/2008827548 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7ff28c0053e0 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff27f400640 1 -- 172.21.15.57:0/2008827548 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff28c035050 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff2a79ef740 10 client.17427 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558862f10b0 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.352+0000 7ff27f400640 1 -- 172.21.15.57:0/2008827548 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff298046070 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff27f400640 1 -- 172.21.15.57:0/2008827548 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff28c00e850 con 0x555886274e30 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff2a79ef740 10 client.17427 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:31.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff2a79ef740 10 client.17427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:27:31.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff2a79ef740 20 client.17427 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:31.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5558862f1b30 0x5558862f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff2a79ef740 4 client.17427 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:27:31.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558861e7a40 con 0x5558862f1b30 2024-04-13T05:27:31.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.353+0000 7ff29de00640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5558862f1b30 0x5558862f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.354+0000 7ff29de00640 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5558862f1b30 0x5558862f3f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92284bf75e87fd82 server_cookie=7da2d324231e7f2b in_seq=0 out_seq=0 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.354+0000 7ff27f400640 10 client.17427 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.355+0000 7ff27f400640 1 -- 172.21.15.57:0/2008827548 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558861e7a40 con 0x5558862f1b30 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.355+0000 7ff27f400640 10 client.17427 handle_command_reply: tid=0 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 client.17427 shutdown 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 msgr2=0x555886283b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886283b40 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x5558862708f0 tx=0x7ff28c0038a0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 shutdown_connections 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x5558862f1b30 0x5558862f3f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555886279fa0 0x555886284b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555886274e30 0x555886283b40 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff27f400640 0 client.17427 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 --2- 172.21.15.57:0/2008827548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555886274440 0x555886283600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 >> 172.21.15.57:0/2008827548 conn(0x55588626f9b0 msgr2=0x55588627d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 shutdown_connections 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 1 -- 172.21.15.57:0/2008827548 wait complete. 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.356+0000 7ff2a79ef740 20 client.17427 trim_cache size 0 max 16384 2024-04-13T05:27:31.361 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2840a3c30 msgr2=0x7ff2840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2840a3c30 0x7ff2840a60f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x555886275690 tx=0x7ff298005d80 comp rx=0 tx=0).stop 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 msgr2=0x7ff2a019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 0x7ff2a019e690 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c970 tx=0x7ff29000ce30 comp rx=0 tx=0).stop 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 shutdown_connections 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff2840a3c30 0x7ff2840a60f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2a010a0b0 0x7ff2a01a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a01096f0 0x7ff2a019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 --2- 172.21.15.57:0/1899828125 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2a0108500 0x7ff2a019e690 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 >> 172.21.15.57:0/1899828125 conn(0x7ff2a0077490 msgr2=0x7ff2a010ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 shutdown_connections 2024-04-13T05:27:31.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.357+0000 7ff29f200640 1 -- 172.21.15.57:0/1899828125 wait complete. 2024-04-13T05:27:31.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:27:31.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.596+0000 7fb50ec00640 1 -- 172.21.15.57:0/3332829756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 msgr2=0x7fb508105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.596+0000 7fb50ec00640 1 --2- 172.21.15.57:0/3332829756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb508105770 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402bc30 comp rx=0 tx=0).stop 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 -- 172.21.15.57:0/3332829756 shutdown_connections 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 --2- 172.21.15.57:0/3332829756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb508105cb0 0x7fb50810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 --2- 172.21.15.57:0/3332829756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb508105770 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 --2- 172.21.15.57:0/3332829756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 0x7fb508104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 -- 172.21.15.57:0/3332829756 >> 172.21.15.57:0/3332829756 conn(0x7fb5080ff670 msgr2=0x7fb508101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 -- 172.21.15.57:0/3332829756 shutdown_connections 2024-04-13T05:27:31.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.597+0000 7fb50ec00640 1 -- 172.21.15.57:0/3332829756 wait complete. 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 Processor -- start 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 -- start start 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 0x7fb50819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb50819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb508105cb0 0x7fb5081a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb50810fc40 con 0x7fb5081052f0 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb50810fac0 con 0x7fb508104100 2024-04-13T05:27:31.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.598+0000 7fb50ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb50810fdc0 con 0x7fb508105cb0 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb50819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb50819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42840/0 (socket says 172.21.15.57:42840) 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 -- 172.21.15.57:0/2822516623 learned_addr learned my addr 172.21.15.57:0/2822516623 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50d800640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 0x7fb50819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50e200640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb508105cb0 0x7fb5081a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 -- 172.21.15.57:0/2822516623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb508105cb0 msgr2=0x7fb5081a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb508105cb0 0x7fb5081a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 -- 172.21.15.57:0/2822516623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 msgr2=0x7fb50819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 0x7fb50819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50ce00640 1 -- 172.21.15.57:0/2822516623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5081a1f10 con 0x7fb5081052f0 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.599+0000 7fb50d800640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 0x7fb50819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:27:31.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.600+0000 7fb50ce00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb50819a830 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.600+0000 7fb506000640 1 -- 172.21.15.57:0/2822516623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4043020 con 0x7fb5081052f0 2024-04-13T05:27:31.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.600+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5081a21a0 con 0x7fb5081052f0 2024-04-13T05:27:31.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.600+0000 7fb506000640 1 -- 172.21.15.57:0/2822516623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4f4004aa0 con 0x7fb5081052f0 2024-04-13T05:27:31.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.600+0000 7fb506000640 1 -- 172.21.15.57:0/2822516623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4003c60 con 0x7fb5081052f0 2024-04-13T05:27:31.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.600+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081a26f0 con 0x7fb5081052f0 2024-04-13T05:27:31.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.603+0000 7fb51300a740 1 Processor -- start 2024-04-13T05:27:31.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.603+0000 7fb51300a740 1 -- start start 2024-04-13T05:27:31.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.603+0000 7fb506000640 1 -- 172.21.15.57:0/2822516623 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb4f4046020 con 0x7fb5081052f0 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.603+0000 7fb51300a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb51300a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa3e30 0x55edcbaa8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb51300a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbaab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb51300a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55edcba09020 con 0x55edcbaa3440 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb51300a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55edcb9af950 con 0x55edcbaa8fa0 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb51300a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55edcba04fe0 con 0x55edcbaa3e30 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42844/0 (socket says 172.21.15.57:42844) 2024-04-13T05:27:31.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 -- 172.21.15.57:0/1292615194 learned_addr learned my addr 172.21.15.57:0/1292615194 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50d800640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa3e30 0x55edcbaa8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50ce00640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbaab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 -- 172.21.15.57:0/1292615194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa3e30 msgr2=0x55edcbaa8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa3e30 0x55edcbaa8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 -- 172.21.15.57:0/1292615194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 msgr2=0x55edcbaab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbaab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50e200640 1 -- 172.21.15.57:0/1292615194 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edcb71b3f0 con 0x55edcbaa3440 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50d800640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa3e30 0x55edcbaa8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.604+0000 7fb50ce00640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbaab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb50e200640 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa3860 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb500008dc0 tx=0x7fb50000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88b9a594880b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb4faa00640 1 -- 172.21.15.57:0/1292615194 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500014070 con 0x55edcbaa3440 2024-04-13T05:27:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb506000640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4f00a3bb0 0x7fb4f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb4faa00640 1 -- 172.21.15.57:0/1292615194 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb500003b70 con 0x55edcbaa3440 2024-04-13T05:27:31.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb4faa00640 1 -- 172.21.15.57:0/1292615194 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5000106c0 con 0x55edcbaa3440 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb506000640 1 -- 172.21.15.57:0/2822516623 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb4f40f2c60 con 0x7fb5081052f0 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.605+0000 7fb50d800640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4f00a3bb0 0x7fb4f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 -- 172.21.15.57:0/1292615194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 msgr2=0x55edcbaa3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa3860 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb500008dc0 tx=0x7fb50000e720 comp rx=0 tx=0).stop 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 -- 172.21.15.57:0/1292615194 shutdown_connections 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbaab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa3e30 0x55edcbaa8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 --2- 172.21.15.57:0/1292615194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa3860 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 -- 172.21.15.57:0/1292615194 >> 172.21.15.57:0/1292615194 conn(0x55edcba9e9b0 msgr2=0x55edcba9edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb50d800640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4f00a3bb0 0x7fb4f00a6070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005fd0 tx=0x7fb4fc00b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 -- 172.21.15.57:0/1292615194 shutdown_connections 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.606+0000 7fb51300a740 1 -- 172.21.15.57:0/1292615194 wait complete. 2024-04-13T05:27:31.613 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 -- start start 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 Processor -- start 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3440 0x55edcbaad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbaadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 0x55edcbab4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55edcbaabf50 con 0x55edcbaa3e30 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55edcb9c2c80 con 0x55edcbaa3440 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55edcbaac0d0 con 0x55edcbaa8fa0 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb51300a740 10 client.0 _ll_register_callbacks cb 0x55edcb9c2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.607+0000 7fb50ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbaadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbaadeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42856/0 (socket says 172.21.15.57:42856) 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 -- 172.21.15.57:0/3471842838 learned_addr learned my addr 172.21.15.57:0/3471842838 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50d800640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3440 0x55edcbaad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 -- 172.21.15.57:0/3471842838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 msgr2=0x55edcbab4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 0x55edcbab4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 -- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3440 msgr2=0x55edcbaad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3440 0x55edcbaad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 -- 172.21.15.57:0/3471842838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edcb7e57d0 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50d800640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3440 0x55edcbaad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb50ce00640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbaadeb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40bafa0 tx=0x7fb4f40be4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2751e7d64750317d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb4ef400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb4ef400640 1 -- 172.21.15.57:0/3471842838 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40beb40 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb4ef400640 1 -- 172.21.15.57:0/3471842838 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb4f4003e00 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.608+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb500003b70 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb4ef400640 1 -- 172.21.15.57:0/3471842838 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4f40c2ef0 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 10 client.17439 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55edcbab8160 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb4ef400640 1 -- 172.21.15.57:0/3471842838 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb500003b70 con 0x55edcbaa3e30 2024-04-13T05:27:31.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb4ef400640 1 -- 172.21.15.57:0/3471842838 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb4f40beda0 con 0x55edcbaa3e30 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 10 client.17439 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 10 client.17439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 20 client.17439 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbab9890 0x55edcbabbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 4 client.17439 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.609+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edcb9a8750 con 0x55edcbab9890 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.610+0000 7fb50d800640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbab9890 0x55edcbabbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.611+0000 7fb50d800640 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbab9890 0x55edcbabbd50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d730843007ae3318 server_cookie=8090358d7518f12c in_seq=0 out_seq=0 2024-04-13T05:27:31.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.611+0000 7fb4ef400640 10 client.17439 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:27:31.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.615+0000 7fb4ef400640 1 -- 172.21.15.57:0/3471842838 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edcb9a8750 con 0x55edcbab9890 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.615+0000 7fb4ef400640 10 client.17439 handle_command_reply: tid=0 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 client.17439 shutdown 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 msgr2=0x55edcbaadeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbaadeb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40bafa0 tx=0x7fb4f40be4a0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 shutdown_connections 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbab9890 0x55edcbabbd50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 0x55edcbab4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbaadeb0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 --2- 172.21.15.57:0/3471842838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3440 0x55edcbaad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb4ef400640 0 client.17439 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 >> 172.21.15.57:0/3471842838 conn(0x55edcba9e9b0 msgr2=0x55edcba9dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 shutdown_connections 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 1 -- 172.21.15.57:0/3471842838 wait complete. 2024-04-13T05:27:31.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.616+0000 7fb51300a740 20 client.17439 trim_cache size 0 max 16384 2024-04-13T05:27:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.631+0000 7fb51300a740 1 Processor -- start 2024-04-13T05:27:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.631+0000 7fb51300a740 1 -- start start 2024-04-13T05:27:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb51300a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb51300a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3e30 0x55edcba99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb51300a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 0x55edcbaad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb51300a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081af4e0 con 0x55edcbaa3440 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb51300a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081accf0 con 0x55edcbaa3e30 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb51300a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081aab40 con 0x55edcbaa8fa0 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42868/0 (socket says 172.21.15.57:42868) 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 -- 172.21.15.57:0/4092781722 learned_addr learned my addr 172.21.15.57:0/4092781722 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 -- 172.21.15.57:0/4092781722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 msgr2=0x55edcbaad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 0x55edcbaad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 -- 172.21.15.57:0/4092781722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3e30 msgr2=0x55edcba99b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3e30 0x55edcba99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.632+0000 7fb50d800640 1 -- 172.21.15.57:0/4092781722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edcbaa4490 con 0x55edcbaa3440 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.633+0000 7fb50d800640 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa48d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x55edcbaa4230 tx=0x7fb4fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d88e68a378e847 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.633+0000 7fb4fa000640 1 -- 172.21.15.57:0/4092781722 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc012a20 con 0x55edcbaa3440 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.633+0000 7fb4fa000640 1 -- 172.21.15.57:0/4092781722 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc0054d0 con 0x55edcbaa3440 2024-04-13T05:27:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.633+0000 7fb4fa000640 1 -- 172.21.15.57:0/4092781722 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004bf0 con 0x55edcbaa3440 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.633+0000 7fb51300a740 1 -- 172.21.15.57:0/4092781722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 msgr2=0x55edcbaa48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.633+0000 7fb51300a740 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa48d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x55edcbaa4230 tx=0x7fb4fc016000 comp rx=0 tx=0).stop 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 -- 172.21.15.57:0/4092781722 shutdown_connections 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa3e30 0x55edcba99b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbaa8fa0 0x55edcbaad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 --2- 172.21.15.57:0/4092781722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3440 0x55edcbaa48d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 -- 172.21.15.57:0/4092781722 >> 172.21.15.57:0/4092781722 conn(0x55edcba9e9b0 msgr2=0x55edcba9ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 -- 172.21.15.57:0/4092781722 shutdown_connections 2024-04-13T05:27:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 -- 172.21.15.57:0/4092781722 wait complete. 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 -- start start 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.634+0000 7fb51300a740 1 Processor -- start 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbab1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbab1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbab2f90 0x55edcbab3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081ab1b0 con 0x55edcbaa3e30 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081ac620 con 0x55edcbaa8fa0 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081ab880 con 0x55edcbab2f90 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb51300a740 10 client.0 _ll_register_callbacks cb 0x7fb4f40be930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbab1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbab1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42882/0 (socket says 172.21.15.57:42882) 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 -- 172.21.15.57:0/3785281497 learned_addr learned my addr 172.21.15.57:0/3785281497 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 -- 172.21.15.57:0/3785281497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbab2f90 msgr2=0x55edcbab3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbab2f90 0x55edcbab3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 -- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 msgr2=0x55edcbab1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50ce00640 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbab1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbab1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.635+0000 7fb50d800640 1 -- 172.21.15.57:0/3785281497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edcbab3950 con 0x55edcbaa3e30 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb50d800640 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbab1a40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc029040 tx=0x7fb4fc012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58a322b31a29113 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb4ef400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb4ef400640 1 -- 172.21.15.57:0/3785281497 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc012c70 con 0x55edcbaa3e30 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb4ef400640 1 -- 172.21.15.57:0/3785281497 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fb4fc004020 con 0x55edcbaa3e30 2024-04-13T05:27:31.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc0054d0 con 0x55edcbaa3e30 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb4ef400640 1 -- 172.21.15.57:0/3785281497 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc03c050 con 0x55edcbaa3e30 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb51300a740 10 client.17445 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55edcbb20230 con 0x55edcbaa3e30 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.636+0000 7fb4ef400640 1 -- 172.21.15.57:0/3785281497 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc0054d0 con 0x55edcbaa3e30 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb4ef400640 1 -- 172.21.15.57:0/3785281497 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb4fc00f040 con 0x55edcbaa3e30 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb51300a740 10 client.17445 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb51300a740 10 client.17445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb51300a740 20 client.17445 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb51300a740 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbb20c60 0x55edcbb23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb51300a740 4 client.17445 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edcba16a40 con 0x55edcbb20c60 2024-04-13T05:27:31.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.637+0000 7fb50ce00640 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbb20c60 0x55edcbb23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.638+0000 7fb50ce00640 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbb20c60 0x55edcbb23080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5498d3502850627f server_cookie=79aaa16e7beffa7c in_seq=0 out_seq=0 2024-04-13T05:27:31.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.638+0000 7fb4ef400640 10 client.17445 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:27:31.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb4ef400640 1 -- 172.21.15.57:0/3785281497 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edcba16a40 con 0x55edcbb20c60 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb4ef400640 10 client.17445 handle_command_reply: tid=0 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 client.17445 shutdown 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 msgr2=0x55edcbab1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbab1a40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc029040 tx=0x7fb4fc012b80 comp rx=0 tx=0).stop 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 shutdown_connections 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x55edcbb20c60 0x55edcbb23080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edcbaa8fa0 0x55edcbab1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edcbab2f90 0x55edcbab3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 --2- 172.21.15.57:0/3785281497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55edcbaa3e30 0x55edcbab1a40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb4ef400640 0 client.17445 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 >> 172.21.15.57:0/3785281497 conn(0x55edcba9e9b0 msgr2=0x55edcba99b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 shutdown_connections 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 1 -- 172.21.15.57:0/3785281497 wait complete. 2024-04-13T05:27:31.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.639+0000 7fb51300a740 20 client.17445 trim_cache size 0 max 16384 2024-04-13T05:27:31.646 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:27:31.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4f00a3bb0 msgr2=0x7fb4f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4f00a3bb0 0x7fb4f00a6070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005fd0 tx=0x7fb4fc00b2e0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 msgr2=0x7fb50819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb50819a830 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f40042f0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 shutdown_connections 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fb4f00a3bb0 0x7fb4f00a6070 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb508105cb0 0x7fb5081a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5081052f0 0x7fb50819a830 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 --2- 172.21.15.57:0/2822516623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb508104100 0x7fb50819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 >> 172.21.15.57:0/2822516623 conn(0x7fb5080ff670 msgr2=0x7fb5081086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 shutdown_connections 2024-04-13T05:27:31.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.640+0000 7fb50ec00640 1 -- 172.21.15.57:0/2822516623 wait complete. 2024-04-13T05:27:31.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:31 smithi057 ceph-mon[32331]: pgmap v753: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 451 KiB/s wr, 175 op/s 2024-04-13T05:27:31.656 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:31 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4008490357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:27:31.658 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:27:31.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:31 smithi083 ceph-mon[35133]: pgmap v753: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 451 KiB/s wr, 175 op/s 2024-04-13T05:27:31.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:31 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4008490357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:27:31.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:31 smithi189 ceph-mon[35531]: pgmap v753: 129 pgs: 129 active+clean; 308 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 451 KiB/s wr, 175 op/s 2024-04-13T05:27:31.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:31 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4008490357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:27:31.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 -- 172.21.15.57:0/3544859124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 msgr2=0x7fac200743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 --2- 172.21.15.57:0/3544859124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac200743f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fac10009900 tx=0x7fac1002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 -- 172.21.15.57:0/3544859124 shutdown_connections 2024-04-13T05:27:31.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 --2- 172.21.15.57:0/3544859124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac20109d80 0x7fac2010c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 --2- 172.21.15.57:0/3544859124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac200749c0 0x7fac20074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 --2- 172.21.15.57:0/3544859124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac200743f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.857+0000 7fac1fe00640 1 -- 172.21.15.57:0/3544859124 >> 172.21.15.57:0/3544859124 conn(0x7fac200fd460 msgr2=0x7fac200ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.858+0000 7fac1fe00640 1 -- 172.21.15.57:0/3544859124 shutdown_connections 2024-04-13T05:27:31.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.858+0000 7fac1fe00640 1 -- 172.21.15.57:0/3544859124 wait complete. 2024-04-13T05:27:31.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.858+0000 7fac1fe00640 1 Processor -- start 2024-04-13T05:27:31.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 -- start start 2024-04-13T05:27:31.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac200749c0 0x7fac2019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac2019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac20109d80 0x7fac201a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fac2010fc30 con 0x7fac20075fa0 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac2010fab0 con 0x7fac200749c0 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac2010fdb0 con 0x7fac20109d80 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac2019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac2019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42896/0 (socket says 172.21.15.57:42896) 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1e000640 1 -- 172.21.15.57:0/2655977698 learned_addr learned my addr 172.21.15.57:0/2655977698 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.859+0000 7fac1ea00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac200749c0 0x7fac2019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1f400640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac20109d80 0x7fac201a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1e000640 1 -- 172.21.15.57:0/2655977698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac20109d80 msgr2=0x7fac201a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1e000640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac20109d80 0x7fac201a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1e000640 1 -- 172.21.15.57:0/2655977698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac200749c0 msgr2=0x7fac2019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1e000640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac200749c0 0x7fac2019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1e000640 1 -- 172.21.15.57:0/2655977698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac201a1f10 con 0x7fac20075fa0 2024-04-13T05:27:31.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.860+0000 7fac1e000640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac2019a830 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c890 tx=0x7fac0c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.861+0000 7fac17400640 1 -- 172.21.15.57:0/2655977698 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004220 con 0x7fac20075fa0 2024-04-13T05:27:31.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.861+0000 7fac17400640 1 -- 172.21.15.57:0/2655977698 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac0c0043c0 con 0x7fac20075fa0 2024-04-13T05:27:31.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.861+0000 7fac17400640 1 -- 172.21.15.57:0/2655977698 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004a00 con 0x7fac20075fa0 2024-04-13T05:27:31.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.861+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac201a2200 con 0x7fac20075fa0 2024-04-13T05:27:31.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.861+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac201a2700 con 0x7fac20075fa0 2024-04-13T05:27:31.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 Processor -- start 2024-04-13T05:27:31.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 -- start start 2024-04-13T05:27:31.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33e8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a8a3346020 con 0x55a8a33e0e30 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8a32ec950 con 0x55a8a33e5fa0 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.864+0000 7fac28465740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8a3341fe0 con 0x55a8a33e0440 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33e5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42908/0 (socket says 172.21.15.57:42908) 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 -- 172.21.15.57:0/1542009935 learned_addr learned my addr 172.21.15.57:0/1542009935 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1f400640 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 -- 172.21.15.57:0/1542009935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 msgr2=0x55a8a33e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33e0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 -- 172.21.15.57:0/1542009935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 msgr2=0x55a8a33e8430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33e8430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.865+0000 7fac1ea00640 1 -- 172.21.15.57:0/1542009935 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a30583f0 con 0x55a8a33e0e30 2024-04-13T05:27:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.866+0000 7fac1ea00640 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33e5930 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fac10009900 tx=0x7fac10005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d588e154c7bd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.866+0000 7fac0b400640 1 -- 172.21.15.57:0/1542009935 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100043c0 con 0x55a8a33e0e30 2024-04-13T05:27:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.866+0000 7fac0b400640 1 -- 172.21.15.57:0/1542009935 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac1002ce30 con 0x55a8a33e0e30 2024-04-13T05:27:31.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.866+0000 7fac0b400640 1 -- 172.21.15.57:0/1542009935 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10004990 con 0x55a8a33e0e30 2024-04-13T05:27:31.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.866+0000 7fac17400640 1 -- 172.21.15.57:0/2655977698 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fac0c004ba0 con 0x7fac20075fa0 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 -- 172.21.15.57:0/1542009935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 msgr2=0x55a8a33e5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33e5930 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fac10009900 tx=0x7fac10005e80 comp rx=0 tx=0).stop 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 -- 172.21.15.57:0/1542009935 shutdown_connections 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33e5930 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 --2- 172.21.15.57:0/1542009935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 -- 172.21.15.57:0/1542009935 >> 172.21.15.57:0/1542009935 conn(0x55a8a33db9b0 msgr2=0x55a8a33dbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 -- 172.21.15.57:0/1542009935 shutdown_connections 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 -- 172.21.15.57:0/1542009935 wait complete. 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.867+0000 7fac28465740 1 -- start start 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 Processor -- start 2024-04-13T05:27:31.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33eae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a8a33e8ef0 con 0x55a8a33e0440 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8a32ffc80 con 0x55a8a33e5fa0 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8a33e9070 con 0x55a8a33e0e30 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac17400640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fac040a3b40 0x7fac040a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac28465740 10 client.0 _ll_register_callbacks cb 0x55a8a32ff670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac1f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac17400640 1 -- 172.21.15.57:0/2655977698 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fac0c0d0510 con 0x7fac20075fa0 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac1f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33ea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42922/0 (socket says 172.21.15.57:42922) 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.868+0000 7fac1f400640 1 -- 172.21.15.57:0/1066872629 learned_addr learned my addr 172.21.15.57:0/1066872629 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fac040a3b40 0x7fac040a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1e000640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1ea00640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 -- 172.21.15.57:0/1066872629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 msgr2=0x55a8a33eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33eae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 -- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 msgr2=0x55a8a33f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 -- 172.21.15.57:0/1066872629 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a31227d0 con 0x55a8a33e0440 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fac1f400640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33ea930 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fac1800fa50 tx=0x7fac1800ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cd911732ab61c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.869+0000 7fabffe00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fac1f400640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fac040a3b40 0x7fac040a6000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fac18011fd0 tx=0x7fac18016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fabffe00640 1 -- 172.21.15.57:0/1066872629 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18022070 con 0x55a8a33e0440 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fabffe00640 1 -- 172.21.15.57:0/1066872629 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fac18004b20 con 0x55a8a33e0440 2024-04-13T05:27:31.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1002ce30 con 0x55a8a33e0440 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fabffe00640 1 -- 172.21.15.57:0/1066872629 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fac1802d050 con 0x55a8a33e0440 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fac28465740 10 client.17457 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a8a33f52e0 con 0x55a8a33e0440 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.870+0000 7fabffe00640 1 -- 172.21.15.57:0/1066872629 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac1002ce30 con 0x55a8a33e0440 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fabffe00640 1 -- 172.21.15.57:0/1066872629 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fac18005020 con 0x55a8a33e0440 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fac28465740 10 client.17457 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fac28465740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fac28465740 20 client.17457 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fac28465740 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a33f6970 0x55a8a33f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fac28465740 4 client.17457 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.871+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8a32e5750 con 0x55a8a33f6970 2024-04-13T05:27:31.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.872+0000 7fac1ea00640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a33f6970 0x55a8a33f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.873+0000 7fac1ea00640 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a33f6970 0x55a8a33f8e30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6ed0052a7e15b5 server_cookie=93cd9e7ef18d30dd in_seq=0 out_seq=0 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.873+0000 7fabffe00640 10 client.17457 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.874+0000 7fabffe00640 1 -- 172.21.15.57:0/1066872629 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8a32e5750 con 0x55a8a33f6970 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.874+0000 7fabffe00640 10 client.17457 handle_command_reply: tid=0 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 client.17457 shutdown 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 msgr2=0x55a8a33ea930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33ea930 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fac1800fa50 tx=0x7fac1800ff10 comp rx=0 tx=0).stop 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 shutdown_connections 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a33f6970 0x55a8a33f8e30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 --2- 172.21.15.57:0/1066872629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33ea930 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fabffe00640 0 client.17457 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 >> 172.21.15.57:0/1066872629 conn(0x55a8a33db9b0 msgr2=0x55a8a33dacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 shutdown_connections 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 1 -- 172.21.15.57:0/1066872629 wait complete. 2024-04-13T05:27:31.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.875+0000 7fac28465740 20 client.17457 trim_cache size 0 max 16384 2024-04-13T05:27:31.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.890+0000 7fac28465740 1 Processor -- start 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.890+0000 7fac28465740 1 -- start start 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.890+0000 7fac28465740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.890+0000 7fac28465740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac28465740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac28465740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fac201af550 con 0x55a8a33e0440 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac28465740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac201acd60 con 0x55a8a33e5fa0 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac28465740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac201aabb0 con 0x55a8a33e0e30 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33eadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42924/0 (socket says 172.21.15.57:42924) 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 -- 172.21.15.57:0/2191622874 learned_addr learned my addr 172.21.15.57:0/2191622874 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 -- 172.21.15.57:0/2191622874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 msgr2=0x55a8a33f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 -- 172.21.15.57:0/2191622874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 msgr2=0x55a8a33df320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 -- 172.21.15.57:0/2191622874 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a33e55e0 con 0x55a8a33e0440 2024-04-13T05:27:31.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.891+0000 7fac1ea00640 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33eadd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac10005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77066421ad181a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac0aa00640 1 -- 172.21.15.57:0/2191622874 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10004340 con 0x55a8a33e0440 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac0aa00640 1 -- 172.21.15.57:0/2191622874 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac10005500 con 0x55a8a33e0440 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac0aa00640 1 -- 172.21.15.57:0/2191622874 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10004db0 con 0x55a8a33e0440 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 -- 172.21.15.57:0/2191622874 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 msgr2=0x55a8a33eadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33eadd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac10005fd0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 -- 172.21.15.57:0/2191622874 shutdown_connections 2024-04-13T05:27:31.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33df320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0e30 0x55a8a33f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 --2- 172.21.15.57:0/2191622874 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0440 0x55a8a33eadd0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 -- 172.21.15.57:0/2191622874 >> 172.21.15.57:0/2191622874 conn(0x55a8a33db9b0 msgr2=0x55a8a33dec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 -- 172.21.15.57:0/2191622874 shutdown_connections 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.892+0000 7fac28465740 1 -- 172.21.15.57:0/2191622874 wait complete. 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.893+0000 7fac28465740 1 -- start start 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.893+0000 7fac28465740 1 Processor -- start 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33efb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fac201ab220 con 0x55a8a33e0e30 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac201ac690 con 0x55a8a33e5fa0 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac201ab8f0 con 0x55a8a33e0440 2024-04-13T05:27:31.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac28465740 10 client.0 _ll_register_callbacks cb 0x7fac1800f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33f00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42928/0 (socket says 172.21.15.57:42928) 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 -- 172.21.15.57:0/471624213 learned_addr learned my addr 172.21.15.57:0/471624213 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1ea00640 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 -- 172.21.15.57:0/471624213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 msgr2=0x55a8a33efb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33efb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 -- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 msgr2=0x55a8a33f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.894+0000 7fac1e000640 1 -- 172.21.15.57:0/471624213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a33f16b0 con 0x55a8a33e0e30 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fac1e000640 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33f00b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fac0c004770 tx=0x7fac0c09c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37514f289903573f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fabffe00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fabffe00640 1 -- 172.21.15.57:0/471624213 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0b5060 con 0x55a8a33e0e30 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fabffe00640 1 -- 172.21.15.57:0/471624213 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fac0c0050b0 con 0x55a8a33e0e30 2024-04-13T05:27:31.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac10005500 con 0x55a8a33e0e30 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fabffe00640 1 -- 172.21.15.57:0/471624213 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fac0c01d040 con 0x55a8a33e0e30 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fac28465740 10 client.17463 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a8a345d590 con 0x55a8a33e0e30 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fabffe00640 1 -- 172.21.15.57:0/471624213 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac10005500 con 0x55a8a33e0e30 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fabffe00640 1 -- 172.21.15.57:0/471624213 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fac0c098140 con 0x55a8a33e0e30 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fac28465740 10 client.17463 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.895+0000 7fac28465740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.896+0000 7fac28465740 20 client.17463 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.896+0000 7fac28465740 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a345dfd0 0x55a8a34603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.896+0000 7fac28465740 4 client.17463 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:27:31.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.896+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8a3353a40 con 0x55a8a345dfd0 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.896+0000 7fac1ea00640 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a345dfd0 0x55a8a34603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.897+0000 7fac1ea00640 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a345dfd0 0x55a8a34603f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fba7fe77b5dc7cc server_cookie=1a72ec96f44a056e in_seq=0 out_seq=0 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.897+0000 7fabffe00640 10 client.17463 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fabffe00640 1 -- 172.21.15.57:0/471624213 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8a3353a40 con 0x55a8a345dfd0 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fabffe00640 10 client.17463 handle_command_reply: tid=0 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 client.17463 shutdown 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 msgr2=0x55a8a33f00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33f00b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fac0c004770 tx=0x7fac0c09c050 comp rx=0 tx=0).stop 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 shutdown_connections 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55a8a345dfd0 0x55a8a34603f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8a33e5fa0 0x55a8a33f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.903 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8a33e0e30 0x55a8a33f00b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 --2- 172.21.15.57:0/471624213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8a33e0440 0x55a8a33efb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fabffe00640 0 client.17463 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 >> 172.21.15.57:0/471624213 conn(0x55a8a33db9b0 msgr2=0x55a8a33df4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 shutdown_connections 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.898+0000 7fac28465740 1 -- 172.21.15.57:0/471624213 wait complete. 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac28465740 20 client.17463 trim_cache size 0 max 16384 2024-04-13T05:27:31.904 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fac040a3b40 msgr2=0x7fac040a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fac040a3b40 0x7fac040a6000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fac18011fd0 tx=0x7fac18016000 comp rx=0 tx=0).stop 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 msgr2=0x7fac2019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac2019a830 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c890 tx=0x7fac0c00cd50 comp rx=0 tx=0).stop 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 shutdown_connections 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fac040a3b40 0x7fac040a6000 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac20109d80 0x7fac201a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20075fa0 0x7fac2019a830 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 --2- 172.21.15.57:0/2655977698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac200749c0 0x7fac2019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:31.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 >> 172.21.15.57:0/2655977698 conn(0x7fac200fd460 msgr2=0x7fac20105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:31.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 shutdown_connections 2024-04-13T05:27:31.906 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:31.899+0000 7fac1fe00640 1 -- 172.21.15.57:0/2655977698 wait complete. 2024-04-13T05:27:32.613 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-04-13T05:27:32.613 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-13T05:27:32.919 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-13T05:27:32.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:27:32.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-13T05:27:33.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 -- 172.21.15.57:0/1840823583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 msgr2=0x7f51e0100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 --2- 172.21.15.57:0/1840823583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e0100d10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009900 tx=0x7f51d002bcd0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 -- 172.21.15.57:0/1840823583 shutdown_connections 2024-04-13T05:27:33.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 --2- 172.21.15.57:0/1840823583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e0101250 0x7f51e01016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 --2- 172.21.15.57:0/1840823583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0101bf0 0x7f51e0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 --2- 172.21.15.57:0/1840823583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e0100d10 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.131+0000 7f51e4e00640 1 -- 172.21.15.57:0/1840823583 >> 172.21.15.57:0/1840823583 conn(0x7f51e00fc470 msgr2=0x7f51e00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.132+0000 7f51e4e00640 1 -- 172.21.15.57:0/1840823583 shutdown_connections 2024-04-13T05:27:33.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.132+0000 7f51e4e00640 1 -- 172.21.15.57:0/1840823583 wait complete. 2024-04-13T05:27:33.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 Processor -- start 2024-04-13T05:27:33.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 -- start start 2024-04-13T05:27:33.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e0101250 0x7f51e019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0101bf0 0x7f51e01a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010fd90 con 0x7f51e01008f0 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010fc10 con 0x7f51e0101250 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51e4e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010ff10 con 0x7f51e0101bf0 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51dde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51dde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42936/0 (socket says 172.21.15.57:42936) 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.133+0000 7f51dde00640 1 -- 172.21.15.57:0/3267886968 learned_addr learned my addr 172.21.15.57:0/3267886968 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51d7e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e0101250 0x7f51e019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51dde00640 1 -- 172.21.15.57:0/3267886968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0101bf0 msgr2=0x7f51e01a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51dde00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0101bf0 0x7f51e01a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51dde00640 1 -- 172.21.15.57:0/3267886968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e0101250 msgr2=0x7f51e019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51dde00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e0101250 0x7f51e019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51dde00640 1 -- 172.21.15.57:0/3267886968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e01a2040 con 0x7f51e01008f0 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51dde00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e019a420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c00 tx=0x7f51d002dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51d6a00640 1 -- 172.21.15.57:0/3267886968 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0043020 con 0x7f51e01008f0 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51d6a00640 1 -- 172.21.15.57:0/3267886968 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d002de30 con 0x7f51e01008f0 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e01a22d0 con 0x7f51e01008f0 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.134+0000 7f51d6a00640 1 -- 172.21.15.57:0/3267886968 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0003d70 con 0x7f51e01008f0 2024-04-13T05:27:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.135+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e01a2820 con 0x7f51e01008f0 2024-04-13T05:27:33.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 Processor -- start 2024-04-13T05:27:33.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51d6a00640 1 -- 172.21.15.57:0/3267886968 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51d0046020 con 0x7f51e01008f0 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 -- start start 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b54096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b540e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5411270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5563b536fbc0 con 0x5563b540ede0 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563b5316660 con 0x5563b5409280 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51e7d85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563b536bb80 con 0x5563b5409c70 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5411270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.138+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5411270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42940/0 (socket says 172.21.15.57:42940) 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51d7e00640 1 -- 172.21.15.57:0/3497074207 learned_addr learned my addr 172.21.15.57:0/3497074207 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51de800640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b54096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51dde00640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b540e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51d7e00640 1 -- 172.21.15.57:0/3497074207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 msgr2=0x5563b540e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51d7e00640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b540e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51d7e00640 1 -- 172.21.15.57:0/3497074207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 msgr2=0x5563b54096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51d7e00640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b54096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51d7e00640 1 -- 172.21.15.57:0/3497074207 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b5339220 con 0x5563b540ede0 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51de800640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b54096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.139+0000 7f51dde00640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b540e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:33.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.140+0000 7f51d7e00640 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5411270 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f51c800bb10 tx=0x7f51c800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee327052aa3869d server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.140+0000 7f51cf400640 1 -- 172.21.15.57:0/3497074207 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8017420 con 0x5563b540ede0 2024-04-13T05:27:33.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.140+0000 7f51cf400640 1 -- 172.21.15.57:0/3497074207 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51c8017a70 con 0x5563b540ede0 2024-04-13T05:27:33.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.140+0000 7f51cf400640 1 -- 172.21.15.57:0/3497074207 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c800fa40 con 0x5563b540ede0 2024-04-13T05:27:33.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 -- 172.21.15.57:0/3497074207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 msgr2=0x5563b5411270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5411270 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f51c800bb10 tx=0x7f51c800d340 comp rx=0 tx=0).stop 2024-04-13T05:27:33.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 -- 172.21.15.57:0/3497074207 shutdown_connections 2024-04-13T05:27:33.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5411270 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b540e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 --2- 172.21.15.57:0/3497074207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b54096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 -- 172.21.15.57:0/3497074207 >> 172.21.15.57:0/3497074207 conn(0x5563b54047f0 msgr2=0x5563b5404bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51d6a00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51c40a3bb0 0x7f51c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 -- 172.21.15.57:0/3497074207 shutdown_connections 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51e7d85740 1 -- 172.21.15.57:0/3497074207 wait complete. 2024-04-13T05:27:33.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51d6a00640 1 -- 172.21.15.57:0/3267886968 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f51d00f2c00 con 0x7f51e01008f0 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.141+0000 7f51d7e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51c40a3bb0 0x7f51c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.142+0000 7f51e7d85740 1 -- start start 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.142+0000 7f51e7d85740 1 Processor -- start 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.142+0000 7f51d7e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51c40a3bb0 0x7f51c40a6070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f51c800bae0 tx=0x7f51c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.142+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409280 0x5563b5413740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 0x5563b5413c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b541acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51e7d85740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5563b5411d90 con 0x5563b5409c70 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51e7d85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563b5329a30 con 0x5563b540ede0 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51e7d85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563b5411f10 con 0x5563b5409280 2024-04-13T05:27:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51e7d85740 10 client.0 _ll_register_callbacks cb 0x5563b5081c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 0x5563b5413c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 0x5563b5413c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42946/0 (socket says 172.21.15.57:42946) 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 -- 172.21.15.57:0/1403078535 learned_addr learned my addr 172.21.15.57:0/1403078535 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51dde00640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409280 0x5563b5413740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51de800640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b541acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 -- 172.21.15.57:0/1403078535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409280 msgr2=0x5563b5413740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409280 0x5563b5413740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 -- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 msgr2=0x5563b541acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.143+0000 7f51d7e00640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b541acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.144+0000 7f51d7e00640 1 -- 172.21.15.57:0/1403078535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b514a7e0 con 0x5563b5409c70 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.144+0000 7f51d7e00640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 0x5563b5413c80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f51c8018000 tx=0x7f51c800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b45bca391b840e server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.144+0000 7f51c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.144+0000 7f51c3e00640 1 -- 172.21.15.57:0/1403078535 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8023650 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.144+0000 7f51c3e00640 1 -- 172.21.15.57:0/1403078535 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f51c80237f0 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.144+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c8017a70 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51c3e00640 1 -- 172.21.15.57:0/1403078535 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51c803c050 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 10 client.17475 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5563b51aaf70 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51c3e00640 1 -- 172.21.15.57:0/1403078535 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51c8017a70 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51c3e00640 1 -- 172.21.15.57:0/1403078535 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f51c802c070 con 0x5563b5409c70 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 10 client.17475 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 20 client.17475 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b541f470 0x5563b54218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 4 client.17475 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.145+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563b537d5e0 con 0x5563b541f470 2024-04-13T05:27:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.146+0000 7f51dde00640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b541f470 0x5563b54218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.147+0000 7f51dde00640 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b541f470 0x5563b54218f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48edfef35e439e4b server_cookie=e6a8fbdba5bddeaf in_seq=0 out_seq=0 2024-04-13T05:27:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.147+0000 7f51c3e00640 10 client.17475 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51c3e00640 1 -- 172.21.15.57:0/1403078535 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563b537d5e0 con 0x5563b541f470 2024-04-13T05:27:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51c3e00640 10 client.17475 handle_command_reply: tid=0 2024-04-13T05:27:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 client.17475 shutdown 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 msgr2=0x5563b5413c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 0x5563b5413c80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f51c8018000 tx=0x7f51c800d340 comp rx=0 tx=0).stop 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 shutdown_connections 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b541f470 0x5563b54218f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b541acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409c70 0x5563b5413c80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1403078535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409280 0x5563b5413740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 >> 172.21.15.57:0/1403078535 conn(0x5563b54047f0 msgr2=0x5563b54039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51c3e00640 0 client.17475 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 shutdown_connections 2024-04-13T05:27:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 1 -- 172.21.15.57:0/1403078535 wait complete. 2024-04-13T05:27:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.149+0000 7f51e7d85740 20 client.17475 trim_cache size 0 max 16384 2024-04-13T05:27:33.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.168+0000 7f51e7d85740 1 Processor -- start 2024-04-13T05:27:33.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.168+0000 7f51e7d85740 1 -- start start 2024-04-13T05:27:33.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 0x5563b5403160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b541ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b54036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51e7d85740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f51e01af730 con 0x5563b5409280 2024-04-13T05:27:33.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51e7d85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51e01acf40 con 0x5563b540ede0 2024-04-13T05:27:33.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51e7d85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51e01aad90 con 0x5563b5409c70 2024-04-13T05:27:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 0x5563b5403160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 0x5563b5403160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42962/0 (socket says 172.21.15.57:42962) 2024-04-13T05:27:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 -- 172.21.15.57:0/2655995405 learned_addr learned my addr 172.21.15.57:0/2655995405 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51d7e00640 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b541ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51de800640 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b54036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 -- 172.21.15.57:0/2655995405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 msgr2=0x5563b541ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b541ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 -- 172.21.15.57:0/2655995405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 msgr2=0x5563b54036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b54036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51dde00640 1 -- 172.21.15.57:0/2655995405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b5042170 con 0x5563b5409280 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.169+0000 7f51de800640 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b54036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.170+0000 7f51dde00640 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 0x5563b5403160 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f51d00092a0 tx=0x7f51d00f5050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c5365e73d5640f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.170+0000 7f51cea00640 1 -- 172.21.15.57:0/2655995405 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d00d8030 con 0x5563b5409280 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.170+0000 7f51cea00640 1 -- 172.21.15.57:0/2655995405 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d00bf1e0 con 0x5563b5409280 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.170+0000 7f51cea00640 1 -- 172.21.15.57:0/2655995405 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d00f7040 con 0x5563b5409280 2024-04-13T05:27:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 -- 172.21.15.57:0/2655995405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 msgr2=0x5563b5403160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 0x5563b5403160 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f51d00092a0 tx=0x7f51d00f5050 comp rx=0 tx=0).stop 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 -- 172.21.15.57:0/2655995405 shutdown_connections 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b540ede0 0x5563b54036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b541ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 --2- 172.21.15.57:0/2655995405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b5409280 0x5563b5403160 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 -- 172.21.15.57:0/2655995405 >> 172.21.15.57:0/2655995405 conn(0x5563b54047f0 msgr2=0x5563b540b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 -- 172.21.15.57:0/2655995405 shutdown_connections 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 -- 172.21.15.57:0/2655995405 wait complete. 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.171+0000 7f51e7d85740 1 -- start start 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 Processor -- start 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b5418260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b54187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5419740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f51e01ab400 con 0x5563b540ede0 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51e01ac870 con 0x5563b5409280 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51e01abad0 con 0x5563b5409c70 2024-04-13T05:27:33.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51e7d85740 10 client.0 _ll_register_callbacks cb 0x7f51c800b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51de800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5419740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51de800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5419740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42966/0 (socket says 172.21.15.57:42966) 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.172+0000 7f51de800640 1 -- 172.21.15.57:0/1548618056 learned_addr learned my addr 172.21.15.57:0/1548618056 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51dde00640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b5418260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51de800640 1 -- 172.21.15.57:0/1548618056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 msgr2=0x5563b54187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51d7e00640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b54187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51de800640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b54187a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51de800640 1 -- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 msgr2=0x5563b5418260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51de800640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b5418260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51de800640 1 -- 172.21.15.57:0/1548618056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b532abe0 con 0x5563b540ede0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51dde00640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b5418260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51de800640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5419740 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f51d8011850 tx=0x7f51d8011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5a5d6405eb8564 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:33.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51c3e00640 1 -- 172.21.15.57:0/1548618056 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d801f060 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d00bf1e0 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.173+0000 7f51c3e00640 1 -- 172.21.15.57:0/1548618056 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f51d800fcf0 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51c3e00640 1 -- 172.21.15.57:0/1548618056 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51d8026050 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 10 client.17481 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5563b541af60 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51c3e00640 1 -- 172.21.15.57:0/1548618056 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d00bf1e0 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51c3e00640 1 -- 172.21.15.57:0/1548618056 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f51d80183d0 con 0x5563b540ede0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 10 client.17481 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 20 client.17481 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b5486140 0x5563b5488560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 4 client.17481 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.174+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5563b5377450 con 0x5563b5486140 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.175+0000 7f51dde00640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b5486140 0x5563b5488560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.176+0000 7f51dde00640 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b5486140 0x5563b5488560 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f422132601cdf00 server_cookie=ea9967ae5069c512 in_seq=0 out_seq=0 2024-04-13T05:27:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.176+0000 7f51c3e00640 10 client.17481 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.178+0000 7f51c3e00640 1 -- 172.21.15.57:0/1548618056 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5563b5377450 con 0x5563b5486140 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.178+0000 7f51c3e00640 10 client.17481 handle_command_reply: tid=0 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.178+0000 7f51e7d85740 1 client.17481 shutdown 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.178+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 msgr2=0x5563b5419740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.178+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5419740 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f51d8011850 tx=0x7f51d8011d10 comp rx=0 tx=0).stop 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 shutdown_connections 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x5563b5486140 0x5563b5488560 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563b540ede0 0x5563b5419740 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563b5409c70 0x5563b54187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 --2- 172.21.15.57:0/1548618056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563b5409280 0x5563b5418260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51c3e00640 0 client.17481 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 >> 172.21.15.57:0/1548618056 conn(0x5563b54047f0 msgr2=0x5563b5407a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 shutdown_connections 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 1 -- 172.21.15.57:0/1548618056 wait complete. 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.179+0000 7f51e7d85740 20 client.17481 trim_cache size 0 max 16384 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "692fa1e8-c5ca-4188-a317-306e48173fbb", 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2024-04-13T05:27:33.187 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:27:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51c40a3bb0 msgr2=0x7f51c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51c40a3bb0 0x7f51c40a6070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f51c800bae0 tx=0x7f51c8005850 comp rx=0 tx=0).stop 2024-04-13T05:27:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 msgr2=0x7f51e019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e019a420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c00 tx=0x7f51d002dc40 comp rx=0 tx=0).stop 2024-04-13T05:27:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 shutdown_connections 2024-04-13T05:27:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f51c40a3bb0 0x7f51c40a6070 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e0101250 0x7f51e019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0101bf0 0x7f51e01a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 --2- 172.21.15.57:0/3267886968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51e01008f0 0x7f51e019a420 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 >> 172.21.15.57:0/3267886968 conn(0x7f51e00fc470 msgr2=0x7f51e00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 shutdown_connections 2024-04-13T05:27:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.180+0000 7f51e4e00640 1 -- 172.21.15.57:0/3267886968 wait complete. 2024-04-13T05:27:33.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:27:33.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:27:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 -- 172.21.15.57:0/3826477293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc103a40 msgr2=0x7f49fc10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 --2- 172.21.15.57:0/3826477293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc103a40 0x7f49fc10a360 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ca0 tx=0x7f49f402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 -- 172.21.15.57:0/3826477293 shutdown_connections 2024-04-13T05:27:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 --2- 172.21.15.57:0/3826477293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc103a40 0x7f49fc10a360 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 --2- 172.21.15.57:0/3826477293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49fc103080 0x7f49fc103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 --2- 172.21.15.57:0/3826477293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49fc101ea0 0x7f49fc1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 -- 172.21.15.57:0/3826477293 >> 172.21.15.57:0/3826477293 conn(0x7f49fc0fd460 msgr2=0x7f49fc0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 -- 172.21.15.57:0/3826477293 shutdown_connections 2024-04-13T05:27:33.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.404+0000 7f4a01600640 1 -- 172.21.15.57:0/3826477293 wait complete. 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.405+0000 7f4a01600640 1 Processor -- start 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 -- start start 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 0x7f49fc075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49fc103080 0x7f49fc0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49fc103a40 0x7f49fc0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10da50 con 0x7f49fc101ea0 2024-04-13T05:27:33.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10d8d0 con 0x7f49fc103a40 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f4a01600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10dbd0 con 0x7f49fc103080 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f49fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 0x7f49fc075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f49fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 0x7f49fc075d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42974/0 (socket says 172.21.15.57:42974) 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.406+0000 7f49fbe00640 1 -- 172.21.15.57:0/903010526 learned_addr learned my addr 172.21.15.57:0/903010526 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49fbe00640 1 -- 172.21.15.57:0/903010526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49fc103080 msgr2=0x7f49fc0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49fbe00640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49fc103080 0x7f49fc0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49fbe00640 1 -- 172.21.15.57:0/903010526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49fc103a40 msgr2=0x7f49fc0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f4a00c00640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49fc103a40 0x7f49fc0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49fbe00640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49fc103a40 0x7f49fc0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49fbe00640 1 -- 172.21.15.57:0/903010526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc075180 con 0x7f49fc101ea0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49fbe00640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 0x7f49fc075d70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c970 tx=0x7f49ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49f9600640 1 -- 172.21.15.57:0/903010526 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec007c10 con 0x7f49fc101ea0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc1a2100 con 0x7f49fc101ea0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49f9600640 1 -- 172.21.15.57:0/903010526 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49ec007db0 con 0x7f49fc101ea0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f49f9600640 1 -- 172.21.15.57:0/903010526 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00f7d0 con 0x7f49fc101ea0 2024-04-13T05:27:33.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.407+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1a26a0 con 0x7f49fc101ea0 2024-04-13T05:27:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.410+0000 7f4a05a12740 1 Processor -- start 2024-04-13T05:27:33.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 -- start start 2024-04-13T05:27:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 0x556507c07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c0f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556507b6e020 con 0x556507c076f0 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556507b146d0 con 0x556507c0d250 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a05a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556507b69fe0 con 0x556507c080e0 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a00c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 0x556507c07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a00c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 0x556507c07b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42982/0 (socket says 172.21.15.57:42982) 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.411+0000 7f4a00c00640 1 -- 172.21.15.57:0/1200372632 learned_addr learned my addr 172.21.15.57:0/1200372632 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f49f3e00640 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c0f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f4a00c00640 1 -- 172.21.15.57:0/1200372632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 msgr2=0x556507c0cbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f4a00c00640 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c0cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f4a00c00640 1 -- 172.21.15.57:0/1200372632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 msgr2=0x556507c0f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.416 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f4a00c00640 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c0f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f4a00c00640 1 -- 172.21.15.57:0/1200372632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650794a7d0 con 0x556507c076f0 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f49f3e00640 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f49f9600640 1 -- 172.21.15.57:0/903010526 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49ec00fa10 con 0x7f49fc101ea0 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f4a00c00640 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 0x556507c07b10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009800 tx=0x7f49f4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725cf0951dfb879f server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f49f1200640 1 -- 172.21.15.57:0/1200372632 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4039050 con 0x556507c076f0 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f49f1200640 1 -- 172.21.15.57:0/1200372632 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49f4045070 con 0x556507c076f0 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.412+0000 7f49f1200640 1 -- 172.21.15.57:0/1200372632 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f40042a0 con 0x556507c076f0 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 -- 172.21.15.57:0/1200372632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 msgr2=0x556507c07b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 0x556507c07b10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009800 tx=0x7f49f4009830 comp rx=0 tx=0).stop 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 -- 172.21.15.57:0/1200372632 shutdown_connections 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c0f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 --2- 172.21.15.57:0/1200372632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c076f0 0x556507c07b10 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 -- 172.21.15.57:0/1200372632 >> 172.21.15.57:0/1200372632 conn(0x556507c02c60 msgr2=0x556507c03060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 -- 172.21.15.57:0/1200372632 shutdown_connections 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.413+0000 7f4a05a12740 1 -- 172.21.15.57:0/1200372632 wait complete. 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 -- start start 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 Processor -- start 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c076f0 0x556507c119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f49f9600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49e00a3ac0 0x7f49e00a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c11f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556507c101a0 con 0x556507c080e0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556507b27a90 con 0x556507c0d250 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.414+0000 7f4a05a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556507c10320 con 0x556507c076f0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f4a05a12740 10 client.0 _ll_register_callbacks cb 0x556507b27480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f49f9600640 1 -- 172.21.15.57:0/903010526 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49ec0d0c00 con 0x7f49fc101ea0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c11f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f4a00c00640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49e00a3ac0 0x7f49e00a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c11f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42988/0 (socket says 172.21.15.57:42988) 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f49f3e00640 1 -- 172.21.15.57:0/2762616975 learned_addr learned my addr 172.21.15.57:0/2762616975 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f49fbe00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c076f0 0x556507c119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.415+0000 7f4a00c00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49fbe00640 1 -- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 msgr2=0x556507c18f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49fbe00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c18f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49fbe00640 1 -- 172.21.15.57:0/2762616975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 msgr2=0x556507c11f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49fbe00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c11f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49fbe00640 1 -- 172.21.15.57:0/2762616975 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565078803f0 con 0x556507c076f0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49f3e00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c11f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f4a00c00640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49e00a3ac0 0x7f49e00a5f80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ca0 tx=0x7f49f40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49fbe00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c076f0 0x556507c119f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x556507c08b50 tx=0x7f49ec018ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79203cad82601724 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.416+0000 7f49e9600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f49e9600640 1 -- 172.21.15.57:0/2762616975 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0a0f20 con 0x556507c076f0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f4045070 con 0x556507c076f0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f49e9600640 1 -- 172.21.15.57:0/2762616975 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49ec077330 con 0x556507c076f0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f49e9600640 1 -- 172.21.15.57:0/2762616975 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f49ec077550 con 0x556507c076f0 2024-04-13T05:27:33.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f4a05a12740 10 client.26539 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5565078400f0 con 0x556507c076f0 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.417+0000 7f49e9600640 1 -- 172.21.15.57:0/2762616975 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49f4045070 con 0x556507c076f0 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f49e9600640 1 -- 172.21.15.57:0/2762616975 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f49ec020030 con 0x556507c076f0 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f4a05a12740 10 client.26539 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f4a05a12740 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f4a05a12740 20 client.26539 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c1d710 0x556507c1fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f4a05a12740 4 client.26539 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.418+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556507b0d600 con 0x556507c1d710 2024-04-13T05:27:33.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.419+0000 7f49f3e00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c1d710 0x556507c1fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.419+0000 7f49f3e00640 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c1d710 0x556507c1fb30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc09abef8dcab3ec server_cookie=9049b308ebedbb26 in_seq=0 out_seq=0 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.419+0000 7f49e9600640 10 client.26539 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f49e9600640 1 -- 172.21.15.57:0/2762616975 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556507b0d600 con 0x556507c1d710 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f49e9600640 10 client.26539 handle_command_reply: tid=0 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f4a05a12740 1 client.26539 shutdown 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c076f0 msgr2=0x556507c119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c076f0 0x556507c119f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x556507c08b50 tx=0x7f49ec018ac0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 shutdown_connections 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.421+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c1d710 0x556507c1fb30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c0d250 0x556507c18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c11f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f49e9600640 0 client.26539 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2762616975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c076f0 0x556507c119f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 >> 172.21.15.57:0/2762616975 conn(0x556507c02c60 msgr2=0x556507c01e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 shutdown_connections 2024-04-13T05:27:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 1 -- 172.21.15.57:0/2762616975 wait complete. 2024-04-13T05:27:33.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.422+0000 7f4a05a12740 20 client.26539 trim_cache size 0 max 16384 2024-04-13T05:27:33.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 Processor -- start 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 -- start start 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c11d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c18f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 0x556507c015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1af350 con 0x556507c0d250 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1acb60 con 0x556507c076f0 2024-04-13T05:27:33.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a05a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1aa9b0 con 0x556507c080e0 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a00c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 0x556507c015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a00c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 0x556507c015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42992/0 (socket says 172.21.15.57:42992) 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.441+0000 7f4a00c00640 1 -- 172.21.15.57:0/2592239966 learned_addr learned my addr 172.21.15.57:0/2592239966 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f49fbe00640 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c11d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f49f3e00640 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c18f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f4a00c00640 1 -- 172.21.15.57:0/2592239966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 msgr2=0x556507c18f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f4a00c00640 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c18f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f4a00c00640 1 -- 172.21.15.57:0/2592239966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 msgr2=0x556507c11d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f4a00c00640 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c11d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f4a00c00640 1 -- 172.21.15.57:0/2592239966 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556507c1fea0 con 0x556507c0d250 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f49fbe00640 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c11d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f4a00c00640 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 0x556507c015f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f49f40098a0 tx=0x7f49f4051040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fd0070e05f5a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f49ebe00640 1 -- 172.21.15.57:0/2592239966 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4032970 con 0x556507c0d250 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f49ebe00640 1 -- 172.21.15.57:0/2592239966 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49f4045050 con 0x556507c0d250 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.442+0000 7f49ebe00640 1 -- 172.21.15.57:0/2592239966 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4059b80 con 0x556507c0d250 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 -- 172.21.15.57:0/2592239966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 msgr2=0x556507c015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 0x556507c015f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f49f40098a0 tx=0x7f49f4051040 comp rx=0 tx=0).stop 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 -- 172.21.15.57:0/2592239966 shutdown_connections 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c0d250 0x556507c015f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c080e0 0x556507c18f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 --2- 172.21.15.57:0/2592239966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c11d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 -- 172.21.15.57:0/2592239966 >> 172.21.15.57:0/2592239966 conn(0x556507c02c60 msgr2=0x556507c11600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 -- 172.21.15.57:0/2592239966 shutdown_connections 2024-04-13T05:27:33.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.443+0000 7f4a05a12740 1 -- 172.21.15.57:0/2592239966 wait complete. 2024-04-13T05:27:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 -- start start 2024-04-13T05:27:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 Processor -- start 2024-04-13T05:27:33.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c15f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c16450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c0d250 0x556507c8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1ab020 con 0x556507c080e0 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1ac490 con 0x556507c076f0 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1ab6f0 con 0x556507c0d250 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.444+0000 7f4a05a12740 10 client.0 _ll_register_callbacks cb 0x7f49ec09c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c16450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c16450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42994/0 (socket says 172.21.15.57:42994) 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 -- 172.21.15.57:0/965199441 learned_addr learned my addr 172.21.15.57:0/965199441 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f4a00c00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c0d250 0x556507c8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49fbe00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c15f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 -- 172.21.15.57:0/965199441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c0d250 msgr2=0x556507c8a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c0d250 0x556507c8a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 -- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 msgr2=0x556507c15f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c15f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 -- 172.21.15.57:0/965199441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556507c8aa60 con 0x556507c080e0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f4a00c00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c0d250 0x556507c8a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49f3e00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c16450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f49e400beb0 tx=0x7f49e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2baef13d26422e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.445+0000 7f49e9600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.446+0000 7f49e9600640 1 -- 172.21.15.57:0/965199441 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4005670 con 0x556507c080e0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.446+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f4045050 con 0x556507c080e0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.446+0000 7f49e9600640 1 -- 172.21.15.57:0/965199441 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f49e4004020 con 0x556507c080e0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.447+0000 7f49e9600640 1 -- 172.21.15.57:0/965199441 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f49e4029050 con 0x556507c080e0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.447+0000 7f4a05a12740 10 client.17499 fetch_fsmap learned FSMap version 33 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.447+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556507c8be30 con 0x556507c080e0 2024-04-13T05:27:33.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.447+0000 7f49e9600640 1 -- 172.21.15.57:0/965199441 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49f4045050 con 0x556507c080e0 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f49e9600640 1 -- 172.21.15.57:0/965199441 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f49e4003820 con 0x556507c080e0 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f4a05a12740 10 client.17499 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f4a05a12740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f4a05a12740 20 client.17499 populate_metadata read hostname 'smithi057' 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f4a05a12740 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c8c870 0x556507c8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f4a05a12740 4 client.17499 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556507b7ba40 con 0x556507c8c870 2024-04-13T05:27:33.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.448+0000 7f49fbe00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c8c870 0x556507c8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:27:33.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.449+0000 7f49fbe00640 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c8c870 0x556507c8ec90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926fae577a818d8b server_cookie=6dbafe14e813e9df in_seq=0 out_seq=0 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.449+0000 7f49e9600640 10 client.17499 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f49e9600640 1 -- 172.21.15.57:0/965199441 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556507b7ba40 con 0x556507c8c870 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f49e9600640 10 client.17499 handle_command_reply: tid=0 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 client.17499 shutdown 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 msgr2=0x556507c16450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c16450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f49e400beb0 tx=0x7f49e4004ad0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 shutdown_connections 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x556507c8c870 0x556507c8ec90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556507c0d250 0x556507c8a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556507c080e0 0x556507c16450 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 --2- 172.21.15.57:0/965199441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556507c076f0 0x556507c15f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 >> 172.21.15.57:0/965199441 conn(0x556507c02c60 msgr2=0x556507c05ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f49e9600640 0 client.17499 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 shutdown_connections 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 1 -- 172.21.15.57:0/965199441 wait complete. 2024-04-13T05:27:33.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.450+0000 7f4a05a12740 20 client.17499 trim_cache size 0 max 16384 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: "692fa1e8-c5ca-4188-a317-306e48173fbb": { 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "692fa1e8-c5ca-4188-a317-306e48173fbb", 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:27:33.457 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49e00a3ac0 msgr2=0x7f49e00a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49e00a3ac0 0x7f49e00a5f80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ca0 tx=0x7f49f40095c0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 msgr2=0x7f49fc075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 0x7f49fc075d70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c970 tx=0x7f49ec00ce30 comp rx=0 tx=0).stop 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 shutdown_connections 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f49e00a3ac0 0x7f49e00a5f80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49fc103a40 0x7f49fc0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49fc103080 0x7f49fc0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 --2- 172.21.15.57:0/903010526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49fc101ea0 0x7f49fc075d70 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:27:33.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 >> 172.21.15.57:0/903010526 conn(0x7f49fc0fd460 msgr2=0x7f49fc106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:27:33.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 shutdown_connections 2024-04-13T05:27:33.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:27:33.451+0000 7f4a01600640 1 -- 172.21.15.57:0/903010526 wait complete. 2024-04-13T05:27:33.470 INFO:tasks.cephfs.filesystem:scrub status for tag:692fa1e8-c5ca-4188-a317-306e48173fbb - {'path': '/', 'tag': '692fa1e8-c5ca-4188-a317-306e48173fbb', 'options': 'recursive,force'} 2024-04-13T05:27:33.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:33 smithi057 ceph-mon[32331]: pgmap v754: 129 pgs: 129 active+clean; 305 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 295 KiB/s wr, 134 op/s 2024-04-13T05:27:33.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:33 smithi083 ceph-mon[35133]: pgmap v754: 129 pgs: 129 active+clean; 305 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 295 KiB/s wr, 134 op/s 2024-04-13T05:27:33.783 DEBUG:teuthology.parallel:result is None 2024-04-13T05:27:33.783 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-13T05:27:33.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:33 smithi189 ceph-mon[35531]: pgmap v754: 129 pgs: 129 active+clean; 305 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 295 KiB/s wr, 134 op/s 2024-04-13T05:27:33.923 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-13T05:27:33.923 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-13T05:27:33.934 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-13T05:27:34.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:34 smithi083 ceph-mon[35133]: scrub queued for path: / 2024-04-13T05:27:34.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:34 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:34.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:34 smithi083 ceph-mon[35133]: scrub summary: active paths [/] 2024-04-13T05:27:34.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:34 smithi083 ceph-mon[35133]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:34.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:34 smithi057 ceph-mon[32331]: scrub queued for path: / 2024-04-13T05:27:34.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:34 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:34.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:34 smithi057 ceph-mon[32331]: scrub summary: active paths [/] 2024-04-13T05:27:34.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:34 smithi057 ceph-mon[32331]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:34.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:34 smithi189 ceph-mon[35531]: scrub queued for path: / 2024-04-13T05:27:34.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:34 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:34.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:34 smithi189 ceph-mon[35531]: scrub summary: active paths [/] 2024-04-13T05:27:34.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:34 smithi189 ceph-mon[35531]: scrub summary: idle+waiting paths [/] 2024-04-13T05:27:35.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:35 smithi083 ceph-mon[35133]: pgmap v755: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 449 KiB/s wr, 161 op/s 2024-04-13T05:27:35.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:35 smithi057 ceph-mon[32331]: pgmap v755: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 449 KiB/s wr, 161 op/s 2024-04-13T05:27:35.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:35 smithi189 ceph-mon[35531]: pgmap v755: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 449 KiB/s wr, 161 op/s 2024-04-13T05:27:37.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:37 smithi083 ceph-mon[35133]: pgmap v756: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 299 KiB/s wr, 153 op/s 2024-04-13T05:27:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:37 smithi057 ceph-mon[32331]: pgmap v756: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 299 KiB/s wr, 153 op/s 2024-04-13T05:27:37.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:37 smithi189 ceph-mon[35531]: pgmap v756: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 299 KiB/s wr, 153 op/s 2024-04-13T05:27:38.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:38 smithi083 ceph-mon[35133]: scrub summary: idle 2024-04-13T05:27:38.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:38 smithi057 ceph-mon[32331]: scrub summary: idle 2024-04-13T05:27:38.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:38 smithi189 ceph-mon[35531]: scrub summary: idle 2024-04-13T05:27:39.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:39 smithi083 ceph-mon[35133]: pgmap v757: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 293 KiB/s wr, 130 op/s 2024-04-13T05:27:39.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:39 smithi057 ceph-mon[32331]: pgmap v757: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 293 KiB/s wr, 130 op/s 2024-04-13T05:27:39.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:39 smithi189 ceph-mon[35531]: pgmap v757: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 293 KiB/s wr, 130 op/s 2024-04-13T05:27:41.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:41 smithi083 ceph-mon[35133]: pgmap v758: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 302 KiB/s wr, 135 op/s 2024-04-13T05:27:41.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:41 smithi057 ceph-mon[32331]: pgmap v758: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 302 KiB/s wr, 135 op/s 2024-04-13T05:27:41.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:41 smithi189 ceph-mon[35531]: pgmap v758: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 302 KiB/s wr, 135 op/s 2024-04-13T05:27:42.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:42 smithi083 ceph-mon[35133]: pgmap v759: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 170 KiB/s wr, 97 op/s 2024-04-13T05:27:42.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:42 smithi057 ceph-mon[32331]: pgmap v759: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 170 KiB/s wr, 97 op/s 2024-04-13T05:27:42.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:42 smithi189 ceph-mon[35531]: pgmap v759: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 170 KiB/s wr, 97 op/s 2024-04-13T05:27:45.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:44 smithi083 ceph-mon[35133]: pgmap v760: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 84 op/s 2024-04-13T05:27:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:44 smithi057 ceph-mon[32331]: pgmap v760: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 84 op/s 2024-04-13T05:27:45.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:44 smithi189 ceph-mon[35531]: pgmap v760: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 84 op/s 2024-04-13T05:27:47.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:46 smithi083 ceph-mon[35133]: pgmap v761: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 44 op/s 2024-04-13T05:27:47.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:46 smithi057 ceph-mon[32331]: pgmap v761: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 44 op/s 2024-04-13T05:27:47.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:46 smithi189 ceph-mon[35531]: pgmap v761: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 44 op/s 2024-04-13T05:27:49.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:49 smithi057 ceph-mon[32331]: pgmap v762: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-13T05:27:49.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:49 smithi189 ceph-mon[35531]: pgmap v762: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-13T05:27:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:49 smithi083 ceph-mon[35133]: pgmap v762: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-13T05:27:51.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:51 smithi057 ceph-mon[32331]: pgmap v763: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-13T05:27:51.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:51 smithi189 ceph-mon[35531]: pgmap v763: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-13T05:27:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:51 smithi083 ceph-mon[35133]: pgmap v763: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-13T05:27:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:53 smithi057 ceph-mon[32331]: pgmap v764: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-13T05:27:53.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:53 smithi189 ceph-mon[35531]: pgmap v764: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-13T05:27:53.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:53 smithi083 ceph-mon[35133]: pgmap v764: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-13T05:27:55.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:55 smithi057 ceph-mon[32331]: pgmap v765: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-13T05:27:55.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:55 smithi189 ceph-mon[35531]: pgmap v765: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-13T05:27:55.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:55 smithi083 ceph-mon[35133]: pgmap v765: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-13T05:27:57.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:57 smithi189 ceph-mon[35531]: pgmap v766: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-13T05:27:57.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:57 smithi083 ceph-mon[35133]: pgmap v766: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-13T05:27:57.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:57 smithi057 ceph-mon[32331]: pgmap v766: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-13T05:27:59.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:27:59 smithi189 ceph-mon[35531]: pgmap v767: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-13T05:27:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:27:59 smithi083 ceph-mon[35133]: pgmap v767: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-13T05:27:59.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:27:59 smithi057 ceph-mon[32331]: pgmap v767: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-13T05:28:01.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:01 smithi189 ceph-mon[35531]: pgmap v768: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-13T05:28:01.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:01 smithi083 ceph-mon[35133]: pgmap v768: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-13T05:28:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:01 smithi057 ceph-mon[32331]: pgmap v768: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-13T05:28:03.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:03 smithi189 ceph-mon[35531]: pgmap v769: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-13T05:28:03.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-13T05:28:03.472 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-13T05:28:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:03 smithi083 ceph-mon[35133]: pgmap v769: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-13T05:28:03.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:03 smithi057 ceph-mon[32331]: pgmap v769: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 -- 172.21.15.57:0/554783295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 msgr2=0x7f8074076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 --2- 172.21.15.57:0/554783295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f8074076da0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8064009900 tx=0x7f806402bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 -- 172.21.15.57:0/554783295 shutdown_connections 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 --2- 172.21.15.57:0/554783295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074078560 0x7f8074078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 --2- 172.21.15.57:0/554783295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8074077bc0 0x7f8074078020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 --2- 172.21.15.57:0/554783295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f8074076da0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 -- 172.21.15.57:0/554783295 >> 172.21.15.57:0/554783295 conn(0x7f80740ff670 msgr2=0x7f8074101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 -- 172.21.15.57:0/554783295 shutdown_connections 2024-04-13T05:28:03.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.708+0000 7f8078c00640 1 -- 172.21.15.57:0/554783295 wait complete. 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.709+0000 7f8078c00640 1 Processor -- start 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.709+0000 7f8078c00640 1 -- start start 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.709+0000 7f8078c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f807419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8078c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074077bc0 0x7f807419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8078c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8074078560 0x7f80741a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8078c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8074114080 con 0x7f8074076980 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8078c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8074113f00 con 0x7f8074077bc0 2024-04-13T05:28:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8078c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8074114200 con 0x7f8074078560 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f807419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f807419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49476/0 (socket says 172.21.15.57:49476) 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 -- 172.21.15.57:0/3985056471 learned_addr learned my addr 172.21.15.57:0/3985056471 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073e00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8074078560 0x7f80741a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 -- 172.21.15.57:0/3985056471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8074078560 msgr2=0x7f80741a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8074078560 0x7f80741a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8072a00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074077bc0 0x7f807419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 -- 172.21.15.57:0/3985056471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074077bc0 msgr2=0x7f807419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074077bc0 0x7f807419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8073400640 1 -- 172.21.15.57:0/3985056471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80741a6330 con 0x7f8074076980 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.710+0000 7f8072a00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074077bc0 0x7f807419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.711+0000 7f8073400640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f807419e710 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8064000c00 tx=0x7f80640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.711+0000 7f806be00640 1 -- 172.21.15.57:0/3985056471 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064043020 con 0x7f8074076980 2024-04-13T05:28:03.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.711+0000 7f806be00640 1 -- 172.21.15.57:0/3985056471 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80640054c0 con 0x7f8074076980 2024-04-13T05:28:03.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.711+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80741a65c0 con 0x7f8074076980 2024-04-13T05:28:03.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.711+0000 7f806be00640 1 -- 172.21.15.57:0/3985056471 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064003d40 con 0x7f8074076980 2024-04-13T05:28:03.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.711+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80741a6b10 con 0x7f8074076980 2024-04-13T05:28:03.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.714+0000 7f807d231740 1 Processor -- start 2024-04-13T05:28:03.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 -- start start 2024-04-13T05:28:03.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f806be00640 1 -- 172.21.15.57:0/3985056471 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8064046020 con 0x7f8074076980 2024-04-13T05:28:03.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0f96f0 0x559a2a0f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a0febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 0x559a2a1016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559a2a060020 con 0x559a2a0ff250 2024-04-13T05:28:03.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a2a0066d0 con 0x559a2a0fa0e0 2024-04-13T05:28:03.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f807d231740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a2a05bfe0 con 0x559a2a0f96f0 2024-04-13T05:28:03.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f8072a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 0x559a2a1016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f8072a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 0x559a2a1016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49484/0 (socket says 172.21.15.57:49484) 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.715+0000 7f8072a00640 1 -- 172.21.15.57:0/1425610210 learned_addr learned my addr 172.21.15.57:0/1425610210 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8073e00640 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0f96f0 0x559a2a0f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8073400640 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a0febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8072a00640 1 -- 172.21.15.57:0/1425610210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0f96f0 msgr2=0x559a2a0f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8072a00640 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0f96f0 0x559a2a0f9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8072a00640 1 -- 172.21.15.57:0/1425610210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 msgr2=0x559a2a0febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8072a00640 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a0febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8072a00640 1 -- 172.21.15.57:0/1425610210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a29e3c7d0 con 0x559a2a0ff250 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8073e00640 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0f96f0 0x559a2a0f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.716+0000 7f8072a00640 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 0x559a2a1016e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8060009c20 tx=0x7f806000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a85e15aed067b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.717+0000 7f806be00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80580a3bb0 0x7f80580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.717+0000 7f806be00640 1 -- 172.21.15.57:0/3985056471 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f80640f2bf0 con 0x7f8074076980 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.717+0000 7f805fe00640 1 -- 172.21.15.57:0/1425610210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060016070 con 0x559a2a0ff250 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.717+0000 7f8073e00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80580a3bb0 0x7f80580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.717+0000 7f805fe00640 1 -- 172.21.15.57:0/1425610210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8060011920 con 0x559a2a0ff250 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.717+0000 7f805fe00640 1 -- 172.21.15.57:0/1425610210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060010890 con 0x559a2a0ff250 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f8073e00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80580a3bb0 0x7f80580a6070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x559a2a0fa940 tx=0x7f806c0094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 -- 172.21.15.57:0/1425610210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 msgr2=0x559a2a1016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 0x559a2a1016e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8060009c20 tx=0x7f806000e6c0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 -- 172.21.15.57:0/1425610210 shutdown_connections 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a0febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0ff250 0x559a2a1016e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 --2- 172.21.15.57:0/1425610210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0f96f0 0x559a2a0f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 -- 172.21.15.57:0/1425610210 >> 172.21.15.57:0/1425610210 conn(0x559a2a0f4c60 msgr2=0x559a2a0f5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 -- 172.21.15.57:0/1425610210 shutdown_connections 2024-04-13T05:28:03.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 -- 172.21.15.57:0/1425610210 wait complete. 2024-04-13T05:28:03.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.718+0000 7f807d231740 1 -- start start 2024-04-13T05:28:03.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 Processor -- start 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a103f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a10af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559a2a102200 con 0x559a2a0f96f0 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a2a019a90 con 0x559a2a0ff250 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a2a102380 con 0x559a2a0fa0e0 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f807d231740 10 client.0 _ll_register_callbacks cb 0x559a2a019480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.719+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a1039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49498/0 (socket says 172.21.15.57:49498) 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 -- 172.21.15.57:0/1588967046 learned_addr learned my addr 172.21.15.57:0/1588967046 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8072a00640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073e00640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a10af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 -- 172.21.15.57:0/1588967046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 msgr2=0x559a2a103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a103f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 -- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 msgr2=0x559a2a10af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a10af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 -- 172.21.15.57:0/1588967046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a29d723f0 con 0x559a2a0f96f0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073e00640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a10af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f8073400640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a1039d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8064005ec0 tx=0x7f80640044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c61455a9a9d1313 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.720+0000 7f805cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f805cc00640 1 -- 172.21.15.57:0/1588967046 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640bfe60 con 0x559a2a0f96f0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f805cc00640 1 -- 172.21.15.57:0/1588967046 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f80640bce30 con 0x559a2a0f96f0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8060011920 con 0x559a2a0f96f0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f805cc00640 1 -- 172.21.15.57:0/1588967046 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8064004c50 con 0x559a2a0f96f0 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f807d231740 10 client.17511 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:03.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a29d320f0 con 0x559a2a0f96f0 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f805cc00640 1 -- 172.21.15.57:0/1588967046 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8060011920 con 0x559a2a0f96f0 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f805cc00640 1 -- 172.21.15.57:0/1588967046 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f80640f9070 con 0x559a2a0f96f0 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f807d231740 10 client.17511 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.721+0000 7f807d231740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.722+0000 7f807d231740 20 client.17511 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.722+0000 7f807d231740 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a10f8f0 0x559a2a111d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.722+0000 7f807d231740 4 client.17511 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.722+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a29fff600 con 0x559a2a10f8f0 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.722+0000 7f8072a00640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a10f8f0 0x559a2a111d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.723+0000 7f8072a00640 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a10f8f0 0x559a2a111d70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6427e98160317ab server_cookie=6bef5fff3152f3ab in_seq=0 out_seq=0 2024-04-13T05:28:03.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.723+0000 7f805cc00640 10 client.17511 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f805cc00640 1 -- 172.21.15.57:0/1588967046 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a29fff600 con 0x559a2a10f8f0 2024-04-13T05:28:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f805cc00640 10 client.17511 handle_command_reply: tid=0 2024-04-13T05:28:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 client.17511 shutdown 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 msgr2=0x559a2a1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a1039d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8064005ec0 tx=0x7f80640044e0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 shutdown_connections 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a10f8f0 0x559a2a111d70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a10af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f805cc00640 0 client.17511 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 --2- 172.21.15.57:0/1588967046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a1039d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 >> 172.21.15.57:0/1588967046 conn(0x559a2a0f4c60 msgr2=0x559a2a0f3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 shutdown_connections 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.725+0000 7f807d231740 1 -- 172.21.15.57:0/1588967046 wait complete. 2024-04-13T05:28:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.726+0000 7f807d231740 20 client.17511 trim_cache size 0 max 16384 2024-04-13T05:28:03.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 Processor -- start 2024-04-13T05:28:03.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 -- start start 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a103e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a10af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a0f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f80741b3820 con 0x559a2a0f96f0 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80741b1030 con 0x559a2a0ff250 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.745+0000 7f807d231740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80741aee80 con 0x559a2a0fa0e0 2024-04-13T05:28:03.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a103e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49512/0 (socket says 172.21.15.57:49512) 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 -- 172.21.15.57:0/416755632 learned_addr learned my addr 172.21.15.57:0/416755632 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8072a00640 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a10af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073e00640 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a0f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 -- 172.21.15.57:0/416755632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 msgr2=0x559a2a10af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a10af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 -- 172.21.15.57:0/416755632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 msgr2=0x559a2a0f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a0f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 -- 172.21.15.57:0/416755632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2a0fa4c0 con 0x559a2a0f96f0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.746+0000 7f8073400640 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a103e70 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f80640bb500 tx=0x7f8064005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f877a1d50ae267 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f805f400640 1 -- 172.21.15.57:0/416755632 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064004c50 con 0x559a2a0f96f0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f805f400640 1 -- 172.21.15.57:0/416755632 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80640bf230 con 0x559a2a0f96f0 2024-04-13T05:28:03.750 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f805f400640 1 -- 172.21.15.57:0/416755632 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640bcf00 con 0x559a2a0f96f0 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 -- 172.21.15.57:0/416755632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 msgr2=0x559a2a103e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a103e70 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f80640bb500 tx=0x7f8064005db0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 -- 172.21.15.57:0/416755632 shutdown_connections 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0ff250 0x559a2a0f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0fa0e0 0x559a2a10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 --2- 172.21.15.57:0/416755632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a103e70 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 -- 172.21.15.57:0/416755632 >> 172.21.15.57:0/416755632 conn(0x559a2a0f4c60 msgr2=0x559a2a0f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.747+0000 7f807d231740 1 -- 172.21.15.57:0/416755632 shutdown_connections 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 -- 172.21.15.57:0/416755632 wait complete. 2024-04-13T05:28:03.754 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 -- start start 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 Processor -- start 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a108670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a108bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0ff250 0x559a2a109b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f80741af4f0 con 0x559a2a0f96f0 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80741b0960 con 0x559a2a0fa0e0 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.748+0000 7f807d231740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80741afbc0 con 0x559a2a0ff250 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f807d231740 10 client.0 _ll_register_callbacks cb 0x7f8064009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a108670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a108670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49520/0 (socket says 172.21.15.57:49520) 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 -- 172.21.15.57:0/2801859391 learned_addr learned my addr 172.21.15.57:0/2801859391 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073e00640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0ff250 0x559a2a109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8072a00640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a108bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.755 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 -- 172.21.15.57:0/2801859391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0ff250 msgr2=0x559a2a109b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0ff250 0x559a2a109b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 -- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 msgr2=0x559a2a108bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a108bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8073400640 1 -- 172.21.15.57:0/2801859391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a2a10a140 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.749+0000 7f8072a00640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f8073400640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a108670 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f80640be830 tx=0x7f80640baf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e52cf58640f586 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f805cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f805cc00640 1 -- 172.21.15.57:0/2801859391 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064004c50 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f805cc00640 1 -- 172.21.15.57:0/2801859391 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f80640df0b0 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80640bf230 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f805cc00640 1 -- 172.21.15.57:0/2801859391 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8064005670 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f807d231740 10 client.17517 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a2a175ef0 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.750+0000 7f805cc00640 1 -- 172.21.15.57:0/2801859391 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80640bf230 con 0x559a2a0f96f0 2024-04-13T05:28:03.756 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f805cc00640 1 -- 172.21.15.57:0/2801859391 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f80640f9040 con 0x559a2a0f96f0 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f807d231740 10 client.17517 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f807d231740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f807d231740 20 client.17517 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f807d231740 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a176660 0x559a2a178a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f807d231740 4 client.17517 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a2a06da40 con 0x559a2a176660 2024-04-13T05:28:03.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.751+0000 7f8072a00640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a176660 0x559a2a178a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:03.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.752+0000 7f8072a00640 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a176660 0x559a2a178a80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9294c530806e80 server_cookie=6cfa6aa1a4f52f36 in_seq=0 out_seq=0 2024-04-13T05:28:03.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.752+0000 7f805cc00640 10 client.17517 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:03.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f805cc00640 1 -- 172.21.15.57:0/2801859391 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a2a06da40 con 0x559a2a176660 2024-04-13T05:28:03.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f805cc00640 10 client.17517 handle_command_reply: tid=0 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 client.17517 shutdown 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 msgr2=0x559a2a108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a108670 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f80640be830 tx=0x7f80640baf00 comp rx=0 tx=0).stop 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 shutdown_connections 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x559a2a176660 0x559a2a178a80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a2a0fa0e0 0x559a2a108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a2a0ff250 0x559a2a109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 --2- 172.21.15.57:0/2801859391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559a2a0f96f0 0x559a2a108670 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f805cc00640 0 client.17517 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 >> 172.21.15.57:0/2801859391 conn(0x559a2a0f4c60 msgr2=0x559a2a0f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.753+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 shutdown_connections 2024-04-13T05:28:03.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f807d231740 1 -- 172.21.15.57:0/2801859391 wait complete. 2024-04-13T05:28:03.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f807d231740 20 client.17517 trim_cache size 0 max 16384 2024-04-13T05:28:03.760 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:28:03.760 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2024-04-13T05:28:03.760 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2024-04-13T05:28:03.760 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80580a3bb0 msgr2=0x7f80580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80580a3bb0 0x7f80580a6070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x559a2a0fa940 tx=0x7f806c0094a0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 msgr2=0x7f807419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f807419e710 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8064000c00 tx=0x7f80640042f0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 shutdown_connections 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80580a3bb0 0x7f80580a6070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8074077bc0 0x7f807419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8074078560 0x7f80741a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 --2- 172.21.15.57:0/3985056471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8074076980 0x7f807419e710 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.754+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 >> 172.21.15.57:0/3985056471 conn(0x7f80740ff670 msgr2=0x7f8074100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.755+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 shutdown_connections 2024-04-13T05:28:03.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.755+0000 7f8078c00640 1 -- 172.21.15.57:0/3985056471 wait complete. 2024-04-13T05:28:03.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-13T05:28:03.776 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 -- 172.21.15.57:0/3898766984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 msgr2=0x7fc090110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 --2- 172.21.15.57:0/3898766984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc090110880 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc088009ca0 tx=0x7fc08802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 -- 172.21.15.57:0/3898766984 shutdown_connections 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 --2- 172.21.15.57:0/3898766984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 0x7fc090109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 --2- 172.21.15.57:0/3898766984 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc090110880 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 --2- 172.21.15.57:0/3898766984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 0x7fc090108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 -- 172.21.15.57:0/3898766984 >> 172.21.15.57:0/3898766984 conn(0x7fc090077400 msgr2=0x7fc090079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 -- 172.21.15.57:0/3898766984 shutdown_connections 2024-04-13T05:28:03.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.996+0000 7fc08f200640 1 -- 172.21.15.57:0/3898766984 wait complete. 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 Processor -- start 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 -- start start 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 0x7fc09019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 0x7fc09019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc0901a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc090113f50 con 0x7fc090109ff0 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc090113dd0 con 0x7fc090109630 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.997+0000 7fc08f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0901140d0 con 0x7fc090108440 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 0x7fc09019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc0901a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 0x7fc09019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.57:35600/0 (socket says 172.21.15.57:35600) 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08de00640 1 -- 172.21.15.57:0/1663436805 learned_addr learned my addr 172.21.15.57:0/1663436805 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc085400640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 0x7fc09019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08e800640 1 -- 172.21.15.57:0/1663436805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 msgr2=0x7fc09019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08e800640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 0x7fc09019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08e800640 1 -- 172.21.15.57:0/1663436805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 msgr2=0x7fc09019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.998+0000 7fc08e800640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 0x7fc09019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc08e800640 1 -- 172.21.15.57:0/1663436805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0901a61e0 con 0x7fc090109ff0 2024-04-13T05:28:04.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc085400640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 0x7fc09019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:04.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc08e800640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc0901a5b70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc088037040 tx=0x7fc088034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088041020 con 0x7fc090109ff0 2024-04-13T05:28:04.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0901a6470 con 0x7fc090109ff0 2024-04-13T05:28:04.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc088003e20 con 0x7fc090109ff0 2024-04-13T05:28:04.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088039040 con 0x7fc090109ff0 2024-04-13T05:28:04.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:03.999+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0901a6930 con 0x7fc090109ff0 2024-04-13T05:28:04.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.001+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09010f760 con 0x7fc090109ff0 2024-04-13T05:28:04.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.003+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc088044020 con 0x7fc090109ff0 2024-04-13T05:28:04.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.004+0000 7fc087400640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0700a3bb0 0x7fc0700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.004+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc0880f29b0 con 0x7fc090109ff0 2024-04-13T05:28:04.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.005+0000 7fc08de00640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0700a3bb0 0x7fc0700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.005+0000 7fc08de00640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0700a3bb0 0x7fc0700a6070 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc08000ace0 tx=0x7fc08000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.009+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc08804b020 con 0x7fc090109ff0 2024-04-13T05:28:04.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.255+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc090062bc0 con 0x7fc090109ff0 2024-04-13T05:28:04.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.257+0000 7fc087400640 1 -- 172.21.15.57:0/1663436805 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11864 (secure 0 0 0) 0x7fc0880a7e20 con 0x7fc090109ff0 2024-04-13T05:28:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:28:04.313 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:53.740918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:28:04.314 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-04-13T05:28:04.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0700a3bb0 msgr2=0x7fc0700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0700a3bb0 0x7fc0700a6070 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc08000ace0 tx=0x7fc08000a3b0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 msgr2=0x7fc0901a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc0901a5b70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc088037040 tx=0x7fc088034880 comp rx=0 tx=0).stop 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 shutdown_connections 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc0700a3bb0 0x7fc0700a6070 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc090109630 0x7fc09019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc090109ff0 0x7fc0901a5b70 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 --2- 172.21.15.57:0/1663436805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc090108440 0x7fc09019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 >> 172.21.15.57:0/1663436805 conn(0x7fc090077400 msgr2=0x7fc09010c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 shutdown_connections 2024-04-13T05:28:04.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.260+0000 7fc08f200640 1 -- 172.21.15.57:0/1663436805 wait complete. 2024-04-13T05:28:04.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-13T05:28:04.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:04 smithi057 ceph-mon[32331]: pgmap v770: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-13T05:28:04.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:04 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1663436805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:28:04.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 -- 172.21.15.57:0/4148772431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c068470 msgr2=0x7fe89c101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 --2- 172.21.15.57:0/4148772431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c068470 0x7fe89c101300 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009900 tx=0x7fe88c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:04.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 -- 172.21.15.57:0/4148772431 shutdown_connections 2024-04-13T05:28:04.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 --2- 172.21.15.57:0/4148772431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c101840 0x7fe89c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 --2- 172.21.15.57:0/4148772431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 0x7fe89c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 --2- 172.21.15.57:0/4148772431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c068470 0x7fe89c101300 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 -- 172.21.15.57:0/4148772431 >> 172.21.15.57:0/4148772431 conn(0x7fe89c0fb680 msgr2=0x7fe89c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 -- 172.21.15.57:0/4148772431 shutdown_connections 2024-04-13T05:28:04.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.553+0000 7fe89b200640 1 -- 172.21.15.57:0/4148772431 wait complete. 2024-04-13T05:28:04.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.554+0000 7fe89b200640 1 Processor -- start 2024-04-13T05:28:04.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 -- start start 2024-04-13T05:28:04.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c068470 0x7fe89c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 0x7fe89c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 0x7fe89c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c10ac00 con 0x7fe89c101840 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c10aa80 con 0x7fe89c068470 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe89b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c10ad80 con 0x7fe89c106540 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe899400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 0x7fe89c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe899400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 0x7fe89c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49548/0 (socket says 172.21.15.57:49548) 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe899400640 1 -- 172.21.15.57:0/1255355583 learned_addr learned my addr 172.21.15.57:0/1255355583 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.555+0000 7fe899e00640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c068470 0x7fe89c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe89a800640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 0x7fe89c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe899400640 1 -- 172.21.15.57:0/1255355583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 msgr2=0x7fe89c1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe899400640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 0x7fe89c1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe899400640 1 -- 172.21.15.57:0/1255355583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c068470 msgr2=0x7fe89c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe899400640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c068470 0x7fe89c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe899400640 1 -- 172.21.15.57:0/1255355583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c1a1fb0 con 0x7fe89c101840 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe89a800640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 0x7fe89c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe899400640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 0x7fe89c19a8d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe88800d950 tx=0x7fe88800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe892a00640 1 -- 172.21.15.57:0/1255355583 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888017020 con 0x7fe89c101840 2024-04-13T05:28:04.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe892a00640 1 -- 172.21.15.57:0/1255355583 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe888009c30 con 0x7fe89c101840 2024-04-13T05:28:04.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c1a22a0 con 0x7fe89c101840 2024-04-13T05:28:04.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe892a00640 1 -- 172.21.15.57:0/1255355583 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88800f6b0 con 0x7fe89c101840 2024-04-13T05:28:04.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.556+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c1a27f0 con 0x7fe89c101840 2024-04-13T05:28:04.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 Processor -- start 2024-04-13T05:28:04.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 -- start start 2024-04-13T05:28:04.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe892a00640 1 -- 172.21.15.57:0/1255355583 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe88801a020 con 0x7fe89c101840 2024-04-13T05:28:04.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a589860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a58e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a591430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55eb3a4ef020 con 0x55eb3a589440 2024-04-13T05:28:04.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb3a495950 con 0x55eb3a589e30 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.560+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb3a4eafe0 con 0x55eb3a58efa0 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a589860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a589860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49560/0 (socket says 172.21.15.57:49560) 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 -- 172.21.15.57:0/218190598 learned_addr learned my addr 172.21.15.57:0/218190598 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe899400640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a591430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 -- 172.21.15.57:0/218190598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 msgr2=0x55eb3a591430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe899e00640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a58e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a591430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 -- 172.21.15.57:0/218190598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 msgr2=0x55eb3a58e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a58e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe89a800640 1 -- 172.21.15.57:0/218190598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3a2013f0 con 0x55eb3a589440 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe899400640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a591430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.561+0000 7fe899e00640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a58e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe89a800640 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a589860 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe894009e40 tx=0x7fe89400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dbe52fbfd88e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe887400640 1 -- 172.21.15.57:0/218190598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894004d20 con 0x55eb3a589440 2024-04-13T05:28:04.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe887400640 1 -- 172.21.15.57:0/218190598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8940054d0 con 0x55eb3a589440 2024-04-13T05:28:04.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe892a00640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe8800a3bb0 0x7fe8800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe887400640 1 -- 172.21.15.57:0/218190598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894011800 con 0x55eb3a589440 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe892a00640 1 -- 172.21.15.57:0/1255355583 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe8880d0820 con 0x7fe89c101840 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.562+0000 7fe899e00640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe8800a3bb0 0x7fe8800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 -- 172.21.15.57:0/218190598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 msgr2=0x55eb3a589860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a589860 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe894009e40 tx=0x7fe89400f730 comp rx=0 tx=0).stop 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 -- 172.21.15.57:0/218190598 shutdown_connections 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a58e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a591430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/218190598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a589860 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 -- 172.21.15.57:0/218190598 >> 172.21.15.57:0/218190598 conn(0x55eb3a5849b0 msgr2=0x55eb3a584db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 -- 172.21.15.57:0/218190598 shutdown_connections 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe8a3de2740 1 -- 172.21.15.57:0/218190598 wait complete. 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.563+0000 7fe899e00640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe8800a3bb0 0x7fe8800a6070 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe88c006010 tx=0x7fe88c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 -- start start 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 Processor -- start 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589440 0x55eb3a5938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a593e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55eb3a591fa0 con 0x55eb3a58efa0 2024-04-13T05:28:04.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb3a4a8c80 con 0x55eb3a589e30 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.564+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb3a592120 con 0x55eb3a589440 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe8a3de2740 10 client.0 _ll_register_callbacks cb 0x55eb3a4a8670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49564/0 (socket says 172.21.15.57:49564) 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 -- 172.21.15.57:0/1713899837 learned_addr learned my addr 172.21.15.57:0/1713899837 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe899e00640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589440 0x55eb3a5938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe899400640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a593e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 -- 172.21.15.57:0/1713899837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589440 msgr2=0x55eb3a5938d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589440 0x55eb3a5938d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 -- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 msgr2=0x55eb3a593e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a593e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.565+0000 7fe89a800640 1 -- 172.21.15.57:0/1713899837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3a2cb7d0 con 0x55eb3a58efa0 2024-04-13T05:28:04.571 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe89a800640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe894009c00 tx=0x7fe8940059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55865814781b534 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe87be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe87be00640 1 -- 172.21.15.57:0/1713899837 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894028060 con 0x55eb3a58efa0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8940054d0 con 0x55eb3a58efa0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe87be00640 1 -- 172.21.15.57:0/1713899837 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe8940049c0 con 0x55eb3a58efa0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe87be00640 1 -- 172.21.15.57:0/1713899837 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe89402f050 con 0x55eb3a58efa0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe8a3de2740 10 client.17535 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.566+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb3a59e1f0 con 0x55eb3a58efa0 2024-04-13T05:28:04.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe87be00640 1 -- 172.21.15.57:0/1713899837 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8940054d0 con 0x55eb3a58efa0 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe87be00640 1 -- 172.21.15.57:0/1713899837 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fe894019430 con 0x55eb3a58efa0 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe8a3de2740 10 client.17535 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe8a3de2740 10 client.17535 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe8a3de2740 20 client.17535 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a59f590 0x55eb3a5a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe8a3de2740 4 client.17535 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.567+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb3a48e750 con 0x55eb3a59f590 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.568+0000 7fe899e00640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a59f590 0x55eb3a5a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.569+0000 7fe899e00640 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a59f590 0x55eb3a5a19c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c61e141aa8fefc9 server_cookie=d4b296d7ded38d1a in_seq=0 out_seq=0 2024-04-13T05:28:04.574 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.569+0000 7fe87be00640 10 client.17535 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe87be00640 1 -- 172.21.15.57:0/1713899837 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb3a48e750 con 0x55eb3a59f590 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe87be00640 10 client.17535 handle_command_reply: tid=0 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 client.17535 shutdown 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 msgr2=0x55eb3a59ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe894009c00 tx=0x7fe8940059f0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 shutdown_connections 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a59f590 0x55eb3a5a19c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a593e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae50 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe87be00640 0 client.17535 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1713899837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589440 0x55eb3a5938d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 >> 172.21.15.57:0/1713899837 conn(0x55eb3a5849b0 msgr2=0x55eb3a583cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 shutdown_connections 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.571+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1713899837 wait complete. 2024-04-13T05:28:04.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.572+0000 7fe8a3de2740 20 client.17535 trim_cache size 0 max 16384 2024-04-13T05:28:04.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.586+0000 7fe8a3de2740 1 Processor -- start 2024-04-13T05:28:04.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 -- start start 2024-04-13T05:28:04.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a58b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589e30 0x55eb3a58b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1af6a0 con 0x55eb3a589440 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1aceb0 con 0x55eb3a58efa0 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1aad00 con 0x55eb3a589e30 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe899e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a58b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe899e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a58b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49580/0 (socket says 172.21.15.57:49580) 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.587+0000 7fe899e00640 1 -- 172.21.15.57:0/3749403445 learned_addr learned my addr 172.21.15.57:0/3749403445 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe89a800640 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899400640 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589e30 0x55eb3a58b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899e00640 1 -- 172.21.15.57:0/3749403445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589e30 msgr2=0x55eb3a58b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899e00640 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589e30 0x55eb3a58b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899e00640 1 -- 172.21.15.57:0/3749403445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a58efa0 msgr2=0x55eb3a59ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899e00640 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899e00640 1 -- 172.21.15.57:0/3749403445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3a59dff0 con 0x55eb3a589440 2024-04-13T05:28:04.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899400640 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589e30 0x55eb3a58b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe899e00640 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a58b300 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe88c02ce10 tx=0x7fe88c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88c4290be9e8d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe886a00640 1 -- 172.21.15.57:0/3749403445 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c054070 con 0x55eb3a589440 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe886a00640 1 -- 172.21.15.57:0/3749403445 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe88c045040 con 0x55eb3a589440 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.588+0000 7fe886a00640 1 -- 172.21.15.57:0/3749403445 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c04f070 con 0x55eb3a589440 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 -- 172.21.15.57:0/3749403445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 msgr2=0x55eb3a58b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a58b300 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe88c02ce10 tx=0x7fe88c03e000 comp rx=0 tx=0).stop 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 -- 172.21.15.57:0/3749403445 shutdown_connections 2024-04-13T05:28:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a58efa0 0x55eb3a59ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a589e30 0x55eb3a58b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/3749403445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a58b300 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 -- 172.21.15.57:0/3749403445 >> 172.21.15.57:0/3749403445 conn(0x55eb3a5849b0 msgr2=0x55eb3a57fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 -- 172.21.15.57:0/3749403445 shutdown_connections 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.589+0000 7fe8a3de2740 1 -- 172.21.15.57:0/3749403445 wait complete. 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 -- start start 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 Processor -- start 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a597ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a598020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a603af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1ab370 con 0x55eb3a589440 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1ac7e0 con 0x55eb3a589e30 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1aba40 con 0x55eb3a58efa0 2024-04-13T05:28:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.590+0000 7fe8a3de2740 10 client.0 _ll_register_callbacks cb 0x7fe894005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a597ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a597ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49594/0 (socket says 172.21.15.57:49594) 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 -- 172.21.15.57:0/1689498414 learned_addr learned my addr 172.21.15.57:0/1689498414 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 -- 172.21.15.57:0/1689498414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 msgr2=0x55eb3a603af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899400640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a598020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a603af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 -- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 msgr2=0x55eb3a598020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a598020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 -- 172.21.15.57:0/1689498414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3a598630 con 0x55eb3a589440 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899400640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a598020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe899e00640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a597ae0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe88c036040 tx=0x7fe88c04f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba61e923fa9d9cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.591+0000 7fe87be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe87be00640 1 -- 172.21.15.57:0/1689498414 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c054070 con 0x55eb3a589440 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe87be00640 1 -- 172.21.15.57:0/1689498414 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fe88c005030 con 0x55eb3a589440 2024-04-13T05:28:04.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c045040 con 0x55eb3a589440 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe87be00640 1 -- 172.21.15.57:0/1689498414 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe88c060050 con 0x55eb3a589440 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 10 client.17541 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb3a605030 con 0x55eb3a589440 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe87be00640 1 -- 172.21.15.57:0/1689498414 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe88c045040 con 0x55eb3a589440 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe87be00640 1 -- 172.21.15.57:0/1689498414 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fe88c04a040 con 0x55eb3a589440 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 10 client.17541 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 10 client.17541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 20 client.17541 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a605a70 0x55eb3a607e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.592+0000 7fe8a3de2740 4 client.17541 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:28:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.593+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 --> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb3a4fca40 con 0x55eb3a605a70 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.593+0000 7fe899400640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a605a70 0x55eb3a607e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.594+0000 7fe899400640 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a605a70 0x55eb3a607e90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe991137ec12797a server_cookie=b22714a7fee0b47e in_seq=0 out_seq=0 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.594+0000 7fe87be00640 10 client.17541 ms_handle_connect on v2:172.21.15.83:6834/3245344075 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe87be00640 1 -- 172.21.15.57:0/1689498414 <== mds.2 v2:172.21.15.83:6834/3245344075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb3a4fca40 con 0x55eb3a605a70 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe87be00640 10 client.17541 handle_command_reply: tid=0 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 client.17541 shutdown 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 msgr2=0x55eb3a597ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a597ae0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe88c036040 tx=0x7fe88c04f930 comp rx=0 tx=0).stop 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 shutdown_connections 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:6834/3245344075,v1:172.21.15.83:6835/3245344075] conn(0x55eb3a605a70 0x55eb3a607e90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb3a589e30 0x55eb3a598020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb3a58efa0 0x55eb3a603af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 --2- 172.21.15.57:0/1689498414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55eb3a589440 0x55eb3a597ae0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 >> 172.21.15.57:0/1689498414 conn(0x55eb3a5849b0 msgr2=0x55eb3a58b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe87be00640 0 client.17541 ms_handle_reset on v2:172.21.15.83:6834/3245344075 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 shutdown_connections 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 1 -- 172.21.15.57:0/1689498414 wait complete. 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.595+0000 7fe8a3de2740 20 client.17541 trim_cache size 0 max 16384 2024-04-13T05:28:04.601 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe8800a3bb0 msgr2=0x7fe8800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe8800a3bb0 0x7fe8800a6070 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe88c006010 tx=0x7fe88c005a20 comp rx=0 tx=0).stop 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 msgr2=0x7fe89c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 0x7fe89c19a8d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe88800d950 tx=0x7fe88800de10 comp rx=0 tx=0).stop 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 shutdown_connections 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fe8800a3bb0 0x7fe8800a6070 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe89c106540 0x7fe89c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe89c101840 0x7fe89c19a8d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 --2- 172.21.15.57:0/1255355583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe89c068470 0x7fe89c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 >> 172.21.15.57:0/1255355583 conn(0x7fe89c0fb680 msgr2=0x7fe89c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 shutdown_connections 2024-04-13T05:28:04.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.596+0000 7fe89b200640 1 -- 172.21.15.57:0/1255355583 wait complete. 2024-04-13T05:28:04.619 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-13T05:28:04.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:04 smithi083 ceph-mon[35133]: pgmap v770: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-13T05:28:04.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:04 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1663436805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:28:04.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:04 smithi189 ceph-mon[35531]: pgmap v770: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-13T05:28:04.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:04 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1663436805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 -- 172.21.15.57:0/3708213214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6548108620 msgr2=0x7f6548108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 --2- 172.21.15.57:0/3708213214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6548108620 0x7f6548108a40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6538009ca0 tx=0x7f653802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 -- 172.21.15.57:0/3708213214 shutdown_connections 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 --2- 172.21.15.57:0/3708213214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f654810a1d0 0x7f6548110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 --2- 172.21.15.57:0/3708213214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548109810 0x7f6548109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 --2- 172.21.15.57:0/3708213214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6548108620 0x7f6548108a40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 -- 172.21.15.57:0/3708213214 >> 172.21.15.57:0/3708213214 conn(0x7f65480775c0 msgr2=0x7f6548079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 -- 172.21.15.57:0/3708213214 shutdown_connections 2024-04-13T05:28:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.841+0000 7f654dc00640 1 -- 172.21.15.57:0/3708213214 wait complete. 2024-04-13T05:28:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.842+0000 7f654dc00640 1 Processor -- start 2024-04-13T05:28:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 -- start start 2024-04-13T05:28:04.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548108620 0x7f654819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6548109810 0x7f654819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 0x7f65481a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6548114150 con 0x7f654810a1d0 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6548113fd0 con 0x7f6548109810 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f654dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65481142d0 con 0x7f6548108620 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f6547400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 0x7f65481a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f6547400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 0x7f65481a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49610/0 (socket says 172.21.15.57:49610) 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.843+0000 7f6547400640 1 -- 172.21.15.57:0/2288216275 learned_addr learned my addr 172.21.15.57:0/2288216275 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6546a00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548108620 0x7f654819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6546000640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6548109810 0x7f654819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6547400640 1 -- 172.21.15.57:0/2288216275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548108620 msgr2=0x7f654819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6547400640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548108620 0x7f654819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6547400640 1 -- 172.21.15.57:0/2288216275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6548109810 msgr2=0x7f654819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6547400640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6548109810 0x7f654819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6547400640 1 -- 172.21.15.57:0/2288216275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65481a64b0 con 0x7f654810a1d0 2024-04-13T05:28:04.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6546a00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548108620 0x7f654819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:04.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.844+0000 7f6547400640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 0x7f65481a5e40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f654000e970 tx=0x7f654000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.845+0000 7f653f400640 1 -- 172.21.15.57:0/2288216275 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654000ccc0 con 0x7f654810a1d0 2024-04-13T05:28:04.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.845+0000 7f653f400640 1 -- 172.21.15.57:0/2288216275 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f654000ce60 con 0x7f654810a1d0 2024-04-13T05:28:04.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.845+0000 7f653f400640 1 -- 172.21.15.57:0/2288216275 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540010430 con 0x7f654810a1d0 2024-04-13T05:28:04.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.845+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65481a67a0 con 0x7f654810a1d0 2024-04-13T05:28:04.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.845+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65481a6ca0 con 0x7f654810a1d0 2024-04-13T05:28:04.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.848+0000 7f6550c34740 1 Processor -- start 2024-04-13T05:28:04.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.848+0000 7f653f400640 1 -- 172.21.15.57:0/2288216275 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6540004ff0 con 0x7f654810a1d0 2024-04-13T05:28:04.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 -- start start 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b86670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86c40 0x561833b8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b8e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561833aec020 con 0x561833b86250 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561833a92950 con 0x561833b86c40 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6550c34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561833ae7fe0 con 0x561833b8bdb0 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6546000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b86670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6546000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b86670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49614/0 (socket says 172.21.15.57:49614) 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6546000640 1 -- 172.21.15.57:0/1118397218 learned_addr learned my addr 172.21.15.57:0/1118397218 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.849+0000 7f6546a00640 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86c40 0x561833b8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546000640 1 -- 172.21.15.57:0/1118397218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 msgr2=0x561833b8e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546000640 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b8e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546000640 1 -- 172.21.15.57:0/1118397218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86c40 msgr2=0x561833b8b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546000640 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86c40 0x561833b8b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546000640 1 -- 172.21.15.57:0/1118397218 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618337fe3f0 con 0x561833b86250 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546a00640 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86c40 0x561833b8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f653f400640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f652c0a3c30 0x7f652c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546000640 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b86670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f6534009ca0 tx=0x7f65340137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f81cb1bc674ce89 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f653f400640 1 -- 172.21.15.57:0/2288216275 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f65400d0c70 con 0x7f654810a1d0 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6546a00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f652c0a3c30 0x7f652c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.850+0000 7f6533e00640 1 -- 172.21.15.57:0/1118397218 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653401c470 con 0x561833b86250 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6533e00640 1 -- 172.21.15.57:0/1118397218 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534017070 con 0x561833b86250 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6533e00640 1 -- 172.21.15.57:0/1118397218 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534011970 con 0x561833b86250 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 -- 172.21.15.57:0/1118397218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 msgr2=0x561833b86670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6546a00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f652c0a3c30 0x7f652c0a60f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f653802c630 tx=0x7f653802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b86670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f6534009ca0 tx=0x7f65340137d0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 -- 172.21.15.57:0/1118397218 shutdown_connections 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86c40 0x561833b8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b8e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 --2- 172.21.15.57:0/1118397218 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b86670 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 -- 172.21.15.57:0/1118397218 >> 172.21.15.57:0/1118397218 conn(0x561833b817c0 msgr2=0x561833b81bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.851+0000 7f6550c34740 1 -- 172.21.15.57:0/1118397218 shutdown_connections 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.852+0000 7f6550c34740 1 -- 172.21.15.57:0/1118397218 wait complete. 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.852+0000 7f6550c34740 1 -- start start 2024-04-13T05:28:04.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.852+0000 7f6550c34740 1 Processor -- start 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86250 0x561833b90860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b90da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b97d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561833b8ed60 con 0x561833b86c40 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561833aa5c80 con 0x561833b86250 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561833b8eee0 con 0x561833b8bdb0 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6550c34740 10 client.0 _ll_register_callbacks cb 0x561833aa5670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:04.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b90da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b90da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49616/0 (socket says 172.21.15.57:49616) 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 -- 172.21.15.57:0/931889621 learned_addr learned my addr 172.21.15.57:0/931889621 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546a00640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86250 0x561833b90860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6547400640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b97d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 -- 172.21.15.57:0/931889621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 msgr2=0x561833b97d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b97d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 -- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86250 msgr2=0x561833b90860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86250 0x561833b90860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6546000640 1 -- 172.21.15.57:0/931889621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618338c87d0 con 0x561833b86c40 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.853+0000 7f6547400640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b97d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6546a00640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86250 0x561833b90860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6546000640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b90da0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6534016040 tx=0x7f6534009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62696f5666b1217 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6530c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6530c00640 1 -- 172.21.15.57:0/931889621 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534021070 con 0x561833b86c40 2024-04-13T05:28:04.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6530c00640 1 -- 172.21.15.57:0/931889621 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6534004460 con 0x561833b86c40 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6534017070 con 0x561833b86c40 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6530c00640 1 -- 172.21.15.57:0/931889621 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6534030050 con 0x561833b86c40 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6550c34740 10 client.17553 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.854+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561833b9b0e0 con 0x561833b86c40 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6530c00640 1 -- 172.21.15.57:0/931889621 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534017070 con 0x561833b86c40 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6530c00640 1 -- 172.21.15.57:0/931889621 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f65340048f0 con 0x561833b86c40 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6550c34740 10 client.17553 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6550c34740 10 client.17553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6550c34740 20 client.17553 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6550c34740 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833b9c580 0x561833b9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6550c34740 4 client.17553 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.855+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561833a8b750 con 0x561833b9c580 2024-04-13T05:28:04.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.856+0000 7f6546a00640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833b9c580 0x561833b9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.857+0000 7f6546a00640 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833b9c580 0x561833b9e970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1638e79e0af9261e server_cookie=29d0541bb14310a3 in_seq=0 out_seq=0 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.857+0000 7f6530c00640 10 client.17553 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6530c00640 1 -- 172.21.15.57:0/931889621 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561833a8b750 con 0x561833b9c580 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6530c00640 10 client.17553 handle_command_reply: tid=0 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 client.17553 shutdown 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 msgr2=0x561833b90da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b90da0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6534016040 tx=0x7f6534009630 comp rx=0 tx=0).stop 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 shutdown_connections 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833b9c580 0x561833b9e970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b8bdb0 0x561833b97d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b90da0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 --2- 172.21.15.57:0/931889621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b86250 0x561833b90860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6530c00640 0 client.17553 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.859+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 >> 172.21.15.57:0/931889621 conn(0x561833b817c0 msgr2=0x561833b80a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.860+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 shutdown_connections 2024-04-13T05:28:04.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.860+0000 7f6550c34740 1 -- 172.21.15.57:0/931889621 wait complete. 2024-04-13T05:28:04.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.860+0000 7f6550c34740 20 client.17553 trim_cache size 0 max 16384 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 Processor -- start 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 -- start start 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86250 0x561833b801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833b80730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b3b10 con 0x561833b86c40 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b1320 con 0x561833b8bdb0 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6550c34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65481af170 con 0x561833b86250 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6546000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6546000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b97d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44444/0 (socket says 172.21.15.57:44444) 2024-04-13T05:28:04.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.875+0000 7f6546000640 1 -- 172.21.15.57:0/4277427882 learned_addr learned my addr 172.21.15.57:0/4277427882 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546a00640 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86250 0x561833b801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6547400640 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833b80730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546000640 1 -- 172.21.15.57:0/4277427882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86250 msgr2=0x561833b801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546000640 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86250 0x561833b801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546000640 1 -- 172.21.15.57:0/4277427882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 msgr2=0x561833b80730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546000640 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833b80730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546000640 1 -- 172.21.15.57:0/4277427882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561833b910f0 con 0x561833b86c40 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6546000640 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b97d30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f653400a010 tx=0x7f6534005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd69f93a7b34f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6533400640 1 -- 172.21.15.57:0/4277427882 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534039070 con 0x561833b86c40 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6533400640 1 -- 172.21.15.57:0/4277427882 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534017040 con 0x561833b86c40 2024-04-13T05:28:04.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.876+0000 7f6533400640 1 -- 172.21.15.57:0/4277427882 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534021070 con 0x561833b86c40 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 -- 172.21.15.57:0/4277427882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 msgr2=0x561833b97d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b97d30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f653400a010 tx=0x7f6534005a70 comp rx=0 tx=0).stop 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 -- 172.21.15.57:0/4277427882 shutdown_connections 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833b80730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86c40 0x561833b97d30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 --2- 172.21.15.57:0/4277427882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86250 0x561833b801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 -- 172.21.15.57:0/4277427882 >> 172.21.15.57:0/4277427882 conn(0x561833b817c0 msgr2=0x561833b88220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 -- 172.21.15.57:0/4277427882 shutdown_connections 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.877+0000 7f6550c34740 1 -- 172.21.15.57:0/4277427882 wait complete. 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 -- start start 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 Processor -- start 2024-04-13T05:28:04.883 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b94ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86c40 0x561833b950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833c00aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f65481af7e0 con 0x561833b86250 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65481b0c50 con 0x561833b8bdb0 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65481afeb0 con 0x561833b86c40 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.878+0000 7f6550c34740 10 client.0 _ll_register_callbacks cb 0x7f65340054d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b94ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b94ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44458/0 (socket says 172.21.15.57:44458) 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 -- 172.21.15.57:0/4031610000 learned_addr learned my addr 172.21.15.57:0/4031610000 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546000640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86c40 0x561833b950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6547400640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833c00aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 -- 172.21.15.57:0/4031610000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86c40 msgr2=0x561833b950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86c40 0x561833b950e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 -- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 msgr2=0x561833c00aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:04.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833c00aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 -- 172.21.15.57:0/4031610000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561833c00fe0 con 0x561833b86250 2024-04-13T05:28:04.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6546a00640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b94ba0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6538006010 tx=0x7f653802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da46901e3428b4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:04.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.879+0000 7f6530c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6530c00640 1 -- 172.21.15.57:0/4031610000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538039040 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6530c00640 1 -- 172.21.15.57:0/4031610000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f6538003b70 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6534017040 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6530c00640 1 -- 172.21.15.57:0/4031610000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6538061050 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6550c34740 10 client.17559 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561833c02250 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6530c00640 1 -- 172.21.15.57:0/4031610000 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534017040 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.880+0000 7f6530c00640 1 -- 172.21.15.57:0/4031610000 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f6538004270 con 0x561833b86250 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6550c34740 10 client.17559 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6550c34740 10 client.17559 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-13T05:28:04.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6550c34740 20 client.17559 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6550c34740 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833c02c30 0x561833c05050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6550c34740 4 client.17559 mds_command: new command op to 34457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 --> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561833af9a40 con 0x561833c02c30 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.881+0000 7f6546000640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833c02c30 0x561833c05050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.882+0000 7f6546000640 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833c02c30 0x561833c05050 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a77227fcd1787d6f server_cookie=da123186589d948e in_seq=0 out_seq=0 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.882+0000 7f6530c00640 10 client.17559 ms_handle_connect on v2:172.21.15.83:6836/741118790 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6530c00640 1 -- 172.21.15.57:0/4031610000 <== mds.1 v2:172.21.15.83:6836/741118790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561833af9a40 con 0x561833c02c30 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6530c00640 10 client.17559 handle_command_reply: tid=0 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 client.17559 shutdown 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 msgr2=0x561833b94ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b94ba0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6538006010 tx=0x7f653802c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 shutdown_connections 2024-04-13T05:28:04.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:6836/741118790,v1:172.21.15.83:6837/741118790] conn(0x561833c02c30 0x561833c05050 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561833b8bdb0 0x561833c00aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561833b86c40 0x561833b950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6530c00640 0 client.17559 ms_handle_reset on v2:172.21.15.83:6836/741118790 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 --2- 172.21.15.57:0/4031610000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561833b86250 0x561833b94ba0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 >> 172.21.15.57:0/4031610000 conn(0x561833b817c0 msgr2=0x561833b84a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 shutdown_connections 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 1 -- 172.21.15.57:0/4031610000 wait complete. 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.883+0000 7f6550c34740 20 client.17559 trim_cache size 0 max 16384 2024-04-13T05:28:04.890 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:28:04.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f652c0a3c30 msgr2=0x7f652c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f652c0a3c30 0x7f652c0a60f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f653802c630 tx=0x7f653802e040 comp rx=0 tx=0).stop 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 msgr2=0x7f65481a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 0x7f65481a5e40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f654000e970 tx=0x7f654000ed70 comp rx=0 tx=0).stop 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 shutdown_connections 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f652c0a3c30 0x7f652c0a60f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6548109810 0x7f654819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f654810a1d0 0x7f65481a5e40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 --2- 172.21.15.57:0/2288216275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6548108620 0x7f654819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 >> 172.21.15.57:0/2288216275 conn(0x7f65480775c0 msgr2=0x7f654810cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 shutdown_connections 2024-04-13T05:28:04.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:04.884+0000 7f654dc00640 1 -- 172.21.15.57:0/2288216275 wait complete. 2024-04-13T05:28:04.907 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-13T05:28:05.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.132+0000 7f7bd5200640 1 -- 172.21.15.57:0/3349556909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 msgr2=0x7f7bd0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.132+0000 7f7bd5200640 1 --2- 172.21.15.57:0/3349556909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0102510 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009900 tx=0x7f7bb802bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 -- 172.21.15.57:0/3349556909 shutdown_connections 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 --2- 172.21.15.57:0/3349556909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd0102a50 0x7f7bd0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 --2- 172.21.15.57:0/3349556909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0102510 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 --2- 172.21.15.57:0/3349556909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bd0100eb0 0x7f7bd01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 -- 172.21.15.57:0/3349556909 >> 172.21.15.57:0/3349556909 conn(0x7f7bd00fc470 msgr2=0x7f7bd00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 -- 172.21.15.57:0/3349556909 shutdown_connections 2024-04-13T05:28:05.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.133+0000 7f7bd5200640 1 -- 172.21.15.57:0/3349556909 wait complete. 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 Processor -- start 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 -- start start 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd0100eb0 0x7f7bd010b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bd0102a50 0x7f7bd010a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010db90 con 0x7f7bd0102090 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010da10 con 0x7f7bd0100eb0 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.134+0000 7f7bd5200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010dd10 con 0x7f7bd0102a50 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44478/0 (socket says 172.21.15.57:44478) 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 -- 172.21.15.57:0/2085778143 learned_addr learned my addr 172.21.15.57:0/2085778143 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:05.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bce800640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bd0102a50 0x7f7bd010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcde00640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd0100eb0 0x7f7bd010b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 -- 172.21.15.57:0/2085778143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bd0102a50 msgr2=0x7f7bd010a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bd0102a50 0x7f7bd010a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 -- 172.21.15.57:0/2085778143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd0100eb0 msgr2=0x7f7bd010b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd0100eb0 0x7f7bd010b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 -- 172.21.15.57:0/2085778143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd010acc0 con 0x7f7bd0102090 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.135+0000 7f7bcd400640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0109ef0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8006010 tx=0x7f7bb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.136+0000 7f7bc6a00640 1 -- 172.21.15.57:0/2085778143 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8043020 con 0x7f7bd0102090 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.136+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd01a23d0 con 0x7f7bd0102090 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.136+0000 7f7bc6a00640 1 -- 172.21.15.57:0/2085778143 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bb8004aa0 con 0x7f7bd0102090 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.136+0000 7f7bc6a00640 1 -- 172.21.15.57:0/2085778143 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8003c60 con 0x7f7bd0102090 2024-04-13T05:28:05.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.136+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd01a2920 con 0x7f7bd0102090 2024-04-13T05:28:05.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.139+0000 7f7bc6a00640 1 -- 172.21.15.57:0/2085778143 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7bb8046020 con 0x7f7bd0102090 2024-04-13T05:28:05.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 Processor -- start 2024-04-13T05:28:05.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 -- start start 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc364670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc369740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc36c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc2ca020 con 0x55c0fc369db0 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc270950 con 0x55c0fc364250 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.140+0000 7f7bd8171740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc2c5fe0 con 0x55c0fc364c40 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc36c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc36c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44482/0 (socket says 172.21.15.57:44482) 2024-04-13T05:28:05.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 -- 172.21.15.57:0/750638374 learned_addr learned my addr 172.21.15.57:0/750638374 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:05.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcde00640 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc369740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bce800640 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc364670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bc6a00640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7bb40a3bb0 0x7f7bb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bc6a00640 1 -- 172.21.15.57:0/2085778143 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7bb80f2c00 con 0x7f7bd0102090 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 -- 172.21.15.57:0/750638374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 msgr2=0x55c0fc369740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc369740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 -- 172.21.15.57:0/750638374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 msgr2=0x55c0fc364670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc364670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bcd400640 1 -- 172.21.15.57:0/750638374 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fbfdc3f0 con 0x55c0fc369db0 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.141+0000 7f7bce800640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7bb40a3bb0 0x7f7bb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.142+0000 7f7bcd400640 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc36c240 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80f0360 tx=0x7f7bb802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4605742a8806a235 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.142+0000 7f7bbf400640 1 -- 172.21.15.57:0/750638374 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80be910 con 0x55c0fc369db0 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.142+0000 7f7bbf400640 1 -- 172.21.15.57:0/750638374 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bb80035f0 con 0x55c0fc369db0 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.142+0000 7f7bbf400640 1 -- 172.21.15.57:0/750638374 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80be4a0 con 0x55c0fc369db0 2024-04-13T05:28:05.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 -- 172.21.15.57:0/750638374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 msgr2=0x55c0fc36c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bce800640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7bb40a3bb0 0x7f7bb40a6070 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c0fc3654a0 tx=0x7f7bc80046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc36c240 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80f0360 tx=0x7f7bb802bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 -- 172.21.15.57:0/750638374 shutdown_connections 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc36c240 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc369740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 --2- 172.21.15.57:0/750638374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc364670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 -- 172.21.15.57:0/750638374 >> 172.21.15.57:0/750638374 conn(0x55c0fc35f7c0 msgr2=0x55c0fc35fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 -- 172.21.15.57:0/750638374 shutdown_connections 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 -- 172.21.15.57:0/750638374 wait complete. 2024-04-13T05:28:05.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.143+0000 7f7bd8171740 1 -- start start 2024-04-13T05:28:05.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 Processor -- start 2024-04-13T05:28:05.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc36e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc36ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc375a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc36cb20 con 0x55c0fc369db0 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc283c80 con 0x55c0fc364250 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc36cca0 con 0x55c0fc364c40 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bd8171740 10 client.0 _ll_register_callbacks cb 0x55c0fc283670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bce800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc375a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bce800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc375a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44494/0 (socket says 172.21.15.57:44494) 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.144+0000 7f7bce800640 1 -- 172.21.15.57:0/1894451313 learned_addr learned my addr 172.21.15.57:0/1894451313 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bcde00640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc36e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bcd400640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc36ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bce800640 1 -- 172.21.15.57:0/1894451313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 msgr2=0x55c0fc36ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bce800640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc36ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bce800640 1 -- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 msgr2=0x55c0fc36e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bce800640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc36e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bce800640 1 -- 172.21.15.57:0/1894451313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fc0a67d0 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bce800640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc375a80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8012670 tx=0x7f7bc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161ebe7d205d5f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.145+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1894451313 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801e920 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1894451313 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7bc801eac0 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c0fc269750 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1894451313 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7bc801ecc0 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bd8171740 10 client.17571 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c0fc378ec0 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1894451313 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c0fc269750 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1894451313 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7bc801b4d0 con 0x55c0fc369db0 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bd8171740 10 client.17571 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bd8171740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:28:05.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.146+0000 7f7bd8171740 20 client.17571 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:05.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.147+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc37a610 0x55c0fc37cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.147+0000 7f7bd8171740 4 client.17571 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-13T05:28:05.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.147+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0fc2d7a40 con 0x55c0fc37a610 2024-04-13T05:28:05.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.147+0000 7f7bcde00640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc37a610 0x55c0fc37cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.148+0000 7f7bcde00640 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc37a610 0x55c0fc37cab0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d4552ec0fa12d8 server_cookie=2d7ec62c060e7eaf in_seq=0 out_seq=0 2024-04-13T05:28:05.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.148+0000 7f7bb3e00640 10 client.17571 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:05.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1894451313 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0fc2d7a40 con 0x55c0fc37a610 2024-04-13T05:28:05.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bb3e00640 10 client.17571 handle_command_reply: tid=0 2024-04-13T05:28:05.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 client.17571 shutdown 2024-04-13T05:28:05.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 msgr2=0x55c0fc375a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc375a80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8012670 tx=0x7f7bc8004730 comp rx=0 tx=0).stop 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 shutdown_connections 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc37a610 0x55c0fc37cab0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc369db0 0x55c0fc375a80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364c40 0x55c0fc36ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1894451313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364250 0x55c0fc36e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 >> 172.21.15.57:0/1894451313 conn(0x55c0fc35f7c0 msgr2=0x55c0fc35ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bb3e00640 0 client.17571 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 shutdown_connections 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 1 -- 172.21.15.57:0/1894451313 wait complete. 2024-04-13T05:28:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.150+0000 7f7bd8171740 20 client.17571 trim_cache size 0 max 16384 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.165+0000 7f7bd8171740 1 Processor -- start 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.165+0000 7f7bd8171740 1 -- start start 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364250 0x55c0fc363130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 0x55c0fc375a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc369db0 0x55c0fc381370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bd8171740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01af790 con 0x55c0fc364c40 2024-04-13T05:28:05.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bd8171740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01acfa0 con 0x55c0fc369db0 2024-04-13T05:28:05.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bd8171740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01aadf0 con 0x55c0fc364250 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bcd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 0x55c0fc375a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bcd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 0x55c0fc375a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44510/0 (socket says 172.21.15.57:44510) 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bcd400640 1 -- 172.21.15.57:0/3023847903 learned_addr learned my addr 172.21.15.57:0/3023847903 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.166+0000 7f7bce800640 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc369db0 0x55c0fc381370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bcd400640 1 -- 172.21.15.57:0/3023847903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364250 msgr2=0x55c0fc363130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bcd400640 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364250 0x55c0fc363130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bcd400640 1 -- 172.21.15.57:0/3023847903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc369db0 msgr2=0x55c0fc381370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bcd400640 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc369db0 0x55c0fc381370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bcd400640 1 -- 172.21.15.57:0/3023847903 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fc378cb0 con 0x55c0fc364c40 2024-04-13T05:28:05.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bcd400640 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 0x55c0fc375a40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005ab0 tx=0x7f7bb80f6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9099165e269c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:05.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.167+0000 7f7bbea00640 1 -- 172.21.15.57:0/3023847903 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80be680 con 0x55c0fc364c40 2024-04-13T05:28:05.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bbea00640 1 -- 172.21.15.57:0/3023847903 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bb80d8070 con 0x55c0fc364c40 2024-04-13T05:28:05.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bbea00640 1 -- 172.21.15.57:0/3023847903 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80c77b0 con 0x55c0fc364c40 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 -- 172.21.15.57:0/3023847903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 msgr2=0x55c0fc375a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 0x55c0fc375a40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005ab0 tx=0x7f7bb80f6790 comp rx=0 tx=0).stop 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 -- 172.21.15.57:0/3023847903 shutdown_connections 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc369db0 0x55c0fc381370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364c40 0x55c0fc375a40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 --2- 172.21.15.57:0/3023847903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc364250 0x55c0fc363130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 -- 172.21.15.57:0/3023847903 >> 172.21.15.57:0/3023847903 conn(0x55c0fc35f7c0 msgr2=0x55c0fc362a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 -- 172.21.15.57:0/3023847903 shutdown_connections 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.168+0000 7f7bd8171740 1 -- 172.21.15.57:0/3023847903 wait complete. 2024-04-13T05:28:05.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 -- start start 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 Processor -- start 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 0x55c0fc373340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364c40 0x55c0fc373880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc369db0 0x55c0fc374890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ab460 con 0x55c0fc364250 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ac8d0 con 0x55c0fc364c40 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01abb30 con 0x55c0fc369db0 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bd8171740 10 client.0 _ll_register_callbacks cb 0x7f7bc800bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.169+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 0x55c0fc373340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 0x55c0fc373340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44520/0 (socket says 172.21.15.57:44520) 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 -- 172.21.15.57:0/1606907513 learned_addr learned my addr 172.21.15.57:0/1606907513 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 -- 172.21.15.57:0/1606907513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc369db0 msgr2=0x55c0fc374890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc369db0 0x55c0fc374890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 -- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364c40 msgr2=0x55c0fc373880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364c40 0x55c0fc373880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 -- 172.21.15.57:0/1606907513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fc374e80 con 0x55c0fc364250 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bcde00640 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 0x55c0fc373340 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009e70 tx=0x7f7bc0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca516d22269711b server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.170+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1606907513 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc001f690 con 0x55c0fc364250 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1606907513 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f7bc001f830 con 0x55c0fc364250 2024-04-13T05:28:05.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb80d8070 con 0x55c0fc364250 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1606907513 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7bc002d050 con 0x55c0fc364250 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 10 client.17577 fetch_fsmap learned FSMap version 33 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c0fc3e96a0 con 0x55c0fc364250 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1606907513 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bb80d8070 con 0x55c0fc364250 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1606907513 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7bc000f040 con 0x55c0fc364250 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 10 client.17577 fetch_fsmap finished waiting for FSMap version 33 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-13T05:28:05.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 20 client.17577 populate_metadata read hostname 'smithi057' 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc3ea130 0x55c0fc3ec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 4 client.17577 mds_command: new command op to 34499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.171+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 --> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0fc2d18b0 con 0x55c0fc3ea130 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.172+0000 7f7bcd400640 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc3ea130 0x55c0fc3ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.173+0000 7f7bcd400640 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc3ea130 0x55c0fc3ec570 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b64d94da65a28b6 server_cookie=6a27957270ec7fbb in_seq=0 out_seq=0 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.173+0000 7f7bb3e00640 10 client.17577 ms_handle_connect on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bb3e00640 1 -- 172.21.15.57:0/1606907513 <== mds.0 v2:172.21.15.83:6838/3095811120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0fc2d18b0 con 0x55c0fc3ea130 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bb3e00640 10 client.17577 handle_command_reply: tid=0 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 client.17577 shutdown 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 msgr2=0x55c0fc373340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 0x55c0fc373340 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009e70 tx=0x7f7bc0004410 comp rx=0 tx=0).stop 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 shutdown_connections 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:6838/3095811120,v1:172.21.15.83:6839/3095811120] conn(0x55c0fc3ea130 0x55c0fc3ec570 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0fc364c40 0x55c0fc373880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0fc369db0 0x55c0fc374890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 --2- 172.21.15.57:0/1606907513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c0fc364250 0x55c0fc373340 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 >> 172.21.15.57:0/1606907513 conn(0x55c0fc35f7c0 msgr2=0x55c0fc3632b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bb3e00640 0 client.17577 ms_handle_reset on v2:172.21.15.83:6838/3095811120 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 shutdown_connections 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 1 -- 172.21.15.57:0/1606907513 wait complete. 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.174+0000 7f7bd8171740 20 client.17577 trim_cache size 0 max 16384 2024-04-13T05:28:05.179 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7bb40a3bb0 msgr2=0x7f7bb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7bb40a3bb0 0x7f7bb40a6070 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c0fc3654a0 tx=0x7f7bc80046c0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 msgr2=0x7f7bd0109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0109ef0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8006010 tx=0x7f7bb80042f0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 shutdown_connections 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f7bb40a3bb0 0x7f7bb40a6070 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bd0102a50 0x7f7bd010a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bd0102090 0x7f7bd0109ef0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 --2- 172.21.15.57:0/2085778143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd0100eb0 0x7f7bd010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 >> 172.21.15.57:0/2085778143 conn(0x7f7bd00fc470 msgr2=0x7f7bd0105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 shutdown_connections 2024-04-13T05:28:05.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:05.175+0000 7f7bd5200640 1 -- 172.21.15.57:0/2085778143 wait complete. 2024-04-13T05:28:06.196 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-13T05:28:06.196 INFO:tasks.fwd_scrub:done joining 2024-04-13T05:28:06.196 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-13T05:28:06.208 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 -- 172.21.15.57:0/1603053027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109630 msgr2=0x7f1938109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 --2- 172.21.15.57:0/1603053027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109630 0x7f1938109ab0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402bc40 comp rx=0 tx=0).stop 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 -- 172.21.15.57:0/1603053027 shutdown_connections 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 --2- 172.21.15.57:0/1603053027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938109ff0 0x7f1938110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 --2- 172.21.15.57:0/1603053027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109630 0x7f1938109ab0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 --2- 172.21.15.57:0/1603053027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938108440 0x7f1938108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 -- 172.21.15.57:0/1603053027 >> 172.21.15.57:0/1603053027 conn(0x7f1938077400 msgr2=0x7f1938079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:06.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 -- 172.21.15.57:0/1603053027 shutdown_connections 2024-04-13T05:28:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.444+0000 7f1937200640 1 -- 172.21.15.57:0/1603053027 wait complete. 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.445+0000 7f1937200640 1 Processor -- start 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.445+0000 7f1937200640 1 -- start start 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.445+0000 7f1937200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938108440 0x7f193819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1937200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109630 0x7f193819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1937200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 0x7f19381a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1937200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1938113fd0 con 0x7f1938109ff0 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1937200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1938113e50 con 0x7f1938109630 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1937200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1938114150 con 0x7f1938108440 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 0x7f19381a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 0x7f19381a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44544/0 (socket says 172.21.15.57:44544) 2024-04-13T05:28:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 -- 172.21.15.57:0/2971223571 learned_addr learned my addr 172.21.15.57:0/2971223571 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1935e00640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938108440 0x7f193819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1935400640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109630 0x7f193819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 -- 172.21.15.57:0/2971223571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938108440 msgr2=0x7f193819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938108440 0x7f193819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 -- 172.21.15.57:0/2971223571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109630 msgr2=0x7f193819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.446+0000 7f1936800640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109630 0x7f193819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f1936800640 1 -- 172.21.15.57:0/2971223571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381a61a0 con 0x7f1938109ff0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f1935400640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109630 0x7f193819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f1935e00640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938108440 0x7f193819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f1936800640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 0x7f19381a5b30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f193000cce0 tx=0x7f193000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193000f450 con 0x7f1938109ff0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f193000faa0 con 0x7f1938109ff0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19381a6490 con 0x7f1938109ff0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930018b70 con 0x7f1938109ff0 2024-04-13T05:28:06.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.447+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381a6990 con 0x7f1938109ff0 2024-04-13T05:28:06.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.449+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193810f700 con 0x7f1938109ff0 2024-04-13T05:28:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.451+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1930018d10 con 0x7f1938109ff0 2024-04-13T05:28:06.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.453+0000 7f192ea00640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f191c0a3bb0 0x7f191c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.454+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f19300d0da0 con 0x7f1938109ff0 2024-04-13T05:28:06.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.454+0000 7f1935e00640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f191c0a3bb0 0x7f191c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.454+0000 7f1935e00640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f191c0a3bb0 0x7f191c0a6070 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1938109490 tx=0x7f192800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.456+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1930014030 con 0x7f1938109ff0 2024-04-13T05:28:06.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.663+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1938062bc0 con 0x7f1938109ff0 2024-04-13T05:28:06.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.665+0000 7f192ea00640 1 -- 172.21.15.57:0/2971223571 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f193009d410 con 0x7f1938109ff0 2024-04-13T05:28:06.668 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:28:06.668 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f191c0a3bb0 msgr2=0x7f191c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f191c0a3bb0 0x7f191c0a6070 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1938109490 tx=0x7f192800a5e0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 msgr2=0x7f19381a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 0x7f19381a5b30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f193000cce0 tx=0x7f193000a620 comp rx=0 tx=0).stop 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 shutdown_connections 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f191c0a3bb0 0x7f191c0a6070 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938109630 0x7f193819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938109ff0 0x7f19381a5b30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 --2- 172.21.15.57:0/2971223571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1938108440 0x7f193819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 >> 172.21.15.57:0/2971223571 conn(0x7f1938077400 msgr2=0x7f193810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 shutdown_connections 2024-04-13T05:28:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.668+0000 7f1937200640 1 -- 172.21.15.57:0/2971223571 wait complete. 2024-04-13T05:28:06.690 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:28:06.691 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:28:06.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 -- 172.21.15.57:0/2044609761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 msgr2=0x7fa980104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 --2- 172.21.15.57:0/2044609761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980104d90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa974009ca0 tx=0x7fa97402c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 -- 172.21.15.57:0/2044609761 shutdown_connections 2024-04-13T05:28:06.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 --2- 172.21.15.57:0/2044609761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa980107c00 0x7fa98010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 --2- 172.21.15.57:0/2044609761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 0x7fa9801076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 --2- 172.21.15.57:0/2044609761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980104d90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 -- 172.21.15.57:0/2044609761 >> 172.21.15.57:0/2044609761 conn(0x7fa9800fc390 msgr2=0x7fa9800fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:06.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 -- 172.21.15.57:0/2044609761 shutdown_connections 2024-04-13T05:28:06.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.920+0000 7fa986800640 1 -- 172.21.15.57:0/2044609761 wait complete. 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.921+0000 7fa986800640 1 Processor -- start 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 -- start start 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 0x7fa9801986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa980107c00 0x7fa98019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010da60 con 0x7fa9801029a0 2024-04-13T05:28:06.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010d8e0 con 0x7fa980107c00 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa986800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010dbe0 con 0x7fa9801052d0 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa985400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa985400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44570/0 (socket says 172.21.15.57:44570) 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.922+0000 7fa985400640 1 -- 172.21.15.57:0/3628996010 learned_addr learned my addr 172.21.15.57:0/3628996010 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985e00640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa980107c00 0x7fa98019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa984a00640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 0x7fa9801986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985400640 1 -- 172.21.15.57:0/3628996010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 msgr2=0x7fa9801986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985400640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 0x7fa9801986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985400640 1 -- 172.21.15.57:0/3628996010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa980107c00 msgr2=0x7fa98019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985400640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa980107c00 0x7fa98019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985400640 1 -- 172.21.15.57:0/3628996010 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98019fd80 con 0x7fa9801029a0 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa984a00640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 0x7fa9801986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa985400640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980198160 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa974009dc0 tx=0x7fa9740328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:06.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974041020 con 0x7fa9801029a0 2024-04-13T05:28:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa974003e20 con 0x7fa9801029a0 2024-04-13T05:28:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974039040 con 0x7fa9801029a0 2024-04-13T05:28:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.923+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9801a0010 con 0x7fa9801029a0 2024-04-13T05:28:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.924+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9801a0550 con 0x7fa9801029a0 2024-04-13T05:28:06.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.925+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9800fde80 con 0x7fa9801029a0 2024-04-13T05:28:06.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.927+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa974044020 con 0x7fa9801029a0 2024-04-13T05:28:06.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.928+0000 7fa97e000640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9680a3c30 0x7fa9680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:06.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.928+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fa9740f2b20 con 0x7fa9801029a0 2024-04-13T05:28:06.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.929+0000 7fa984a00640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9680a3c30 0x7fa9680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:06.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.929+0000 7fa984a00640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9680a3c30 0x7fa9680a60f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa980199670 tx=0x7fa96c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:06.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:06.933+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa97404b020 con 0x7fa9801029a0 2024-04-13T05:28:07.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.141+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa980062bc0 con 0x7fa9801029a0 2024-04-13T05:28:07.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.142+0000 7fa97e000640 1 -- 172.21.15.57:0/3628996010 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fa9740bf100 con 0x7fa9801029a0 2024-04-13T05:28:07.145 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9680a3c30 msgr2=0x7fa9680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9680a3c30 0x7fa9680a60f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa980199670 tx=0x7fa96c005e50 comp rx=0 tx=0).stop 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 msgr2=0x7fa980198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980198160 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa974009dc0 tx=0x7fa9740328a0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 shutdown_connections 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa9680a3c30 0x7fa9680a60f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa980107c00 0x7fa98019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9801052d0 0x7fa9801986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 --2- 172.21.15.57:0/3628996010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9801029a0 0x7fa980198160 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 >> 172.21.15.57:0/3628996010 conn(0x7fa9800fc390 msgr2=0x7fa9801033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 shutdown_connections 2024-04-13T05:28:07.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.145+0000 7fa986800640 1 -- 172.21.15.57:0/3628996010 wait complete. 2024-04-13T05:28:07.169 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:28:07.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:06 smithi083 ceph-mon[35133]: pgmap v771: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:07.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:06 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2971223571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:28:07.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:06 smithi057 ceph-mon[32331]: pgmap v771: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:07.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:06 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2971223571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:28:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:06 smithi189 ceph-mon[35531]: pgmap v771: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:07.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:06 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2971223571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:28:07.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.396+0000 7f4b62200640 1 -- 172.21.15.57:0/1945319590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c1011f0 msgr2=0x7f4b5c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.396+0000 7f4b62200640 1 --2- 172.21.15.57:0/1945319590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c101610 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009ca0 tx=0x7f4b4c02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.396+0000 7f4b62200640 1 -- 172.21.15.57:0/1945319590 shutdown_connections 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.396+0000 7f4b62200640 1 --2- 172.21.15.57:0/1945319590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c100680 0x7f4b5c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.396+0000 7f4b62200640 1 --2- 172.21.15.57:0/1945319590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c0ffcc0 0x7f4b5c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.396+0000 7f4b62200640 1 --2- 172.21.15.57:0/1945319590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c101610 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.397+0000 7f4b62200640 1 -- 172.21.15.57:0/1945319590 >> 172.21.15.57:0/1945319590 conn(0x7f4b5c0fb7c0 msgr2=0x7f4b5c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.397+0000 7f4b62200640 1 -- 172.21.15.57:0/1945319590 shutdown_connections 2024-04-13T05:28:07.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.397+0000 7f4b62200640 1 -- 172.21.15.57:0/1945319590 wait complete. 2024-04-13T05:28:07.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 Processor -- start 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 -- start start 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c0ffcc0 0x7f4b5c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 0x7f4b5c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10fd50 con 0x7f4b5c100680 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10fbd0 con 0x7f4b5c0ffcc0 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b62200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10fed0 con 0x7f4b5c1011f0 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b52a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 0x7f4b5c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b52a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 0x7f4b5c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44582/0 (socket says 172.21.15.57:44582) 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.398+0000 7f4b52a00640 1 -- 172.21.15.57:0/2663266508 learned_addr learned my addr 172.21.15.57:0/2663266508 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b5b400640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c0ffcc0 0x7f4b5c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b5be00640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b52a00640 1 -- 172.21.15.57:0/2663266508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c1011f0 msgr2=0x7f4b5c1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b52a00640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b52a00640 1 -- 172.21.15.57:0/2663266508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c0ffcc0 msgr2=0x7f4b5c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b52a00640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c0ffcc0 0x7f4b5c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b52a00640 1 -- 172.21.15.57:0/2663266508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1a20a0 con 0x7f4b5c100680 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b5be00640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.399+0000 7f4b52a00640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 0x7f4b5c19aa10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d950 tx=0x7f4b4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.400+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44017070 con 0x7f4b5c100680 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.400+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1a2390 con 0x7f4b5c100680 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.400+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b44009c30 con 0x7f4b5c100680 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.400+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44021060 con 0x7f4b5c100680 2024-04-13T05:28:07.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.400+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a2890 con 0x7f4b5c100680 2024-04-13T05:28:07.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.401+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c10c5f0 con 0x7f4b5c100680 2024-04-13T05:28:07.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.403+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b4401a070 con 0x7f4b5c100680 2024-04-13T05:28:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.404+0000 7f4b58c00640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4b400a3bb0 0x7f4b400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.404+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f4b440d03f0 con 0x7f4b5c100680 2024-04-13T05:28:07.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.405+0000 7f4b5b400640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4b400a3bb0 0x7f4b400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.405+0000 7f4b5b400640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4b400a3bb0 0x7f4b400a6070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c006010 tx=0x7f4b4c032510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:07.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.410+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b44009de0 con 0x7f4b5c100680 2024-04-13T05:28:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.626+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b5c062bf0 con 0x7f4b5c100680 2024-04-13T05:28:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.627+0000 7f4b58c00640 1 -- 172.21.15.57:0/2663266508 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f4b4409c9d0 con 0x7f4b5c100680 2024-04-13T05:28:07.629 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.629+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4b400a3bb0 msgr2=0x7f4b400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.629+0000 7f4b62200640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4b400a3bb0 0x7f4b400a6070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c006010 tx=0x7f4b4c032510 comp rx=0 tx=0).stop 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.629+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 msgr2=0x7f4b5c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.629+0000 7f4b62200640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 0x7f4b5c19aa10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d950 tx=0x7f4b4400de10 comp rx=0 tx=0).stop 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 shutdown_connections 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f4b400a3bb0 0x7f4b400a6070 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b5c1011f0 0x7f4b5c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b5c100680 0x7f4b5c19aa10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 --2- 172.21.15.57:0/2663266508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c0ffcc0 0x7f4b5c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 >> 172.21.15.57:0/2663266508 conn(0x7f4b5c0fb7c0 msgr2=0x7f4b5c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 shutdown_connections 2024-04-13T05:28:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.630+0000 7f4b62200640 1 -- 172.21.15.57:0/2663266508 wait complete. 2024-04-13T05:28:07.653 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:28:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 -- 172.21.15.57:0/4174571007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 msgr2=0x7fc848105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 --2- 172.21.15.57:0/4174571007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc848105890 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc834009900 tx=0x7fc83402bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:07.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 -- 172.21.15.57:0/4174571007 shutdown_connections 2024-04-13T05:28:07.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 --2- 172.21.15.57:0/4174571007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848105dd0 0x7fc84810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 --2- 172.21.15.57:0/4174571007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc848105890 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 --2- 172.21.15.57:0/4174571007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc848104220 0x7fc848104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 -- 172.21.15.57:0/4174571007 >> 172.21.15.57:0/4174571007 conn(0x7fc8480ff7b0 msgr2=0x7fc848101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:07.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 -- 172.21.15.57:0/4174571007 shutdown_connections 2024-04-13T05:28:07.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.878+0000 7fc850400640 1 -- 172.21.15.57:0/4174571007 wait complete. 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.879+0000 7fc850400640 1 Processor -- start 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 -- start start 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848104220 0x7fc84819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc84819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc848105dd0 0x7fc8481a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810fd60 con 0x7fc848105410 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810fbe0 con 0x7fc848104220 2024-04-13T05:28:07.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc850400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810fee0 con 0x7fc848105dd0 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc84d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc84819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc84d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc84819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44596/0 (socket says 172.21.15.57:44596) 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc84d000640 1 -- 172.21.15.57:0/3180513604 learned_addr learned my addr 172.21.15.57:0/3180513604 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc84da00640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848104220 0x7fc84819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.880+0000 7fc84e400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc848105dd0 0x7fc8481a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84d000640 1 -- 172.21.15.57:0/3180513604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc848105dd0 msgr2=0x7fc8481a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84d000640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc848105dd0 0x7fc8481a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84d000640 1 -- 172.21.15.57:0/3180513604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848104220 msgr2=0x7fc84819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84d000640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848104220 0x7fc84819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84d000640 1 -- 172.21.15.57:0/3180513604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a2090 con 0x7fc848105410 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84da00640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848104220 0x7fc84819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc84d000640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc84819a9b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc834006010 tx=0x7fc8340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:07.885 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.881+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834043020 con 0x7fc848105410 2024-04-13T05:28:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.882+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a2320 con 0x7fc848105410 2024-04-13T05:28:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.882+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc834004aa0 con 0x7fc848105410 2024-04-13T05:28:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.882+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834003c60 con 0x7fc848105410 2024-04-13T05:28:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.882+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a2870 con 0x7fc848105410 2024-04-13T05:28:07.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.883+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84810b660 con 0x7fc848105410 2024-04-13T05:28:07.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.885+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc834046020 con 0x7fc848105410 2024-04-13T05:28:07.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.886+0000 7fc846000640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc8300a3bb0 0x7fc8300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:07.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.887+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc8340f32d0 con 0x7fc848105410 2024-04-13T05:28:07.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.887+0000 7fc84da00640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc8300a3bb0 0x7fc8300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:07.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.888+0000 7fc84da00640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc8300a3bb0 0x7fc8300a6070 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0058c0 tx=0x7fc83c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:07.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:07.889+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc83404d020 con 0x7fc848105410 2024-04-13T05:28:08.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.096+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc84802d830 con 0x7fc848105410 2024-04-13T05:28:08.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.097+0000 7fc846000640 1 -- 172.21.15.57:0/3180513604 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fc8340bf8b0 con 0x7fc848105410 2024-04-13T05:28:08.100 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:28:08.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.099+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc8300a3bb0 msgr2=0x7fc8300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.099+0000 7fc850400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc8300a3bb0 0x7fc8300a6070 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0058c0 tx=0x7fc83c005850 comp rx=0 tx=0).stop 2024-04-13T05:28:08.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.099+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 msgr2=0x7fc84819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.099+0000 7fc850400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc84819a9b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc834006010 tx=0x7fc8340042f0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 shutdown_connections 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc8300a3bb0 0x7fc8300a6070 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc848105dd0 0x7fc8481a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc848105410 0x7fc84819a9b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 --2- 172.21.15.57:0/3180513604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc848104220 0x7fc84819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 >> 172.21.15.57:0/3180513604 conn(0x7fc8480ff7b0 msgr2=0x7fc848108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 shutdown_connections 2024-04-13T05:28:08.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.100+0000 7fc850400640 1 -- 172.21.15.57:0/3180513604 wait complete. 2024-04-13T05:28:08.122 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:28:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:08 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3628996010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:28:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:08 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2663266508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:28:08.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:08 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3628996010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:28:08.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:08 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2663266508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:28:08.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.351+0000 7fbf7d800640 1 -- 172.21.15.57:0/1078341062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 msgr2=0x7fbf78104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.351+0000 7fbf7d800640 1 --2- 172.21.15.57:0/1078341062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf78104640 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009900 tx=0x7fbf6802bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 -- 172.21.15.57:0/1078341062 shutdown_connections 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 --2- 172.21.15.57:0/1078341062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf78105dd0 0x7fbf7810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 --2- 172.21.15.57:0/1078341062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105410 0x7fbf78105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 --2- 172.21.15.57:0/1078341062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf78104640 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 -- 172.21.15.57:0/1078341062 >> 172.21.15.57:0/1078341062 conn(0x7fbf780ff7b0 msgr2=0x7fbf78101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 -- 172.21.15.57:0/1078341062 shutdown_connections 2024-04-13T05:28:08.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.352+0000 7fbf7d800640 1 -- 172.21.15.57:0/1078341062 wait complete. 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 Processor -- start 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 -- start start 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf7819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf78105410 0x7fbf7819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105dd0 0x7fbf781a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810fd60 con 0x7fbf78104220 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810fbe0 con 0x7fbf78105410 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.353+0000 7fbf7d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810fee0 con 0x7fbf78105dd0 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf7819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf7819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44606/0 (socket says 172.21.15.57:44606) 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 -- 172.21.15.57:0/2429025992 learned_addr learned my addr 172.21.15.57:0/2429025992 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:08.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf6fe00640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf78105410 0x7fbf7819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf77400640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105dd0 0x7fbf781a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 -- 172.21.15.57:0/2429025992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105dd0 msgr2=0x7fbf781a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105dd0 0x7fbf781a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 -- 172.21.15.57:0/2429025992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf78105410 msgr2=0x7fbf7819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf78105410 0x7fbf7819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 -- 172.21.15.57:0/2429025992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf781a20a0 con 0x7fbf78104220 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf77400640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105dd0 0x7fbf781a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:08.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.354+0000 7fbf76a00640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf7819a480 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:08.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.355+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68043020 con 0x7fbf78104220 2024-04-13T05:28:08.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.355+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf781a2330 con 0x7fbf78104220 2024-04-13T05:28:08.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.355+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf68004aa0 con 0x7fbf78104220 2024-04-13T05:28:08.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.355+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003d00 con 0x7fbf78104220 2024-04-13T05:28:08.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.355+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781a2880 con 0x7fbf78104220 2024-04-13T05:28:08.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.356+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7810b660 con 0x7fbf78104220 2024-04-13T05:28:08.361 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.359+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf68046020 con 0x7fbf78104220 2024-04-13T05:28:08.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.361+0000 7fbf6f400640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.361+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fbf680f2e60 con 0x7fbf78104220 2024-04-13T05:28:08.363 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.361+0000 7fbf6fe00640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.364 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.362+0000 7fbf6fe00640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005fd0 tx=0x7fbf60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:08.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.365+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf6804d020 con 0x7fbf78104220 2024-04-13T05:28:08.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:08 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3628996010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:28:08.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:08 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2663266508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:28:08.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.578+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbf78062bc0 con 0x7fbf78104220 2024-04-13T05:28:08.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.579+0000 7fbf6f400640 1 -- 172.21.15.57:0/2429025992 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fbf680bf440 con 0x7fbf78104220 2024-04-13T05:28:08.582 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:28:08.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.581+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbf5c0a3bb0 msgr2=0x7fbf5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.581+0000 7fbf7d800640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005fd0 tx=0x7fbf60005850 comp rx=0 tx=0).stop 2024-04-13T05:28:08.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 msgr2=0x7fbf7819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf7819a480 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf68004270 comp rx=0 tx=0).stop 2024-04-13T05:28:08.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 shutdown_connections 2024-04-13T05:28:08.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf78105410 0x7fbf7819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf78105dd0 0x7fbf781a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 --2- 172.21.15.57:0/2429025992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf78104220 0x7fbf7819a480 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 >> 172.21.15.57:0/2429025992 conn(0x7fbf780ff7b0 msgr2=0x7fbf78108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:08.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 shutdown_connections 2024-04-13T05:28:08.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.582+0000 7fbf7d800640 1 -- 172.21.15.57:0/2429025992 wait complete. 2024-04-13T05:28:08.606 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 -- 172.21.15.57:0/341741599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44104040 msgr2=0x7f9a44104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 --2- 172.21.15.57:0/341741599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44104040 0x7f9a44104460 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009900 tx=0x7f9a3402bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 -- 172.21.15.57:0/341741599 shutdown_connections 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 --2- 172.21.15.57:0/341741599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44105bf0 0x7f9a4410c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 --2- 172.21.15.57:0/341741599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a44105230 0x7f9a441056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 --2- 172.21.15.57:0/341741599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44104040 0x7f9a44104460 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 -- 172.21.15.57:0/341741599 >> 172.21.15.57:0/341741599 conn(0x7f9a440ff5d0 msgr2=0x7f9a441019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:08.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 -- 172.21.15.57:0/341741599 shutdown_connections 2024-04-13T05:28:08.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.836+0000 7f9a43200640 1 -- 172.21.15.57:0/341741599 wait complete. 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 Processor -- start 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 -- start start 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a44104040 0x7f9a4419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44105230 0x7f9a4419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 0x7f9a441a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410fb80 con 0x7f9a44105bf0 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410fa00 con 0x7f9a44105230 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.837+0000 7f9a43200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410fd00 con 0x7f9a44104040 2024-04-13T05:28:08.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 0x7f9a441a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 0x7f9a441a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44636/0 (socket says 172.21.15.57:44636) 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 -- 172.21.15.57:0/1044125689 learned_addr learned my addr 172.21.15.57:0/1044125689 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a41e00640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a44104040 0x7f9a4419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a41400640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44105230 0x7f9a4419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 -- 172.21.15.57:0/1044125689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a44104040 msgr2=0x7f9a4419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a44104040 0x7f9a4419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 -- 172.21.15.57:0/1044125689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44105230 msgr2=0x7f9a4419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44105230 0x7f9a4419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 -- 172.21.15.57:0/1044125689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a441a1e90 con 0x7f9a44105bf0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.838+0000 7f9a42800640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 0x7f9a441a1820 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00cce0 tx=0x7f9a3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.839+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00f450 con 0x7f9a44105bf0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.839+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a3c00faa0 con 0x7f9a44105bf0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.839+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441a2180 con 0x7f9a44105bf0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.839+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c018b70 con 0x7f9a44105bf0 2024-04-13T05:28:08.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.839+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a26d0 con 0x7f9a44105bf0 2024-04-13T05:28:08.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.840+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4410b3e0 con 0x7f9a44105bf0 2024-04-13T05:28:08.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.842+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a3c018d10 con 0x7f9a44105bf0 2024-04-13T05:28:08.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.843+0000 7f9a3aa00640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a280a3c30 0x7f9a280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:08.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.844+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f9a3c0d0c60 con 0x7f9a44105bf0 2024-04-13T05:28:08.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.844+0000 7f9a41e00640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a280a3c30 0x7f9a280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:08.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.844+0000 7f9a41e00640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a280a3c30 0x7f9a280a60f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9a34006010 tx=0x7f9a34005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:08.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:08.848+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a3c014030 con 0x7f9a44105bf0 2024-04-13T05:28:09.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.109+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9a44062bc0 con 0x7f9a44105bf0 2024-04-13T05:28:09.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.139+0000 7f9a3aa00640 1 -- 172.21.15.57:0/1044125689 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+745529 (secure 0 0 0) 0x7f9a3c09d240 con 0x7f9a44105bf0 2024-04-13T05:28:09.161 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:28:09.161 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:28:09.161 INFO:teuthology.orchestra.run.smithi057.stdout: "epoch": 18, 2024-04-13T05:28:09.161 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.161 INFO:teuthology.orchestra.run.smithi057.stdout: "active_gid": 14152, 2024-04-13T05:28:09.161 INFO:teuthology.orchestra.run.smithi057.stdout: "active_name": "x", 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addrs": { 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6800", 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3239836246 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6801", 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3239836246 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.162 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addr": "172.21.15.57:6801/3239836246", 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "active_change": "2024-04-13T05:05:11.407901+0000", 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "active_mgr_features": 4540701547738038271, 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "available": true, 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "standbys": [ 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 24112, 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "z", 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540701547738038271, 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2024-04-13T05:28:09.163 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.164 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.165 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.166 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.167 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.168 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.169 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2024-04-13T05:28:09.170 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2024-04-13T05:28:09.171 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.172 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.174 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2024-04-13T05:28:09.175 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.176 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.177 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.178 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.179 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.180 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.181 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.182 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.183 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.184 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2024-04-13T05:28:09.185 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.186 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2024-04-13T05:28:09.187 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2024-04-13T05:28:09.188 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.189 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.191 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.192 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.193 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.194 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.195 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "read", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap", 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap-read" 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.196 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.197 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.198 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.199 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-13T05:28:09.200 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.201 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.202 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.204 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.205 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.206 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.207 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.209 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.210 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.211 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2024-04-13T05:28:09.213 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2024-04-13T05:28:09.214 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2024-04-13T05:28:09.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2024-04-13T05:28:09.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2024-04-13T05:28:09.217 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.218 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2024-04-13T05:28:09.219 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.220 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2024-04-13T05:28:09.221 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.222 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.223 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.224 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.225 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.226 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2024-04-13T05:28:09.227 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_samba": { 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_samba", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Samba/SMB container image", 2024-04-13T05:28:09.228 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.229 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.230 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.231 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2024-04-13T05:28:09.232 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2024-04-13T05:28:09.233 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.234 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.235 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.236 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "grafana_dashboards_path": { 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "grafana_dashboards_path", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.237 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-13T05:28:09.238 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.239 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.240 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.240 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.240 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.240 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.240 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.240 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: "hw_monitoring": { 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hw_monitoring", 2024-04-13T05:28:09.241 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.242 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.242 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.242 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.242 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.242 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.242 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2024-04-13T05:28:09.243 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2024-04-13T05:28:09.244 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.244 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.244 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.244 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.244 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.244 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.245 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.245 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-13T05:28:09.245 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.245 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.245 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.245 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.246 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.247 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.248 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.249 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.250 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.251 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.252 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.253 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.253 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.253 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.254 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.254 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.254 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.254 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.254 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.255 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.255 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.255 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.255 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.255 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-13T05:28:09.255 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.256 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-13T05:28:09.257 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-13T05:28:09.258 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.258 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.258 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.258 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2024-04-13T05:28:09.258 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.259 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.259 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-13T05:28:09.259 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.259 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.259 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2024-04-13T05:28:09.260 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2024-04-13T05:28:09.260 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.260 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.260 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.260 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.261 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.261 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.261 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.261 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2024-04-13T05:28:09.261 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.261 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.262 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.263 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.263 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.263 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.263 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.263 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-13T05:28:09.263 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.264 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2024-04-13T05:28:09.265 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.266 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.266 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.266 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.266 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-04-13T05:28:09.266 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-04-13T05:28:09.266 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.267 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.268 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.269 INFO:teuthology.orchestra.run.smithi057.stdout: "oob_default_addr": { 2024-04-13T05:28:09.269 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "oob_default_addr", 2024-04-13T05:28:09.269 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.269 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.269 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.269 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "169.254.1.1", 2024-04-13T05:28:09.270 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.270 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.270 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.270 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-13T05:28:09.270 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.270 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.271 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.273 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.274 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.274 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.274 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-13T05:28:09.276 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.276 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.278 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.278 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.278 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.279 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.280 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.280 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-13T05:28:09.280 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.280 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.281 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.281 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.281 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2024-04-13T05:28:09.281 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2024-04-13T05:28:09.281 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.281 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.282 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.283 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-13T05:28:09.284 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.285 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.286 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.288 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2024-04-13T05:28:09.289 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.290 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.291 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.292 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.293 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.294 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.295 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.296 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2024-04-13T05:28:09.297 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2024-04-13T05:28:09.298 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.299 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.300 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.301 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.302 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.303 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.305 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.306 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.309 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.310 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.312 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.314 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.315 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.316 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.317 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.318 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.319 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.320 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.321 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.322 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "[]", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "MULTICLUSTER_CONFIG": { 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "{}", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.325 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.326 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.327 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.328 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.329 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.330 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.331 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.332 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.333 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.334 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.335 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.336 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.337 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.338 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.338 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-13T05:28:09.338 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-13T05:28:09.338 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.338 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.338 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-13T05:28:09.339 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-13T05:28:09.340 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.342 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2024-04-13T05:28:09.343 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.345 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.346 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.347 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.348 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.349 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:09.350 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.351 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.352 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.353 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2024-04-13T05:28:09.354 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-04-13T05:28:09.355 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.356 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.357 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.359 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.360 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.361 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.362 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.363 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:09.364 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.365 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.366 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.367 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.368 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-04-13T05:28:09.369 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.370 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-04-13T05:28:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.372 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.373 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.374 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.375 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.376 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.377 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-13T05:28:09.378 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-13T05:28:09.379 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2024-04-13T05:28:09.380 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-13T05:28:09.381 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.382 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.383 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.384 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.385 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.386 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.387 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.388 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.389 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.390 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-04-13T05:28:09.391 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.392 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.393 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.394 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.395 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.396 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.397 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2024-04-13T05:28:09.398 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.399 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.400 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.401 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.402 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.403 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.404 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.405 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.406 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.407 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:09.408 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.409 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.410 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.411 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.412 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.413 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.414 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.415 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.416 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.417 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.418 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.419 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.420 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.421 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.422 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.424 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.425 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.426 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.427 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.428 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.429 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.430 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.431 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2024-04-13T05:28:09.432 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.433 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.434 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.435 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.436 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.437 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.438 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-13T05:28:09.439 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.440 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.441 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.442 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2024-04-13T05:28:09.443 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.444 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.445 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.446 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.447 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.448 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.449 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.450 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.451 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.455 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.455 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.455 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.455 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.456 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.457 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.458 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.458 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.458 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.458 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.458 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.459 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.460 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.461 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.462 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.463 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.464 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.465 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.466 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.467 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.468 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.469 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.470 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2024-04-13T05:28:09.471 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.473 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.474 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.475 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.476 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.477 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.478 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.479 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.480 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.481 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.482 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.483 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.484 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.485 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.486 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.487 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2024-04-13T05:28:09.490 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.491 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.492 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.493 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.494 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.495 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.496 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.497 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.497 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.497 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.497 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.497 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.497 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.498 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.499 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.500 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.501 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2024-04-13T05:28:09.502 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.503 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.504 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.505 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.506 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.507 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.508 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.509 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.510 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.511 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.512 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.513 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.514 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.515 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.516 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-04-13T05:28:09.517 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.518 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.519 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.520 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.521 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.522 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.523 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.524 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.525 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.526 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-04-13T05:28:09.527 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.528 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.529 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.530 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.531 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.532 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.533 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.534 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.536 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.537 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.538 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.539 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.540 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.541 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.542 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.543 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.544 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-13T05:28:09.545 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.546 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.547 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.549 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.550 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.551 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2024-04-13T05:28:09.552 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2024-04-13T05:28:09.553 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2024-04-13T05:28:09.554 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2024-04-13T05:28:09.558 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.561 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.562 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2024-04-13T05:28:09.563 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.564 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.565 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.567 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.568 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.569 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.570 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.571 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.572 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.573 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.574 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.575 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.576 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.577 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2024-04-13T05:28:09.578 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.579 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.580 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.581 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.583 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.584 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.585 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.586 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.586 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.586 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.587 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.588 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.588 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.588 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.588 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.588 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.590 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.591 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2024-04-13T05:28:09.592 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2024-04-13T05:28:09.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.594 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.595 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:09.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.598 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.600 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.601 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.602 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.603 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.605 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.606 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.608 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.608 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.608 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.608 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.608 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.608 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.609 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.610 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.611 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.612 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.613 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.615 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.616 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.617 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.619 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.620 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.621 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.622 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.622 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.622 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.623 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.624 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.625 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.628 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.631 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_no_wait": { 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_no_wait", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.632 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2024-04-13T05:28:09.633 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:09.635 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.636 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.637 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.638 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.638 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.638 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.638 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.638 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.639 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.639 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.639 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.639 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.639 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.640 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.640 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.640 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.640 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.640 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.641 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.641 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.641 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.641 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.642 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.642 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.642 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.642 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.642 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.643 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.643 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.643 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.643 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.644 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.645 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.645 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.645 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.645 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.645 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.646 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.646 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.646 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.646 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.646 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.646 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.647 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.647 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.647 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.647 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.647 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.648 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.648 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.648 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.648 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.649 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.650 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.650 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2024-04-13T05:28:09.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2024-04-13T05:28:09.651 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.651 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.651 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.651 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.651 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2024-04-13T05:28:09.653 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.654 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.655 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2024-04-13T05:28:09.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2024-04-13T05:28:09.655 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.655 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.656 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.656 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-13T05:28:09.656 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.656 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.657 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.657 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.657 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.657 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.657 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.658 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:09.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.658 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.658 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 34109, 2024-04-13T05:28:09.658 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "y", 2024-04-13T05:28:09.658 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540701547738038271, 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:09.659 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:09.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.660 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.660 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.661 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.663 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.664 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.666 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.666 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.666 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.666 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.666 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.667 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.668 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2024-04-13T05:28:09.671 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.672 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.673 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2024-04-13T05:28:09.674 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.675 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.676 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.677 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.678 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.678 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2024-04-13T05:28:09.678 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.678 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.680 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.681 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.682 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.683 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.684 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.685 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.686 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2024-04-13T05:28:09.687 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.688 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.689 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2024-04-13T05:28:09.690 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.691 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.692 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.693 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2024-04-13T05:28:09.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.696 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2024-04-13T05:28:09.697 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2024-04-13T05:28:09.698 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.699 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2024-04-13T05:28:09.700 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.701 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.702 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.703 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.704 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.705 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.706 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.707 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2024-04-13T05:28:09.710 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.711 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.712 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2024-04-13T05:28:09.713 INFO:teuthology.orchestra.run.smithi057.stdout: "read", 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap", 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap-read" 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.715 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.716 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.717 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.718 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2024-04-13T05:28:09.719 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-13T05:28:09.720 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2024-04-13T05:28:09.721 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.722 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.723 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2024-04-13T05:28:09.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.726 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2024-04-13T05:28:09.726 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2024-04-13T05:28:09.726 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.726 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.726 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.726 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.728 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.729 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.730 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.731 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2024-04-13T05:28:09.733 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2024-04-13T05:28:09.734 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.734 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.734 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.734 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.734 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.734 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-13T05:28:09.735 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.736 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.737 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2024-04-13T05:28:09.739 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.743 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.744 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.745 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-13T05:28:09.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.745 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.745 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.746 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.748 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.750 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.751 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2024-04-13T05:28:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.754 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2024-04-13T05:28:09.755 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2024-04-13T05:28:09.756 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.758 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.759 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.760 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.761 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.762 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2024-04-13T05:28:09.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.764 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_samba": { 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_samba", 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.765 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Samba/SMB container image", 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.766 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.767 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.769 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2024-04-13T05:28:09.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2024-04-13T05:28:09.772 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-13T05:28:09.773 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.774 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.775 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.776 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.777 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.778 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2024-04-13T05:28:09.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.780 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "grafana_dashboards_path": { 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "grafana_dashboards_path", 2024-04-13T05:28:09.781 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.782 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "hw_monitoring": { 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hw_monitoring", 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.787 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.789 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-13T05:28:09.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.791 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.792 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.793 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.795 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.796 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.797 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.798 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.799 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.800 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.801 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.802 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2024-04-13T05:28:09.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2024-04-13T05:28:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: "oob_default_addr": { 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "oob_default_addr", 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.811 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "169.254.1.1", 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.812 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-13T05:28:09.813 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.814 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.815 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2024-04-13T05:28:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-13T05:28:09.817 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.818 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.819 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.820 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2024-04-13T05:28:09.821 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2024-04-13T05:28:09.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.824 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.825 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.826 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.828 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2024-04-13T05:28:09.829 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.830 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.831 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-13T05:28:09.832 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.835 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.837 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.838 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.839 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.840 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.841 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.842 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.843 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2024-04-13T05:28:09.845 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.846 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.847 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-13T05:28:09.849 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.850 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.851 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.852 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.853 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.854 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.855 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.856 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.856 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.856 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.856 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.856 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-13T05:28:09.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.859 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.861 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.863 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.865 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.866 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-13T05:28:09.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.868 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.869 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.870 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.871 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.872 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.873 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.874 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.875 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.876 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.877 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.878 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.880 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.881 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.882 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.883 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.884 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-13T05:28:09.885 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.886 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.887 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "[]", 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.888 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "MULTICLUSTER_CONFIG": { 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "{}", 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.889 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2024-04-13T05:28:09.890 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.891 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.892 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.893 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.894 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.895 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.896 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.897 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.898 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.899 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.901 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.902 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.903 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-13T05:28:09.904 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.905 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-13T05:28:09.906 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.907 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.908 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.909 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.910 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.911 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.912 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-13T05:28:09.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.914 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.915 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.915 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.915 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.915 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2024-04-13T05:28:09.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.916 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.917 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.918 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.919 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.920 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.921 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.922 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.924 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.925 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-13T05:28:09.926 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.927 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.928 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.929 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.930 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2024-04-13T05:28:09.931 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.932 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.934 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.935 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.937 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.938 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.939 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.940 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.941 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.943 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.944 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.945 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.946 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.947 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.948 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.949 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.951 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.952 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.953 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.954 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2024-04-13T05:28:09.955 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-04-13T05:28:09.956 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2024-04-13T05:28:09.957 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-04-13T05:28:09.958 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.959 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.960 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.961 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.962 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.964 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.965 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.966 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.967 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.968 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.969 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.970 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.971 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.972 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.973 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2024-04-13T05:28:09.974 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-13T05:28:09.975 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.976 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.977 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.978 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.979 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.980 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.981 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2024-04-13T05:28:09.982 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-13T05:28:09.983 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2024-04-13T05:28:09.984 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.985 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.986 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:09.987 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.988 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.989 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:09.990 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.991 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.993 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.993 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:09.993 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:09.993 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:09.993 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.994 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.995 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.996 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2024-04-13T05:28:09.997 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:09.998 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:09.999 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.000 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.001 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.002 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.003 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.004 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.005 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.006 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.007 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.008 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.009 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.010 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.012 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.013 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.014 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.014 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.014 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.014 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.014 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.014 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.015 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2024-04-13T05:28:10.016 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2024-04-13T05:28:10.017 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.018 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.019 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.020 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.020 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.020 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-04-13T05:28:10.020 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-04-13T05:28:10.020 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.020 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-13T05:28:10.021 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.022 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.023 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.024 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2024-04-13T05:28:10.025 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.026 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.027 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.028 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.029 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.030 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.031 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.032 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.033 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.034 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.035 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.036 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.037 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.038 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.039 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.040 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.041 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.042 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.043 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.044 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2024-04-13T05:28:10.045 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.046 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.047 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2024-04-13T05:28:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2024-04-13T05:28:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2024-04-13T05:28:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.111 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.113 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.114 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.115 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.116 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.117 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.118 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.119 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.120 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.121 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.122 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.123 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.124 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.125 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.126 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.127 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.128 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.129 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.130 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.131 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.132 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.133 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.134 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.135 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.136 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.137 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.138 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.139 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2024-04-13T05:28:10.141 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.142 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.143 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.144 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-04-13T05:28:10.145 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.147 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.148 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.149 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.150 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.151 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.152 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.153 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2024-04-13T05:28:10.154 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.155 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.156 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.157 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.158 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.159 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.160 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.161 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.162 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.163 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.164 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.165 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.166 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.167 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.168 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.169 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.170 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.171 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.172 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.173 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.174 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.175 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.176 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2024-04-13T05:28:10.177 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.178 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.179 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-04-13T05:28:10.180 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.181 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.182 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.183 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.184 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.185 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.186 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.187 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.188 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.189 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.190 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.191 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2024-04-13T05:28:10.192 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.193 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.194 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.195 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.196 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.197 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.198 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.199 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.200 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.201 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.202 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.203 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.204 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.205 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.206 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.207 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.208 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.209 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.210 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.212 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.213 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.214 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.215 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.216 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.217 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.218 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.219 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.220 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.221 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.222 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.223 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.224 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.225 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.226 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.227 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.228 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.229 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2024-04-13T05:28:10.230 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.231 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.232 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.233 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.234 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.235 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.236 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.237 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.238 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.239 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.240 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.241 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.242 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2024-04-13T05:28:10.243 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.244 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.245 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.246 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.247 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.248 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2024-04-13T05:28:10.249 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.250 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.251 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.252 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.253 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2024-04-13T05:28:10.254 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.255 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.256 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2024-04-13T05:28:10.257 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.258 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.259 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.260 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.261 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.262 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2024-04-13T05:28:10.263 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.264 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.265 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.266 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.267 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.268 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.269 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.270 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.271 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.272 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.273 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.274 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.275 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.276 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.277 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.278 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.279 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.280 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.281 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.282 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.283 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.284 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.285 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.286 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.287 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.288 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.289 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.290 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.291 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.292 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.293 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.294 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.295 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.296 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.297 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.298 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.299 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.300 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2024-04-13T05:28:10.301 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.302 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.303 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.304 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.305 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.306 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.307 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.308 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2024-04-13T05:28:10.310 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.311 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-13T05:28:10.312 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-04-13T05:28:10.313 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.314 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.315 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.316 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2024-04-13T05:28:10.317 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.318 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.319 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.320 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.321 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.324 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.325 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.326 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.327 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2024-04-13T05:28:10.328 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.329 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.330 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2024-04-13T05:28:10.331 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.332 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.333 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.334 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.335 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.336 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.337 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.338 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.339 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.340 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.341 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.342 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.343 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.344 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.345 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.346 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.347 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.348 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.349 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_no_wait": { 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_no_wait", 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.350 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.351 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:10.352 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2024-04-13T05:28:10.353 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.354 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.355 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.356 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.357 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.358 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.359 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.360 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.361 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.362 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.363 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.364 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2024-04-13T05:28:10.365 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.366 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.367 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.368 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "modules": [ 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard", 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "iostat", 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "nfs", 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "restful" 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.369 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.370 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.371 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.372 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.373 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.374 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.375 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.376 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.377 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.378 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.379 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.380 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2024-04-13T05:28:10.381 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.382 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.383 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2024-04-13T05:28:10.384 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.385 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.386 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2024-04-13T05:28:10.387 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.388 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2024-04-13T05:28:10.389 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2024-04-13T05:28:10.390 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.391 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.392 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.393 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.394 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.395 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.396 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.397 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.398 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2024-04-13T05:28:10.399 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.400 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2024-04-13T05:28:10.401 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.402 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.403 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.404 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.406 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.407 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.408 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2024-04-13T05:28:10.409 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.410 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "read", 2024-04-13T05:28:10.412 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap", 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap-read" 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.413 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.414 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.415 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:10.416 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.417 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.418 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2024-04-13T05:28:10.419 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.420 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.421 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.422 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2024-04-13T05:28:10.423 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2024-04-13T05:28:10.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.425 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.426 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.427 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.428 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.429 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2024-04-13T05:28:10.430 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2024-04-13T05:28:10.431 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.432 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.433 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.434 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.435 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.436 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.437 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.438 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.439 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.440 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2024-04-13T05:28:10.441 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.442 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-13T05:28:10.443 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.444 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.446 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.447 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2024-04-13T05:28:10.448 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.449 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.450 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.451 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:10.452 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-13T05:28:10.453 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.454 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.455 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.456 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.457 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_samba": { 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_samba", 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.458 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Samba/SMB container image", 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2024-04-13T05:28:10.461 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2024-04-13T05:28:10.462 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.463 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2024-04-13T05:28:10.465 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2024-04-13T05:28:10.466 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.467 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.468 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.469 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.470 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.471 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "grafana_dashboards_path": { 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "grafana_dashboards_path", 2024-04-13T05:28:10.472 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-13T05:28:10.473 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.474 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.475 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "hw_monitoring": { 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hw_monitoring", 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.476 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.477 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.478 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.479 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.481 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.482 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.483 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.484 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.485 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.486 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.487 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.488 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.489 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2024-04-13T05:28:10.491 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.492 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2024-04-13T05:28:10.493 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2024-04-13T05:28:10.494 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2024-04-13T05:28:10.495 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.496 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "oob_default_addr": { 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "oob_default_addr", 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "169.254.1.1", 2024-04-13T05:28:10.497 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.498 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.499 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.500 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.501 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.502 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.503 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-13T05:28:10.504 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.505 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.506 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.507 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2024-04-13T05:28:10.508 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.509 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.510 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-13T05:28:10.511 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.512 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.513 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.514 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.515 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.516 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.517 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.518 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.519 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.520 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.521 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.522 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.523 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.524 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.525 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2024-04-13T05:28:10.526 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.527 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.528 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.529 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.530 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.531 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.532 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.533 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.534 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.535 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.536 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.537 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-13T05:28:10.538 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-13T05:28:10.539 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.540 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.541 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.542 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.543 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.544 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.545 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-13T05:28:10.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.547 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.548 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-13T05:28:10.549 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.551 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.552 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.553 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.554 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.557 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.558 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.559 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "[]", 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "MULTICLUSTER_CONFIG": { 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-13T05:28:10.560 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "{}", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.561 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.562 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.563 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.564 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.565 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-13T05:28:10.566 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.567 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.568 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.569 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-13T05:28:10.570 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.571 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.572 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.573 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-13T05:28:10.574 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.575 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.576 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.577 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.578 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.581 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.582 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-13T05:28:10.583 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.586 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.587 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.588 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.589 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.591 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.592 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.593 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.594 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2024-04-13T05:28:10.596 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.597 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.598 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.600 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.602 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.603 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.604 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.605 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.606 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.607 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.608 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.608 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.610 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2024-04-13T05:28:10.610 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.610 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.610 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.611 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.612 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.612 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.612 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.612 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.612 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.612 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.614 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-04-13T05:28:10.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.616 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.617 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.620 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-04-13T05:28:10.621 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.622 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.623 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.624 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.625 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.626 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.627 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.628 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.629 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.632 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.634 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.635 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.636 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-13T05:28:10.637 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.638 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.638 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.638 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.638 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.639 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.639 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.639 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.639 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:10.639 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.639 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.640 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.641 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.642 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.643 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.644 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.645 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.646 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.648 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.649 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.650 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.651 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.653 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.655 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2024-04-13T05:28:10.656 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-13T05:28:10.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.658 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.659 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.660 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.661 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.662 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.663 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.664 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.665 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.666 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2024-04-13T05:28:10.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.668 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.669 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.670 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2024-04-13T05:28:10.671 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.672 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.673 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.674 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.675 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.676 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.677 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.678 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.680 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.681 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.682 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.683 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.685 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.686 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.687 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2024-04-13T05:28:10.689 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2024-04-13T05:28:10.691 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.692 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.694 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.696 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.697 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.698 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.700 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.701 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.702 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.703 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2024-04-13T05:28:10.704 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.705 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2024-04-13T05:28:10.706 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.707 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.708 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.709 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-13T05:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.712 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.713 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.714 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.716 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.717 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.718 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.719 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.720 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.721 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2024-04-13T05:28:10.722 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.723 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.725 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.726 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.727 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.728 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.729 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.730 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.731 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.733 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.734 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.736 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.737 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.741 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.742 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.746 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.747 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.748 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.749 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.751 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.752 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.754 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.755 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.756 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.757 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.758 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.759 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.760 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.761 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.763 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.764 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.765 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.766 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2024-04-13T05:28:10.767 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.768 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-04-13T05:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.773 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.774 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.775 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.777 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.778 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.779 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2024-04-13T05:28:10.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.781 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.782 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.783 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.784 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.785 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.787 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.788 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.791 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.794 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.797 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.798 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.800 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.801 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.802 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.803 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.805 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.806 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.809 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.810 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2024-04-13T05:28:10.813 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.814 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.816 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.817 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.819 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.820 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.823 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.824 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.825 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.826 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.827 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.829 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.830 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.831 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.832 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.833 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.835 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2024-04-13T05:28:10.836 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.838 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.839 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.841 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.842 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2024-04-13T05:28:10.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.844 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.845 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.846 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.847 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.848 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.850 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.851 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.852 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.853 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2024-04-13T05:28:10.854 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.856 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.857 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.858 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.859 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2024-04-13T05:28:10.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2024-04-13T05:28:10.861 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.862 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.863 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.864 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.866 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.867 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-04-13T05:28:10.868 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.869 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.870 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.871 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.872 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.874 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.875 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_no_wait": { 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_no_wait", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.876 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.877 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.878 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.880 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.881 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.882 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.883 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2024-04-13T05:28:10.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2024-04-13T05:28:10.885 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2024-04-13T05:28:10.886 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.887 INFO:teuthology.orchestra.run.smithi057.stdout: "services": { 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard": "https://172.21.15.57:8443/" 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "always_on_modules": { 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "octopus": [ 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.888 INFO:teuthology.orchestra.run.smithi057.stdout: "pacific": [ 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "quincy": [ 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-04-13T05:28:10.889 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "reef": [ 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-04-13T05:28:10.890 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "squid": [ 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-04-13T05:28:10.891 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "last_failure_osd_epoch": 3, 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "active_clients": [ 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 43445990 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "libcephsqlite", 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2869899963 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2799412934 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.893 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 4167373040 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-04-13T05:28:10.894 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:09 smithi083 ceph-mon[35133]: pgmap v772: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3180513604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2429025992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:10 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1044125689' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:10 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:09 smithi189 ceph-mon[35531]: pgmap v772: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3180513604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:28:10.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2429025992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:28:10.897 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:10 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1044125689' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-13T05:28:10.897 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:10 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a280a3c30 msgr2=0x7f9a280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a280a3c30 0x7f9a280a60f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9a34006010 tx=0x7f9a34005be0 comp rx=0 tx=0).stop 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 msgr2=0x7f9a441a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 0x7f9a441a1820 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00cce0 tx=0x7f9a3c00a620 comp rx=0 tx=0).stop 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 shutdown_connections 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f9a280a3c30 0x7f9a280a60f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:10.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a44105230 0x7f9a4419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a44105bf0 0x7f9a441a1820 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 --2- 172.21.15.57:0/1044125689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a44104040 0x7f9a4419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 >> 172.21.15.57:0/1044125689 conn(0x7f9a440ff5d0 msgr2=0x7f9a441085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 shutdown_connections 2024-04-13T05:28:10.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:09.155+0000 7f9a43200640 1 -- 172.21.15.57:0/1044125689 wait complete. 2024-04-13T05:28:10.904 DEBUG:tasks.check_counter:Getting stats from a 2024-04-13T05:28:10.904 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-13T05:28:10.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:09 smithi057 ceph-mon[32331]: pgmap v772: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:10.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3180513604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:28:10.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2429025992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:28:10.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:10 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1044125689' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-13T05:28:10.905 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:10 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-13T05:28:11.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:11 smithi057 ceph-mon[32331]: pgmap v773: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:11.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:11.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:11 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:11 smithi189 ceph-mon[35531]: pgmap v773: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:11.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:11 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:11.404 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:11 smithi083 ceph-mon[35133]: pgmap v773: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:11.404 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:11.404 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:11 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:28:13.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:13 smithi057 ceph-mon[32331]: pgmap v774: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:13.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:13.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:13 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:13 smithi189 ceph-mon[35531]: pgmap v774: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:13.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:13 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:13 smithi083 ceph-mon[35133]: pgmap v774: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:13.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:13 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 4, 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 4, 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 2944, 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 448, 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-04-13T05:28:14.207 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.004897238, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000078251, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.001246758, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.001771414, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000590471 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-13T05:28:14.208 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 166, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 160, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 107136, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 3, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 2, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.027890872, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.021520599, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.003355867, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 164, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.014057860, 2024-04-13T05:28:14.209 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000085718 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 107136 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 220, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 217, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 480848, 2024-04-13T05:28:14.210 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 133664, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.069424550, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.028030415, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.038485988, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.008576994, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 216, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.017738464, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000082122 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:14.211 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 480848, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 133664 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:14.212 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:14.213 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-04-13T05:28:14.214 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 149047, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 31, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-04-13T05:28:14.215 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-04-13T05:28:14.216 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 221, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 449755, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 221, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 221, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 449755, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:14.217 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:28:14.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:14 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:14 smithi057 ceph-mon[32331]: pgmap v775: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:14.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:14 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:14 smithi083 ceph-mon[35133]: pgmap v775: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:14.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:14 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:14.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:14 smithi189 ceph-mon[35531]: pgmap v775: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:15.082 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-13T05:28:15.083 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-13T05:28:15.083 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-13T05:28:15.083 DEBUG:tasks.check_counter:Getting stats from d 2024-04-13T05:28:15.083 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-13T05:28:15.554 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:28:15.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:15.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:15 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:16.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:15 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:16.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:15 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:16.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:15 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:16.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:15 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:17.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:28:17.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:28:17.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:16 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:17.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:16 smithi083 ceph-mon[35133]: pgmap v776: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:17.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:28:17.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:28:17.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:16 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:17.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:16 smithi057 ceph-mon[32331]: pgmap v776: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:17.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-13T05:28:17.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-13T05:28:17.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:16 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' 2024-04-13T05:28:17.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:16 smithi189 ceph-mon[35531]: pgmap v776: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:17.997 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:28:17.997 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:17.997 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.003854112, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000612731, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:17.998 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 232, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 227, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 485968, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 139232, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-04-13T05:28:17.999 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.070295991, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.028293436, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.036232720, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.008686685, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 227, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.021899753, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000096474 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 485968, 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 139232 2024-04-13T05:28:18.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 172, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 160, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 108192, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 3, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 2, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.027281754, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.021687176, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.002462047, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 170, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.013068090, 2024-04-13T05:28:18.001 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000076871 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 108192 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:18.002 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:18.003 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:18.004 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 148967, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 29, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:18.005 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 229, 2024-04-13T05:28:18.006 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 450795, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 229, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 229, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 450795, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:18.007 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:28:18.979 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-13T05:28:18.979 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-13T05:28:18.979 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-13T05:28:18.979 DEBUG:tasks.check_counter:Getting stats from g 2024-04-13T05:28:18.979 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-13T05:28:19.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:18 smithi083 ceph-mon[35133]: pgmap v777: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:19.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:18 smithi057 ceph-mon[32331]: pgmap v777: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:19.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:18 smithi189 ceph-mon[35531]: pgmap v777: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:19.479 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:28:21.264 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:21 smithi057 ceph-mon[32331]: pgmap v778: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:21.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:21 smithi189 ceph-mon[35531]: pgmap v778: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:21.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:21 smithi083 ceph-mon[35133]: pgmap v778: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:21.826 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:28:21.826 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.005440581, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000406650, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:21.827 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 5, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 170, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 3104, 2024-04-13T05:28:21.828 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 107936, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 3, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 2, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.027268544, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.021483369, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.002718873, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 169, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.014595105, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000086361 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:21.829 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 107936 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 223, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 220, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 481664, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 135584, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.065611009, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.028594904, 2024-04-13T05:28:21.830 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.030522056, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.008248335, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 219, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.018593728, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000084902 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 481664, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 135584 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-04-13T05:28:21.831 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:21.832 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:21.833 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 149007, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 30, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:21.834 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:21.835 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 224, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 450145, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 224, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 224, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 450145, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:21.836 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:21.837 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:21.837 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:21.837 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:28:22.675 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-13T05:28:22.675 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-13T05:28:22.675 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-13T05:28:22.675 DEBUG:tasks.check_counter:Getting stats from j 2024-04-13T05:28:22.676 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-13T05:28:23.178 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.a/config 2024-04-13T05:28:23.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:23 smithi057 ceph-mon[32331]: pgmap v779: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:23.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:23 smithi189 ceph-mon[35531]: pgmap v779: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:23.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:23 smithi083 ceph-mon[35133]: pgmap v779: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:25.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:25 smithi057 ceph-mon[32331]: pgmap v780: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:25.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:25 smithi189 ceph-mon[35531]: pgmap v780: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:25.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:25 smithi083 ceph-mon[35133]: pgmap v780: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:25.521 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-13T05:28:25.521 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:25.521 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 4, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 4, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 2944, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 448, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.004487212, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000078201, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.001321382, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:25.522 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.001692847, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000564282 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 174, 2024-04-13T05:28:25.523 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 160, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 108544, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 3, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 2, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.026749428, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.021161907, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.002505226, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 172, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.014034810, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000081597 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:25.524 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 108544 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 231, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 228, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 483840, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 140704, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.068513895, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.028711043, 2024-04-13T05:28:25.525 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.032979902, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.008973128, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 227, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.022669607, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000099866 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 483840, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 140704 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.526 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:25.527 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:25.528 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 148984, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 30, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-04-13T05:28:25.529 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:25.530 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 232, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 451185, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 232, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 232, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 451185, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-04-13T05:28:25.531 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-04-13T05:28:25.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:25.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-04-13T05:28:25.532 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-13T05:28:26.424 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-13T05:28:26.424 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-13T05:28:26.424 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-13T05:28:26.424 DEBUG:tasks.check_counter:Getting stats from b 2024-04-13T05:28:26.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-13T05:28:26.883 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:28:27.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:27 smithi189 ceph-mon[35531]: pgmap v781: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:27.353 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:27 smithi083 ceph-mon[35133]: pgmap v781: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:27.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:27 smithi057 ceph-mon[32331]: pgmap v781: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:29.107 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-13T05:28:29.107 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:29.107 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 16082, 2024-04-13T05:28:29.107 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 15745, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 4341775, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 58597108, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 154, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 3.164736834, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 1.198089775, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.976069584, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.867162905, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 15744, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.000108397, 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000127039 2024-04-13T05:28:29.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9136, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.003668343, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000401 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 254473, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 92590, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 86547239, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 57161175, 2024-04-13T05:28:29.109 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 13, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 20.871680877, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 8.868151520, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 16.127947435, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.485434854, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 92589, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 6.818126111, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000073638 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 317870, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.083066779, 2024-04-13T05:28:29.110 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000261 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 622608, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 202816 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 49324, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 48402, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 21373839, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 32587667, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 11, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 8.948640518, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 4.519017592, 2024-04-13T05:28:29.111 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 4.313250479, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.873411671, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 48400, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 4.640573286, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000095879 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 70206, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.025733393, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000366 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 352800 2024-04-13T05:28:29.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 22349, 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 21.921251351, 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000980860 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6148, 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.229765482, 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000200026 2024-04-13T05:28:29.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 61820, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 49458, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 49458, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 264.656236206, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005351130 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 12362, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-13T05:28:29.115 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 129, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 13, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 26, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 2, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 2, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 22, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 23, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 4748, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 2, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 2, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 3, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 124399, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 60237, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 12355, 2024-04-13T05:28:29.116 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 337, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 14, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 142, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 2904, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 51, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 4235, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 19244, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 4, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 234, 2024-04-13T05:28:29.117 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 19244, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 212, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 18, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 194, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 19032, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 74, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 3021, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 6246, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 5, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-13T05:28:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 182021, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 2153, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 324, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 12092 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 339, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 275, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-13T05:28:29.119 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 2909, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 2909, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 18, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 155, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-13T05:28:29.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 13, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 79344, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 79344, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 93, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-13T05:28:29.121 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 93, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 8, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 87374036, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5196, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 33.473430957, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006442153 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.122 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 28, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 4489, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 4461, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 22, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 281, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 259, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 26, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 10904, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 10878, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 2, 2024-04-13T05:28:29.123 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 4468, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 4466, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 86392, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 231800 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 75527, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 2395, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 61820, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 204, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 7932, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1974, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 70.328929574, 2024-04-13T05:28:29.124 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.035627623 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 423, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.529081959, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001250784 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 42081, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 68.617650472, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001630608 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.126 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.127 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 16, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.053321119, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003332569 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 650, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.973272188, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001497341 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 196, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.374570942, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.007013117 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1611, 2024-04-13T05:28:29.130 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 110.714281133, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.068723948 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-13T05:28:29.131 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.305471388, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.030547138 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-13T05:28:29.132 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002493815, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001246907 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.133 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.055704266, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006963033 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.008245693, 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002748564 2024-04-13T05:28:29.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2484, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 11.693213657, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004707412 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.135 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 3, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 5680, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 5680, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 639, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:29.136 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:29.137 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-13T05:28:29.138 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 321627, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 117, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:29.139 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8896, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 171, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 113498, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 762, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-13T05:28:29.140 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 32479, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 85636455, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 37, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 32479, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 32479, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 123.132054497, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003791128 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 32479, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 33371 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 32479, 2024-04-13T05:28:29.141 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 19686, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 12793, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 11, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 424, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 5602, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 278, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 13, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 6336, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-13T05:28:29.142 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 19262, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 22, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 5, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-13T05:28:29.143 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 94, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-13T05:28:29.144 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 9, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 1039, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 379 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.145 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 17, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 9, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 5805, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 2909, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:29.146 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 319875, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 83448935, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 319875, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 319875, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 83448935, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.147 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 32479, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 85953432, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 32479, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 85953432, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 32479, 2024-04-13T05:28:29.149 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 32479, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 32479, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 32479, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 2909, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 316977, 2024-04-13T05:28:29.150 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 2909, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 406, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 316977, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55cbd5cd6500": { 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:29.151 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 79344, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 83179732, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 79344, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 5196, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 83179732, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:29.152 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-13T05:28:29.153 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:29 smithi083 ceph-mon[35133]: pgmap v782: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:29.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:29 smithi189 ceph-mon[35531]: pgmap v782: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:29.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:29 smithi057 ceph-mon[32331]: pgmap v782: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:30.083 INFO:tasks.check_counter:Daemon mds.b mds.exported=142 2024-04-13T05:28:30.083 INFO:tasks.check_counter:Daemon mds.b mds.imported=51 2024-04-13T05:28:30.083 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=13 2024-04-13T05:28:30.083 DEBUG:tasks.check_counter:Getting stats from e 2024-04-13T05:28:30.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-13T05:28:30.523 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:28:31.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:31 smithi189 ceph-mon[35531]: pgmap v783: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:31.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:31 smithi083 ceph-mon[35133]: pgmap v783: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:31.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:31 smithi057 ceph-mon[32331]: pgmap v783: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:32.869 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-13T05:28:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 269796, 2024-04-13T05:28:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 130794, 2024-04-13T05:28:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 95495231, 2024-04-13T05:28:32.869 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 76866166, 2024-04-13T05:28:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 40, 2024-04-13T05:28:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 26.567612689, 2024-04-13T05:28:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 12.161794540, 2024-04-13T05:28:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 18.739215669, 2024-04-13T05:28:32.870 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.550308443, 2024-04-13T05:28:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 130792, 2024-04-13T05:28:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9.648730393, 2024-04-13T05:28:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000073771 2024-04-13T05:28:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.871 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 363242, 2024-04-13T05:28:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.093822656, 2024-04-13T05:28:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000258 2024-04-13T05:28:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 557488, 2024-04-13T05:28:32.872 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 161168 2024-04-13T05:28:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 17101, 2024-04-13T05:28:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 17608, 2024-04-13T05:28:32.873 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 5527768, 2024-04-13T05:28:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 35348492, 2024-04-13T05:28:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-04-13T05:28:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-13T05:28:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 2.508795714, 2024-04-13T05:28:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 1.039941315, 2024-04-13T05:28:32.874 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.873532432, 2024-04-13T05:28:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.570015079, 2024-04-13T05:28:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 17610, 2024-04-13T05:28:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.990111589, 2024-04-13T05:28:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000113010 2024-04-13T05:28:32.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8969, 2024-04-13T05:28:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.004368461, 2024-04-13T05:28:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000487 2024-04-13T05:28:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.876 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 341968 2024-04-13T05:28:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 15394, 2024-04-13T05:28:32.877 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 15398, 2024-04-13T05:28:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 4351013, 2024-04-13T05:28:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 40105189, 2024-04-13T05:28:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 19, 2024-04-13T05:28:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 7, 2024-04-13T05:28:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 2.016193762, 2024-04-13T05:28:32.878 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.832614642, 2024-04-13T05:28:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.446406285, 2024-04-13T05:28:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.569362271, 2024-04-13T05:28:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 15394, 2024-04-13T05:28:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.037033514, 2024-04-13T05:28:32.879 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000132326 2024-04-13T05:28:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10, 2024-04-13T05:28:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000001990, 2024-04-13T05:28:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000199 2024-04-13T05:28:32.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-13T05:28:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-13T05:28:32.881 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-13T05:28:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-13T05:28:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 26655, 2024-04-13T05:28:32.882 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 22.620686165, 2024-04-13T05:28:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000848647 2024-04-13T05:28:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-13T05:28:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:32.883 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8582, 2024-04-13T05:28:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.027167994, 2024-04-13T05:28:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000236211 2024-04-13T05:28:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.884 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 60625, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 60177, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 60177, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 249.161813786, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004140482 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 448, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-13T05:28:32.885 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 119, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 12, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 18, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 2, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 3, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 13, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 13, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 2870, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 4, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 135612, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 66702, 2024-04-13T05:28:32.886 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 448, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 269, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 5, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 169, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 1419, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 85, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 4862, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 24085, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 721, 2024-04-13T05:28:32.887 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 16904134, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 24085, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 24085, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 33, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 2312, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 7484, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-13T05:28:32.888 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 162222, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 1739, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 263, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 14881 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 281, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 221, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-13T05:28:32.889 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 4260, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 4260, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 184, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 12, 2024-04-13T05:28:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 80153, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 80153, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 94, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 94, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 8, 2024-04-13T05:28:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 87405109, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5242, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 33.700939444, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006429023 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 23, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 4519, 2024-04-13T05:28:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 4496, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 22, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 262, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 240, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 18, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 12450, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 12432, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 4429, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 4429, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 82348, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 223608 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 76668, 2024-04-13T05:28:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 1819, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 60625, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 121, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 9333, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1737, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 71.847201625, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.041362810 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 52015, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 53.245013820, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001023647 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-13T05:28:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 631, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.515112265, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000816342 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 275, 2024-04-13T05:28:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.460877741, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001675919 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1748, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 105.772655775, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.060510672 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-13T05:28:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3771, 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 17.320952560, 2024-04-13T05:28:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004593198 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 1, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-13T05:28:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 8091, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 8091, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 641, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 294505, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 106, 2024-04-13T05:28:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8896, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 171, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-13T05:28:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 93877, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 456, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 39507, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 85633992, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 33, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 39507, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 39507, 2024-04-13T05:28:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 125.099022695, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003166502 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 39507, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 40336 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 39507, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 24511, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 14996, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 10, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 426, 2024-04-13T05:28:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 5668, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 260, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 12, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 8551, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 24085, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 20, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-13T05:28:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-13T05:28:32.914 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 94, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 2, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-13T05:28:32.915 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 946, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 358 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 16, 2024-04-13T05:28:32.916 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 8, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 8520, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 4260, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 302287, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 78149480, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 302287, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:32.917 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 302287, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 78149480, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:32.918 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 39507, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 86098332, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 39507, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 86098332, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-13T05:28:32.919 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 39507, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 39507, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 39507, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 39507, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.920 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 4260, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 464340, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 4260, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 426, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 464340, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.921 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x565199158500": { 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 80153, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 83210805, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 80153, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 5242, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 83210805, 2024-04-13T05:28:32.922 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.923 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:32.923 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-13T05:28:33.251 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:33 smithi083 ceph-mon[35133]: pgmap v784: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:33.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:33 smithi189 ceph-mon[35531]: pgmap v784: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:33.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:33 smithi057 ceph-mon[32331]: pgmap v784: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:33.859 INFO:tasks.check_counter:Daemon mds.e mds.exported=169 2024-04-13T05:28:33.859 INFO:tasks.check_counter:Daemon mds.e mds.imported=85 2024-04-13T05:28:33.860 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=12 2024-04-13T05:28:33.860 DEBUG:tasks.check_counter:Getting stats from h 2024-04-13T05:28:33.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-13T05:28:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:28:35.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:35 smithi189 ceph-mon[35531]: pgmap v785: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:35.338 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:35 smithi083 ceph-mon[35133]: pgmap v785: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:35.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:35 smithi057 ceph-mon[32331]: pgmap v785: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 493345, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 252396, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 165577792, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 161528774, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 44, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 9, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 43.741022641, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 19.833420616, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 33.030502572, 2024-04-13T05:28:36.467 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.654768671, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 252391, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 15.605498019, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000061830 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 688685, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.193426503, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000280 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 51588, 2024-04-13T05:28:36.468 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 52840, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 15639295, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 101725018, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 15, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 8.778112537, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 4.618526963, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 4.314874688, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.568739884, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 52842, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 5.884886799, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000111367 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 70885, 2024-04-13T05:28:36.469 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.022047036, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000311 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 345216 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 31837, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 31823, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 10908205, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 85500819, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-13T05:28:36.470 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 4.392438045, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 1.493031790, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.838109666, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 1.776154549, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 31822, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 5.179378675, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000162760 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 557984, 2024-04-13T05:28:36.471 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 162512 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 41939, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 73.536985052, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001753427 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:36.472 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10022, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.539503557, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000353173 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 137324, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 91307, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 91307, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 885.420456233, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.009697180 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 46017, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-13T05:28:36.473 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 327, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 24, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 2, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 581, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 378, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 172, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 31, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 31, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 11796, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 4, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 4, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 4, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 214191, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 93376, 2024-04-13T05:28:36.474 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 45563, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 300, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 2, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 94, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 7359, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 269, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 2585, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 32091, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 721, 2024-04-13T05:28:36.475 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 16904134, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 32091, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 567, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 567, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 31524, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 14, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 18376, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 16326, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 1, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 305995, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 8368, 2024-04-13T05:28:36.476 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 385, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 41817 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 300, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 425, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 17, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 4989, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 4972, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-13T05:28:36.477 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 111, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 26, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-13T05:28:36.478 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 144016, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 38919, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 36871, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 107145, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 2048, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 174, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 46, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 44, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 130, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 9, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 2, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 69791196, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 209533262, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-04-13T05:28:36.479 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 11441, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 72.632402347, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.006348431 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 39, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 8978, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 8939, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 157, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 539, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 382, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 581, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 23640, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 23059, 2024-04-13T05:28:36.480 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 4, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 9054, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 9050, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 89060, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 223608 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 151580, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 14995, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 137324, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 237, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 28412, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8372, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 59.635050897, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.007123154 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.481 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 335, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.219652746, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000655679 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-13T05:28:36.482 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 67471, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 606.125485258, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.008983496 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.483 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.485 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-13T05:28:36.485 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.486 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.486 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.486 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-13T05:28:36.486 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.487 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.487 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.487 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-13T05:28:36.487 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 66, 2024-04-13T05:28:36.487 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.245644872, 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003721892 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-13T05:28:36.489 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3067, 2024-04-13T05:28:36.490 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 11.342008433, 2024-04-13T05:28:36.490 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003698079 2024-04-13T05:28:36.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.490 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-13T05:28:36.490 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 277, 2024-04-13T05:28:36.490 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.608852751, 2024-04-13T05:28:36.491 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002198024 2024-04-13T05:28:36.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.491 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-13T05:28:36.491 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 8345, 2024-04-13T05:28:36.491 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 192.080013330, 2024-04-13T05:28:36.491 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.023017377 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.492 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-13T05:28:36.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.493 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.493 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-13T05:28:36.493 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 66, 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2.487568231, 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.037690427 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.494 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.495 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-13T05:28:36.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.496 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.496 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-13T05:28:36.496 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.497 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.498 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-13T05:28:36.498 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.498 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.498 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-13T05:28:36.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3308, 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 12.676179715, 2024-04-13T05:28:36.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003831976 2024-04-13T05:28:36.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.501 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-13T05:28:36.501 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-13T05:28:36.501 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-13T05:28:36.501 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.502 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.502 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.502 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.502 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-13T05:28:36.502 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-13T05:28:36.503 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 1, 2024-04-13T05:28:36.503 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 0, 2024-04-13T05:28:36.503 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-13T05:28:36.503 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-13T05:28:36.503 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 7608, 2024-04-13T05:28:36.503 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 7608, 2024-04-13T05:28:36.504 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 648, 2024-04-13T05:28:36.504 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-13T05:28:36.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.504 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-13T05:28:36.504 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:36.504 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:36.505 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:36.505 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:36.505 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:36.505 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:36.505 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:36.505 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:36.507 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:36.507 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:36.507 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:36.507 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:36.507 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:36.507 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:36.508 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:36.508 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:36.508 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:36.508 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:36.508 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:36.508 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:36.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:36.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:36.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:36.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:36.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:36.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 293272, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 122, 2024-04-13T05:28:36.510 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:36.511 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:36.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-13T05:28:36.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-13T05:28:36.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:36.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:36.511 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:36.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:36.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8896, 2024-04-13T05:28:36.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 171, 2024-04-13T05:28:36.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:36.512 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:36.512 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:36.513 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-13T05:28:36.513 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 728612, 2024-04-13T05:28:36.513 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 6538, 2024-04-13T05:28:36.513 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:36.513 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-13T05:28:36.513 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:36.514 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-13T05:28:36.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.514 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-13T05:28:36.514 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-13T05:28:36.514 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-13T05:28:36.514 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 60092, 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 213329085, 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 33, 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 60092, 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 60092, 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 351.180635693, 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.005844049 2024-04-13T05:28:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 60092, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 67343 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 60092, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 32705, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 27387, 2024-04-13T05:28:36.516 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 93, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 3932, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 614, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 12055, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 382, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-13T05:28:36.517 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 24, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 10228, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 32091, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 5929, 2024-04-13T05:28:36.518 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-13T05:28:36.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 1, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-13T05:28:36.520 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 94, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 2, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 12, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-13T05:28:36.521 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 1483, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 511 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.522 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 23, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 9, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 10148, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 4972, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 576766, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 140197648, 2024-04-13T05:28:36.523 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 576766, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 576766, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 140197648, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:36.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 60092, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 213870921, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 60092, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 213870921, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.525 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 60092, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 60092, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 60092, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 60092, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.526 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 4972, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 541836, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 4972, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 614, 2024-04-13T05:28:36.527 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 541836, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x559b04eea500": { 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 144016, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 205338958, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 144016, 2024-04-13T05:28:36.528 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 11441, 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 205338958, 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:36.529 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-13T05:28:37.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:37 smithi083 ceph-mon[35133]: pgmap v786: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:37.436 INFO:tasks.check_counter:Daemon mds.h mds.exported=94 2024-04-13T05:28:37.436 INFO:tasks.check_counter:Daemon mds.h mds.imported=269 2024-04-13T05:28:37.436 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=24 2024-04-13T05:28:37.436 DEBUG:tasks.check_counter:Getting stats from k 2024-04-13T05:28:37.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-13T05:28:37.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:37 smithi057 ceph-mon[32331]: pgmap v786: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:37.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:37 smithi189 ceph-mon[35531]: pgmap v786: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:37.895 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.b/config 2024-04-13T05:28:39.252 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:39 smithi083 ceph-mon[35133]: pgmap v787: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:39 smithi057 ceph-mon[32331]: pgmap v787: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:39.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:39 smithi189 ceph-mon[35531]: pgmap v787: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:40.124 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-13T05:28:40.124 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:40.124 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1635, 2024-04-13T05:28:40.124 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1637, 2024-04-13T05:28:40.124 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 29842379, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 519940, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.412097238, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.135560814, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.157062817, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.086413586, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1635, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.136748110, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000083637 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.125 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 572, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 749, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 28800507, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 525004, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-13T05:28:40.126 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.169531157, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.046892019, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.081699766, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.028338260, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 747, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.064575533, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000086446 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:40.127 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 343360 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 846, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 832, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 25856147, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 332841, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.241444055, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.066436467, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.124835647, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.042923984, 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:40.128 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 832, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.076840002, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000092355 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 542960, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 145856 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-13T05:28:40.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4663, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.227111986, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000048705 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.130 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-04-13T05:28:40.131 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 2932, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 2932, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 2932, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-13T05:28:40.132 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 4, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 234, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-13T05:28:40.133 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-13T05:28:40.134 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-13T05:28:40.135 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-13T05:28:40.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 79344, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 93, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 8, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-13T05:28:40.137 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 87374036, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 87369399, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 79344 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 23, 2024-04-13T05:28:40.138 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 4346, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 4323, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 18, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 228, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 210, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 2932, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 9721, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 6789, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 55728, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215416 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-13T05:28:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.140 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.141 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.142 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.143 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.144 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.145 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.146 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-13T05:28:40.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-13T05:28:40.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-13T05:28:40.149 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 3, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 1, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-04-13T05:28:40.150 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 2416, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:40.152 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 231431, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 76, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-13T05:28:40.153 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8896, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 171, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 2138682, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 12021, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:40.154 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 2794, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 26, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 4, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 2794, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2794, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3.540915241, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001267328 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-13T05:28:40.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2794, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2795 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 2794, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 2794, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 1585, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1206, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-13T05:28:40.156 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 2, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-13T05:28:40.157 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-13T05:28:40.158 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 94, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 2, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 10, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 10, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-04-13T05:28:40.159 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-13T05:28:40.160 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3049, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 84208639, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3049, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3049, 2024-04-13T05:28:40.161 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 84208639, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:40.162 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2794, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 83181687, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2794, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 83181687, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:40.163 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2794, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 2794, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2794, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2794, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:40.164 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55cfee864780": { 2024-04-13T05:28:40.165 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-13T05:28:40.166 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-13T05:28:40.167 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-13T05:28:41.100 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-13T05:28:41.100 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-13T05:28:41.100 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-04-13T05:28:41.100 DEBUG:tasks.check_counter:Getting stats from c 2024-04-13T05:28:41.100 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-13T05:28:41.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:41 smithi083 ceph-mon[35133]: pgmap v788: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:41.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:41 smithi057 ceph-mon[32331]: pgmap v788: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:41.557 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:28:41.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:41 smithi189 ceph-mon[35531]: pgmap v788: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:43.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:43 smithi189 ceph-mon[35531]: pgmap v789: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:43.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:43 smithi083 ceph-mon[35133]: pgmap v789: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:43.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:43 smithi057 ceph-mon[32331]: pgmap v789: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 736, 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 736, 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 25357960, 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 233287, 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-04-13T05:28:43.937 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.242503561, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.056021887, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.123926058, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.042786510, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 735, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.076393713, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000103937 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 513, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 687, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 33010932, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 507270, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.234972960, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.061200474, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.127295423, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.031511750, 2024-04-13T05:28:43.939 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 685, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.072360768, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000105636 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 344448 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1690, 2024-04-13T05:28:43.940 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1678, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 26133158, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 600926, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.472769499, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.143518911, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.195460500, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.104699735, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1677, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.170184544, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000101481 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:43.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 538768, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 142688 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-13T05:28:43.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4518, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.258252509, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000057160 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-04-13T05:28:43.943 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 4270, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 4270, 2024-04-13T05:28:43.944 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 4270, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-13T05:28:43.945 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-13T05:28:43.946 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-13T05:28:43.947 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-13T05:28:43.948 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 80153, 2024-04-13T05:28:43.949 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 94, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 8, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 87405109, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 87403971, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 80153 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 13, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 4487, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 4474, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 11, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 264, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 253, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 4270, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 11760, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 7490, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-13T05:28:43.951 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 56612, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215416 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.954 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-13T05:28:43.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.957 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.958 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-13T05:28:43.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.961 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-04-13T05:28:43.963 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 2487, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:43.964 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:43.965 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 223288, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 71, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:43.966 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8896, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 171, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 3049672, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 17208, 2024-04-13T05:28:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 2685, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 2685, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2685, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3.512930269, 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001308353 2024-04-13T05:28:43.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2685, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 2686 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 2685, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 2685, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 1514, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 1169, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-04-13T05:28:43.969 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-13T05:28:43.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-13T05:28:43.971 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 94, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 2, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 9, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 9, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-04-13T05:28:43.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-13T05:28:43.973 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 2935, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 84222170, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 2935, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2935, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 84222170, 2024-04-13T05:28:43.974 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2685, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 83211952, 2024-04-13T05:28:43.975 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2685, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 83211952, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:43.976 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2685, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 2685, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2685, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2685, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:43.977 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55f62aeb2780": { 2024-04-13T05:28:43.978 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:43.979 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-13T05:28:44.964 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-13T05:28:44.964 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-13T05:28:44.964 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-13T05:28:44.964 DEBUG:tasks.check_counter:Getting stats from f 2024-04-13T05:28:44.965 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-13T05:28:45.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:45 smithi189 ceph-mon[35531]: pgmap v790: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:45.420 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:28:45.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:45 smithi083 ceph-mon[35133]: pgmap v790: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:45.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:45 smithi057 ceph-mon[32331]: pgmap v790: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:47.325 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:47 smithi189 ceph-mon[35531]: pgmap v791: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:47.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:47 smithi083 ceph-mon[35133]: pgmap v791: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:47.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:47 smithi057 ceph-mon[32331]: pgmap v791: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 0, 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2, 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 0, 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 0, 2024-04-13T05:28:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.005027529, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000000000, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000599465, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 176, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 160, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 76784, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 3, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.023029992, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.016456176, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.002226281, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 174, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.019891850, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000114320 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 76784 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 239, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 234, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 487936, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 143232, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.091430006, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.036221678, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.046122014, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.012951065, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 234, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.028398739, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000121362 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.663 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 487936, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 143232 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:47.666 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 149007, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 30, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-13T05:28:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 236, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 451769, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 236, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 236, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 451769, 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:47.670 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:47.670 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:47.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:47.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:47.670 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-13T05:28:48.565 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-04-13T05:28:48.565 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-04-13T05:28:48.565 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-04-13T05:28:48.565 DEBUG:tasks.check_counter:Getting stats from i 2024-04-13T05:28:48.566 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-13T05:28:49.023 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:28:49.408 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:49 smithi189 ceph-mon[35531]: pgmap v792: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:49.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:49 smithi083 ceph-mon[35133]: pgmap v792: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:49.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:49 smithi057 ceph-mon[32331]: pgmap v792: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:51.256 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-13T05:28:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1782, 2024-04-13T05:28:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1784, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 67450169, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 566683, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 8, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.572144998, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.143886508, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.281784282, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.106089108, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1782, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.172357415, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000096721 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 438, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 610, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 71419394, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 485884, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-13T05:28:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.334024604, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.050300256, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.244550909, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.025589273, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 609, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.067089420, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000110163 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 348128 2024-04-13T05:28:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 837, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 825, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 67796915, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 329672, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.435888505, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.084819994, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.292061331, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.050394914, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 824, 2024-04-13T05:28:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.089024545, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000108039 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 538768, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 142688 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4732, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.299302810, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000063250 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-13T05:28:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 6961, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 819, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 6142, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 5790, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-13T05:28:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-13T05:28:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-13T05:28:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-13T05:28:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-13T05:28:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 144016, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 130, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 13, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 69791196, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 209533262, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 209531611, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 144016 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 30, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 8834, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 8804, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 147, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 487, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 340, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 6961, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 21159, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 14198, 2024-04-13T05:28:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 62532, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215416 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-13T05:28:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-13T05:28:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-13T05:28:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 1, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 0, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 2495, 2024-04-13T05:28:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 240295, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 82, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8896, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 171, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 5169387, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 38876, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 2803, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 2803, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2803, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 4.483910907, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001599682 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2803, 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 2804 2024-04-13T05:28:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 2803, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 2803, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 1535, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 1266, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-13T05:28:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-13T05:28:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-13T05:28:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 94, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 2, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 11, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 11, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-04-13T05:28:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 3053, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 206375978, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 3053, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 3053, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 206375978, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2803, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 205345086, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2803, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 205345086, 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2803, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 2803, 2024-04-13T05:28:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2803, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2803, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x559a617c0780": { 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-13T05:28:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:51.298 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-13T05:28:51.299 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:51 smithi189 ceph-mon[35531]: pgmap v793: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:51.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:51 smithi083 ceph-mon[35133]: pgmap v793: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:51.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:51 smithi057 ceph-mon[32331]: pgmap v793: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:28:52.208 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-13T05:28:52.209 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-13T05:28:52.209 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-13T05:28:52.209 DEBUG:tasks.check_counter:Getting stats from l 2024-04-13T05:28:52.209 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838 shell --fsid 0a178454-f953-11ee-bc8d-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-13T05:28:52.670 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/mon.c/config 2024-04-13T05:28:53.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:53 smithi189 ceph-mon[35531]: pgmap v794: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:53.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:53 smithi083 ceph-mon[35133]: pgmap v794: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:53.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:53 smithi057 ceph-mon[32331]: pgmap v794: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:55.152 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-13T05:28:55.153 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-13T05:28:55.153 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 0, 2024-04-13T05:28:55.153 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2, 2024-04-13T05:28:55.153 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 0, 2024-04-13T05:28:55.153 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 0, 2024-04-13T05:28:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-04-13T05:28:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-04-13T05:28:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.003417968, 2024-04-13T05:28:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000000000, 2024-04-13T05:28:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000578896, 2024-04-13T05:28:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:55.155 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:55.156 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:55.156 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:55.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.156 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-13T05:28:55.156 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-04-13T05:28:55.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1, 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 182, 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 160, 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 77840, 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 3, 2024-04-13T05:28:55.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-04-13T05:28:55.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.023040189, 2024-04-13T05:28:55.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.016526716, 2024-04-13T05:28:55.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.002053093, 2024-04-13T05:28:55.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-13T05:28:55.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:55.158 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 180, 2024-04-13T05:28:55.159 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.019597356, 2024-04-13T05:28:55.159 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000108874 2024-04-13T05:28:55.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.159 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:55.159 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:55.159 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:55.160 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:55.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.160 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-13T05:28:55.160 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 77840 2024-04-13T05:28:55.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.160 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 246, 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 241, 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 489840, 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 147712, 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-04-13T05:28:55.161 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.093954033, 2024-04-13T05:28:55.162 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.036715706, 2024-04-13T05:28:55.162 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.049251500, 2024-04-13T05:28:55.162 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.012111262, 2024-04-13T05:28:55.162 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-13T05:28:55.162 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 241, 2024-04-13T05:28:55.162 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.028441350, 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000118013 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:55.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.164 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 489840, 2024-04-13T05:28:55.164 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 147712 2024-04-13T05:28:55.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.164 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-13T05:28:55.164 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-04-13T05:28:55.164 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-13T05:28:55.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-13T05:28:55.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-13T05:28:55.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-13T05:28:55.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-13T05:28:55.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-13T05:28:55.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-13T05:28:55.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-13T05:28:55.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-13T05:28:55.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-13T05:28:55.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-13T05:28:55.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-13T05:28:55.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-13T05:28:55.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-13T05:28:55.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-13T05:28:55.169 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 149047, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 31, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-13T05:28:55.170 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-13T05:28:55.171 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 243, 2024-04-13T05:28:55.172 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 452679, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 243, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 243, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 452679, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-13T05:28:55.173 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-13T05:28:55.333 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:55 smithi189 ceph-mon[35531]: pgmap v795: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:55.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:55 smithi083 ceph-mon[35133]: pgmap v795: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:55.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:55 smithi057 ceph-mon[32331]: pgmap v795: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-13T05:28:55.956 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-13T05:28:55.957 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-13T05:28:55.957 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-13T05:28:55.957 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-13T05:28:55.967 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-13T05:28:55.967 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:28:55.967 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T05:28:56.021 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:28:56.021 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T05:28:56.068 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:28:56.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 -- 172.21.15.57:0/3910508989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 msgr2=0x7f5728078020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 --2- 172.21.15.57:0/3910508989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f5728078020 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f5714009900 tx=0x7f571402bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:56.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 -- 172.21.15.57:0/3910508989 shutdown_connections 2024-04-13T05:28:56.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 --2- 172.21.15.57:0/3910508989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5728078560 0x7f5728078e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 --2- 172.21.15.57:0/3910508989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f5728078020 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 --2- 172.21.15.57:0/3910508989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5728076980 0x7f5728076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 -- 172.21.15.57:0/3910508989 >> 172.21.15.57:0/3910508989 conn(0x7f57280ff7b0 msgr2=0x7f5728101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:56.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 -- 172.21.15.57:0/3910508989 shutdown_connections 2024-04-13T05:28:56.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.332+0000 7f572f400640 1 -- 172.21.15.57:0/3910508989 wait complete. 2024-04-13T05:28:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.333+0000 7f572f400640 1 Processor -- start 2024-04-13T05:28:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 -- start start 2024-04-13T05:28:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5728076980 0x7f572819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f572819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5728078560 0x7f57281a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f57281141c0 con 0x7f5728077bc0 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5728114040 con 0x7f5728078560 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f572f400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5728114340 con 0x7f5728076980 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f5727e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f572819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f5727e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f572819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51914/0 (socket says 172.21.15.57:51914) 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.334+0000 7f5727e00640 1 -- 172.21.15.57:0/1052768411 learned_addr learned my addr 172.21.15.57:0/1052768411 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f572ca00640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5728076980 0x7f572819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f572d400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5728078560 0x7f57281a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f5727e00640 1 -- 172.21.15.57:0/1052768411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5728076980 msgr2=0x7f572819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f5727e00640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5728076980 0x7f572819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f5727e00640 1 -- 172.21.15.57:0/1052768411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5728078560 msgr2=0x7f57281a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f5727e00640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5728078560 0x7f57281a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f5727e00640 1 -- 172.21.15.57:0/1052768411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57281a6410 con 0x7f5728077bc0 2024-04-13T05:28:56.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.335+0000 7f5727e00640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f572819ed30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5714000c00 tx=0x7f57140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:56.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.336+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714043020 con 0x7f5728077bc0 2024-04-13T05:28:56.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.336+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57140054c0 con 0x7f5728077bc0 2024-04-13T05:28:56.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.336+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714003d40 con 0x7f5728077bc0 2024-04-13T05:28:56.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.336+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57281a66a0 con 0x7f5728077bc0 2024-04-13T05:28:56.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.336+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57281a6bf0 con 0x7f5728077bc0 2024-04-13T05:28:56.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.337+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5728110d50 con 0x7f5728077bc0 2024-04-13T05:28:56.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.339+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5714046020 con 0x7f5728077bc0 2024-04-13T05:28:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.340+0000 7f5725600640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57100a3bb0 0x7f57100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.341+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f57140f2c50 con 0x7f5728077bc0 2024-04-13T05:28:56.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.341+0000 7f572ca00640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57100a3bb0 0x7f57100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.342+0000 7f572ca00640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57100a3bb0 0x7f57100a6070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f571c005fd0 tx=0x7f571c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:56.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.346+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f571404d020 con 0x7f5728077bc0 2024-04-13T05:28:56.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.547+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5728062bc0 con 0x7f5728077bc0 2024-04-13T05:28:56.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.551+0000 7f5725600640 1 -- 172.21.15.57:0/1052768411 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f57140bf230 con 0x7f5728077bc0 2024-04-13T05:28:56.554 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:28:56.555 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57100a3bb0 msgr2=0x7f57100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57100a3bb0 0x7f57100a6070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f571c005fd0 tx=0x7f571c005850 comp rx=0 tx=0).stop 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 msgr2=0x7f572819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f572819ed30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5714000c00 tx=0x7f57140042f0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 shutdown_connections 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f57100a3bb0 0x7f57100a6070 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5728078560 0x7f57281a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5728077bc0 0x7f572819ed30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 --2- 172.21.15.57:0/1052768411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5728076980 0x7f572819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.554+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 >> 172.21.15.57:0/1052768411 conn(0x7f57280ff7b0 msgr2=0x7f5728100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:56.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.555+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 shutdown_connections 2024-04-13T05:28:56.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.555+0000 7f572f400640 1 -- 172.21.15.57:0/1052768411 wait complete. 2024-04-13T05:28:56.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:28:56.578 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:28:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.801+0000 7f2d91200640 1 -- 172.21.15.57:0/2571302908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1029a0 msgr2=0x7f2d8c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.801+0000 7f2d91200640 1 --2- 172.21.15.57:0/2571302908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1029a0 0x7f2d8c104d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009900 tx=0x7f2d7c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 -- 172.21.15.57:0/2571302908 shutdown_connections 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 --2- 172.21.15.57:0/2571302908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c107c00 0x7f2d8c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 --2- 172.21.15.57:0/2571302908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c1076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 --2- 172.21.15.57:0/2571302908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1029a0 0x7f2d8c104d90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 -- 172.21.15.57:0/2571302908 >> 172.21.15.57:0/2571302908 conn(0x7f2d8c0fc390 msgr2=0x7f2d8c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 -- 172.21.15.57:0/2571302908 shutdown_connections 2024-04-13T05:28:56.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.802+0000 7f2d91200640 1 -- 172.21.15.57:0/2571302908 wait complete. 2024-04-13T05:28:56.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.803+0000 7f2d91200640 1 Processor -- start 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.803+0000 7f2d91200640 1 -- start start 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.803+0000 7f2d91200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c1029a0 0x7f2d8c102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.803+0000 7f2d91200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.803+0000 7f2d91200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c107c00 0x7f2d8c100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d91200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c10dab0 con 0x7f2d8c1052d0 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d91200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c10d930 con 0x7f2d8c107c00 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d91200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c10dc30 con 0x7f2d8c1029a0 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c100950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51930/0 (socket says 172.21.15.57:51930) 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 -- 172.21.15.57:0/3708132925 learned_addr learned my addr 172.21.15.57:0/3708132925 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d8b400640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c1029a0 0x7f2d8c102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d8be00640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c107c00 0x7f2d8c100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 -- 172.21.15.57:0/3708132925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c1029a0 msgr2=0x7f2d8c102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c1029a0 0x7f2d8c102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 -- 172.21.15.57:0/3708132925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c107c00 msgr2=0x7f2d8c100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:56.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c107c00 0x7f2d8c100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:56.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.804+0000 7f2d82a00640 1 -- 172.21.15.57:0/3708132925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c101720 con 0x7f2d8c1052d0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.805+0000 7f2d82a00640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c100950 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d740 tx=0x7f2d7400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.805+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74017070 con 0x7f2d8c1052d0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.805+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d7400de50 con 0x7f2d8c1052d0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.805+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8c1a22a0 con 0x7f2d8c1052d0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.805+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004d60 con 0x7f2d8c1052d0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.805+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8c1a27f0 con 0x7f2d8c1052d0 2024-04-13T05:28:56.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.806+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8c0fde80 con 0x7f2d8c1052d0 2024-04-13T05:28:56.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.808+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2d74004f00 con 0x7f2d8c1052d0 2024-04-13T05:28:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.810+0000 7f2d88c00640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d700a3c30 0x7f2d700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.810+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f2d740d05e0 con 0x7f2d8c1052d0 2024-04-13T05:28:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.810+0000 7f2d8b400640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d700a3c30 0x7f2d700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:56.813 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.811+0000 7f2d8b400640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d700a3c30 0x7f2d700a60f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006010 tx=0x7f2d7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:56.814 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:56.812+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d7409cc40 con 0x7f2d8c1052d0 2024-04-13T05:28:57.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.017+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2d8c02d770 con 0x7f2d8c1052d0 2024-04-13T05:28:57.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.019+0000 7f2d88c00640 1 -- 172.21.15.57:0/3708132925 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f2d7409c540 con 0x7f2d8c1052d0 2024-04-13T05:28:57.021 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:28:57.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d700a3c30 msgr2=0x7f2d700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d700a3c30 0x7f2d700a60f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006010 tx=0x7f2d7c000bc0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 msgr2=0x7f2d8c100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c100950 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d740 tx=0x7f2d7400dc00 comp rx=0 tx=0).stop 2024-04-13T05:28:57.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 shutdown_connections 2024-04-13T05:28:57.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f2d700a3c30 0x7f2d700a60f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c107c00 0x7f2d8c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d8c1052d0 0x7f2d8c100950 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 --2- 172.21.15.57:0/3708132925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c1029a0 0x7f2d8c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.021+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 >> 172.21.15.57:0/3708132925 conn(0x7f2d8c0fc390 msgr2=0x7f2d8c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:57.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.022+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 shutdown_connections 2024-04-13T05:28:57.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.022+0000 7f2d91200640 1 -- 172.21.15.57:0/3708132925 wait complete. 2024-04-13T05:28:57.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:28:57.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 -- 172.21.15.57:0/2751768532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e4105cb0 msgr2=0x7f62e410c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 --2- 172.21.15.57:0/2751768532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e4105cb0 0x7f62e410c540 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009ca0 tx=0x7f62dc02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 -- 172.21.15.57:0/2751768532 shutdown_connections 2024-04-13T05:28:57.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 --2- 172.21.15.57:0/2751768532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e4105cb0 0x7f62e410c540 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 --2- 172.21.15.57:0/2751768532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e41052f0 0x7f62e4105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 --2- 172.21.15.57:0/2751768532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e4104100 0x7f62e4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 -- 172.21.15.57:0/2751768532 >> 172.21.15.57:0/2751768532 conn(0x7f62e40ff670 msgr2=0x7f62e4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:57.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 -- 172.21.15.57:0/2751768532 shutdown_connections 2024-04-13T05:28:57.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.276+0000 7f62e9400640 1 -- 172.21.15.57:0/2751768532 wait complete. 2024-04-13T05:28:57.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.277+0000 7f62e9400640 1 Processor -- start 2024-04-13T05:28:57.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 -- start start 2024-04-13T05:28:57.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e4104100 0x7f62e419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 0x7f62e419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e4105cb0 0x7f62e41a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410fc40 con 0x7f62e41052f0 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410fac0 con 0x7f62e4104100 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e9400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410fdc0 con 0x7f62e4105cb0 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 0x7f62e419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 0x7f62e419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51954/0 (socket says 172.21.15.57:51954) 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e3400640 1 -- 172.21.15.57:0/4163490506 learned_addr learned my addr 172.21.15.57:0/4163490506 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.278+0000 7f62e3e00640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e4104100 0x7f62e419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e8a00640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e4105cb0 0x7f62e41a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e3400640 1 -- 172.21.15.57:0/4163490506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e4105cb0 msgr2=0x7f62e41a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e3400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e4105cb0 0x7f62e41a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e3400640 1 -- 172.21.15.57:0/4163490506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e4104100 msgr2=0x7f62e419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e3400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e4104100 0x7f62e419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e3400640 1 -- 172.21.15.57:0/4163490506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41a1e60 con 0x7f62e41052f0 2024-04-13T05:28:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e8a00640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e4105cb0 0x7f62e41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e3400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 0x7f62e419a780 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c920 tx=0x7f62d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0007c70 con 0x7f62e41052f0 2024-04-13T05:28:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62d0007e10 con 0x7f62e41052f0 2024-04-13T05:28:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.279+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e41a2150 con 0x7f62e41052f0 2024-04-13T05:28:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.280+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0018820 con 0x7f62e41052f0 2024-04-13T05:28:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.280+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a26a0 con 0x7f62e41052f0 2024-04-13T05:28:57.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.281+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e410b3c0 con 0x7f62e41052f0 2024-04-13T05:28:57.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.283+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62d00189c0 con 0x7f62e41052f0 2024-04-13T05:28:57.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.284+0000 7f62e0c00640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62c80a3bb0 0x7f62c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.284+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f62d00d0920 con 0x7f62e41052f0 2024-04-13T05:28:57.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.284+0000 7f62e3e00640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62c80a3bb0 0x7f62c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.285+0000 7f62e3e00640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62c80a3bb0 0x7f62c80a6070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f62e4105150 tx=0x7f62d400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:57.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.288+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62d0014030 con 0x7f62e41052f0 2024-04-13T05:28:57.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.498+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62e4062bc0 con 0x7f62e41052f0 2024-04-13T05:28:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:57 smithi083 ceph-mon[35133]: pgmap v796: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:57 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1052768411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:28:57.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:57 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3708132925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:28:57.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.500+0000 7f62e0c00640 1 -- 172.21.15.57:0/4163490506 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f62d009cf00 con 0x7f62e41052f0 2024-04-13T05:28:57.502 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:28:57.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.502+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62c80a3bb0 msgr2=0x7f62c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.502+0000 7f62e9400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62c80a3bb0 0x7f62c80a6070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f62e4105150 tx=0x7f62d400b390 comp rx=0 tx=0).stop 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 msgr2=0x7f62e419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 0x7f62e419a780 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c920 tx=0x7f62d000cde0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 shutdown_connections 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62c80a3bb0 0x7f62c80a6070 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62e4105cb0 0x7f62e41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e41052f0 0x7f62e419a780 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 --2- 172.21.15.57:0/4163490506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62e4104100 0x7f62e419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 >> 172.21.15.57:0/4163490506 conn(0x7f62e40ff670 msgr2=0x7f62e4108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 shutdown_connections 2024-04-13T05:28:57.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.503+0000 7f62e9400640 1 -- 172.21.15.57:0/4163490506 wait complete. 2024-04-13T05:28:57.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:57 smithi057 ceph-mon[32331]: pgmap v796: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:57.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:57 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1052768411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:28:57.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:57 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3708132925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:28:57.527 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:28:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:57 smithi189 ceph-mon[35531]: pgmap v796: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-13T05:28:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:57 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1052768411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:28:57.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:57 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3708132925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 -- 172.21.15.57:0/3048886926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 msgr2=0x7f5bd0110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/3048886926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd0110940 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009900 tx=0x7f5bc802bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 -- 172.21.15.57:0/3048886926 shutdown_connections 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/3048886926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd0110940 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/3048886926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bd01096f0 0x7f5bd0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/3048886926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd0108500 0x7f5bd0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 -- 172.21.15.57:0/3048886926 >> 172.21.15.57:0/3048886926 conn(0x7f5bd0077490 msgr2=0x7f5bd00798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:57.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 -- 172.21.15.57:0/3048886926 shutdown_connections 2024-04-13T05:28:57.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.759+0000 7f5bd5c00640 1 -- 172.21.15.57:0/3048886926 wait complete. 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.760+0000 7f5bd5c00640 1 Processor -- start 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.760+0000 7f5bd5c00640 1 -- start start 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.760+0000 7f5bd5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd0108500 0x7f5bd019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bd01096f0 0x7f5bd019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd01a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd0114040 con 0x7f5bd010a0b0 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd0113ec0 con 0x7f5bd0108500 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd01141c0 con 0x7f5bd01096f0 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd01a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd01a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51974/0 (socket says 172.21.15.57:51974) 2024-04-13T05:28:57.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 -- 172.21.15.57:0/4223604859 learned_addr learned my addr 172.21.15.57:0/4223604859 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bcf400640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bd01096f0 0x7f5bd019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bcfe00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd0108500 0x7f5bd019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 -- 172.21.15.57:0/4223604859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bd01096f0 msgr2=0x7f5bd019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bd01096f0 0x7f5bd019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 -- 172.21.15.57:0/4223604859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd0108500 msgr2=0x7f5bd019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd0108500 0x7f5bd019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.761+0000 7f5bd5200640 1 -- 172.21.15.57:0/4223604859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd01a62b0 con 0x7f5bd010a0b0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.762+0000 7f5bd5200640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd01a5c40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009230 tx=0x7f5bc8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.762+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8043020 con 0x7f5bd010a0b0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.762+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bc80043d0 con 0x7f5bd010a0b0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.762+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8003b40 con 0x7f5bd010a0b0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.762+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd01a6540 con 0x7f5bd010a0b0 2024-04-13T05:28:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.762+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd01a6a40 con 0x7f5bd010a0b0 2024-04-13T05:28:57.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.763+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd010f820 con 0x7f5bd010a0b0 2024-04-13T05:28:57.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.765+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5bc8046020 con 0x7f5bd010a0b0 2024-04-13T05:28:57.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.767+0000 7f5bccc00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bb40a3c30 0x7f5bb40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.767+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f5bc80f2aa0 con 0x7f5bd010a0b0 2024-04-13T05:28:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.767+0000 7f5bcfe00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bb40a3c30 0x7f5bb40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.768+0000 7f5bcfe00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bb40a3c30 0x7f5bb40a60f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0109550 tx=0x7f5bc0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:57.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.771+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5bc804d020 con 0x7f5bd010a0b0 2024-04-13T05:28:57.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.982+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5bd0062bc0 con 0x7f5bd010a0b0 2024-04-13T05:28:57.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.983+0000 7f5bccc00640 1 -- 172.21.15.57:0/4223604859 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f5bc80bf080 con 0x7f5bd010a0b0 2024-04-13T05:28:57.986 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:28:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bb40a3c30 msgr2=0x7f5bb40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bb40a3c30 0x7f5bb40a60f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0109550 tx=0x7f5bc0009500 comp rx=0 tx=0).stop 2024-04-13T05:28:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 msgr2=0x7f5bd01a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:57.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd01a5c40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009230 tx=0x7f5bc8005e40 comp rx=0 tx=0).stop 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 shutdown_connections 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f5bb40a3c30 0x7f5bb40a60f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bd010a0b0 0x7f5bd01a5c40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bd01096f0 0x7f5bd019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 --2- 172.21.15.57:0/4223604859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd0108500 0x7f5bd019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 >> 172.21.15.57:0/4223604859 conn(0x7f5bd0077490 msgr2=0x7f5bd010c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 shutdown_connections 2024-04-13T05:28:57.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:57.986+0000 7f5bd5c00640 1 -- 172.21.15.57:0/4223604859 wait complete. 2024-04-13T05:28:58.008 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:28:58.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.243+0000 7fec7ce00640 1 -- 172.21.15.57:0/1559535021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec78105cb0 msgr2=0x7fec7810c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.243+0000 7fec7ce00640 1 --2- 172.21.15.57:0/1559535021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec78105cb0 0x7fec7810c540 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fec70009900 tx=0x7fec7002bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 -- 172.21.15.57:0/1559535021 shutdown_connections 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 --2- 172.21.15.57:0/1559535021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec781052f0 0x7fec78105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 --2- 172.21.15.57:0/1559535021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec78105cb0 0x7fec7810c540 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 --2- 172.21.15.57:0/1559535021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec78104100 0x7fec78104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 -- 172.21.15.57:0/1559535021 >> 172.21.15.57:0/1559535021 conn(0x7fec780ff670 msgr2=0x7fec78101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 -- 172.21.15.57:0/1559535021 shutdown_connections 2024-04-13T05:28:58.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.244+0000 7fec7ce00640 1 -- 172.21.15.57:0/1559535021 wait complete. 2024-04-13T05:28:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 Processor -- start 2024-04-13T05:28:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 -- start start 2024-04-13T05:28:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec78104100 0x7fec7819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 0x7fec7819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78105cb0 0x7fec781a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fc40 con 0x7fec781052f0 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fac0 con 0x7fec78105cb0 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.245+0000 7fec7ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fdc0 con 0x7fec78104100 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 0x7fec7819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 0x7fec7819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51988/0 (socket says 172.21.15.57:51988) 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 -- 172.21.15.57:0/2709131347 learned_addr learned my addr 172.21.15.57:0/2709131347 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec77e00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78105cb0 0x7fec781a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec77400640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec78104100 0x7fec7819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 -- 172.21.15.57:0/2709131347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec78104100 msgr2=0x7fec7819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec78104100 0x7fec7819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 -- 172.21.15.57:0/2709131347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78105cb0 msgr2=0x7fec781a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78105cb0 0x7fec781a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec76a00640 1 -- 172.21.15.57:0/2709131347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec781a1ee0 con 0x7fec781052f0 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.246+0000 7fec77e00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78105cb0 0x7fec781a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.247+0000 7fec76a00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 0x7fec7819a800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fec6000ebc0 tx=0x7fec6000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.247+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60010630 con 0x7fec781052f0 2024-04-13T05:28:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.247+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec60010c80 con 0x7fec781052f0 2024-04-13T05:28:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.247+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60018b30 con 0x7fec781052f0 2024-04-13T05:28:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.247+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec781a21d0 con 0x7fec781052f0 2024-04-13T05:28:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.247+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781a2720 con 0x7fec781052f0 2024-04-13T05:28:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.248+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7810b420 con 0x7fec781052f0 2024-04-13T05:28:58.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.250+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fec60018cd0 con 0x7fec781052f0 2024-04-13T05:28:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.252+0000 7fec6fe00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fec580a3bb0 0x7fec580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.252+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fec600d0e60 con 0x7fec781052f0 2024-04-13T05:28:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.252+0000 7fec77400640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fec580a3bb0 0x7fec580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.253+0000 7fec77400640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fec580a3bb0 0x7fec580a6070 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fec78105150 tx=0x7fec68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:58.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.256+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec6009d440 con 0x7fec781052f0 2024-04-13T05:28:58.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4163490506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:28:58.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:58 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4223604859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:28:58.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.463+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fec78062bc0 con 0x7fec781052f0 2024-04-13T05:28:58.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.464+0000 7fec6fe00640 1 -- 172.21.15.57:0/2709131347 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fec6009cd40 con 0x7fec781052f0 2024-04-13T05:28:58.467 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:28:58.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fec580a3bb0 msgr2=0x7fec580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fec580a3bb0 0x7fec580a6070 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fec78105150 tx=0x7fec68005f90 comp rx=0 tx=0).stop 2024-04-13T05:28:58.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 msgr2=0x7fec7819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 0x7fec7819a800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fec6000ebc0 tx=0x7fec6000c6a0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 shutdown_connections 2024-04-13T05:28:58.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fec580a3bb0 0x7fec580a6070 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78105cb0 0x7fec781a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec781052f0 0x7fec7819a800 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 --2- 172.21.15.57:0/2709131347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec78104100 0x7fec7819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 >> 172.21.15.57:0/2709131347 conn(0x7fec780ff670 msgr2=0x7fec781085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:58.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 shutdown_connections 2024-04-13T05:28:58.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.467+0000 7fec7ce00640 1 -- 172.21.15.57:0/2709131347 wait complete. 2024-04-13T05:28:58.489 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T05:28:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4163490506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:28:58.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:58 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4223604859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:28:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4163490506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:28:58.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:58 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4223604859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 -- 172.21.15.57:0/1681331215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4103a40 msgr2=0x7fc7e410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1681331215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4103a40 0x7fc7e410a360 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009ca0 tx=0x7fc7dc02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 -- 172.21.15.57:0/1681331215 shutdown_connections 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1681331215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4103a40 0x7fc7e410a360 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1681331215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7e4103080 0x7fc7e4103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1681331215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4101ea0 0x7fc7e41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 -- 172.21.15.57:0/1681331215 >> 172.21.15.57:0/1681331215 conn(0x7fc7e40fd460 msgr2=0x7fc7e40ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:58.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 -- 172.21.15.57:0/1681331215 shutdown_connections 2024-04-13T05:28:58.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.716+0000 7fc7e9000640 1 -- 172.21.15.57:0/1681331215 wait complete. 2024-04-13T05:28:58.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.717+0000 7fc7e9000640 1 Processor -- start 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.717+0000 7fc7e9000640 1 -- start start 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7e4103080 0x7fc7e40743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4103a40 0x7fc7e40748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e9000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e410da50 con 0x7fc7e4101ea0 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e410d8d0 con 0x7fc7e4103a40 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e9000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e410dbd0 con 0x7fc7e4103080 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4075d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52016/0 (socket says 172.21.15.57:52016) 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 -- 172.21.15.57:0/1965873194 learned_addr learned my addr 172.21.15.57:0/1965873194 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e2a00640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7e4103080 0x7fc7e40743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 -- 172.21.15.57:0/1965873194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7e4103080 msgr2=0x7fc7e40743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7e4103080 0x7fc7e40743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3e00640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4103a40 0x7fc7e40748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 -- 172.21.15.57:0/1965873194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4103a40 msgr2=0x7fc7e40748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4103a40 0x7fc7e40748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3400640 1 -- 172.21.15.57:0/1965873194 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e4075180 con 0x7fc7e4101ea0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.718+0000 7fc7e3e00640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4103a40 0x7fc7e40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.719+0000 7fc7e3400640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4075d70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400d770 tx=0x7fc7d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.719+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4017070 con 0x7fc7e4101ea0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.719+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7d4009c30 con 0x7fc7e4101ea0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.719+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4003d00 con 0x7fc7e4101ea0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.719+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e41a2180 con 0x7fc7e4101ea0 2024-04-13T05:28:58.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.719+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e41a26d0 con 0x7fc7e4101ea0 2024-04-13T05:28:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.721+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e4109240 con 0x7fc7e4101ea0 2024-04-13T05:28:58.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.723+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc7d4003ea0 con 0x7fc7e4101ea0 2024-04-13T05:28:58.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.724+0000 7fc7dbe00640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7c80a3bb0 0x7fc7c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:58.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.724+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fc7d40d05c0 con 0x7fc7e4101ea0 2024-04-13T05:28:58.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.724+0000 7fc7e2a00640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7c80a3bb0 0x7fc7c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:58.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.725+0000 7fc7e2a00640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7c80a3bb0 0x7fc7c80a6070 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005fd0 tx=0x7fc7d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:58.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.731+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7d409cba0 con 0x7fc7e4101ea0 2024-04-13T05:28:58.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.948+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7e402d5e0 con 0x7fc7e4101ea0 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.948+0000 7fc7dbe00640 1 -- 172.21.15.57:0/1965873194 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+588 (secure 0 0 0) 0x7fc7d4009de0 con 0x7fc7e4101ea0 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2270359290 2024-04-14T05:05:11.407697+0000 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1023620189 2024-04-14T05:04:34.865891+0000 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/863867962 2024-04-14T05:05:11.407697+0000 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1502699862 2024-04-14T05:04:34.865891+0000 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:28:58.951 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:28:58.952 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1172825336 2024-04-14T05:04:34.865891+0000 2024-04-13T05:28:58.952 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:28:58.952 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/980869677 2024-04-14T05:05:11.407697+0000 2024-04-13T05:28:58.952 INFO:teuthology.orchestra.run.smithi057.stderr:listed 10 entries 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7c80a3bb0 msgr2=0x7fc7c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7c80a3bb0 0x7fc7c80a6070 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005fd0 tx=0x7fc7d0005850 comp rx=0 tx=0).stop 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 msgr2=0x7fc7e4075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4075d70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400d770 tx=0x7fc7d400dc30 comp rx=0 tx=0).stop 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 shutdown_connections 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fc7c80a3bb0 0x7fc7c80a6070 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4103a40 0x7fc7e40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7e4103080 0x7fc7e40743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 --2- 172.21.15.57:0/1965873194 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4075d70 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 >> 172.21.15.57:0/1965873194 conn(0x7fc7e40fd460 msgr2=0x7fc7e4106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:58.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 shutdown_connections 2024-04-13T05:28:58.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:58.951+0000 7fc7e9000640 1 -- 172.21.15.57:0/1965873194 wait complete. 2024-04-13T05:28:58.974 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-13T05:28:58.974 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-13T05:28:59.027 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:28:59.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:59 smithi057 ceph-mon[32331]: pgmap v797: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:59.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2709131347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:28:59.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:28:59 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1965873194' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:28:59.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 -- 172.21.15.57:0/4059570992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 msgr2=0x7f14681056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 --2- 172.21.15.57:0/4059570992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f14681056b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1454009900 tx=0x7f145402bc30 comp rx=0 tx=0).stop 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 -- 172.21.15.57:0/4059570992 shutdown_connections 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 --2- 172.21.15.57:0/4059570992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1468105bf0 0x7f146810c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 --2- 172.21.15.57:0/4059570992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f14681056b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 --2- 172.21.15.57:0/4059570992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1468104040 0x7f1468104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 -- 172.21.15.57:0/4059570992 >> 172.21.15.57:0/4059570992 conn(0x7f14680ff5d0 msgr2=0x7f14681019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 -- 172.21.15.57:0/4059570992 shutdown_connections 2024-04-13T05:28:59.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.293+0000 7f146ca00640 1 -- 172.21.15.57:0/4059570992 wait complete. 2024-04-13T05:28:59.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 Processor -- start 2024-04-13T05:28:59.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 -- start start 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1468104040 0x7f146819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f146819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1468105bf0 0x7f14681a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f146810fbd0 con 0x7f1468105230 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f146810fa50 con 0x7f1468105bf0 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.294+0000 7f146ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f146810fd50 con 0x7f1468104040 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f146819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f146819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52028/0 (socket says 172.21.15.57:52028) 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 -- 172.21.15.57:0/1873808366 learned_addr learned my addr 172.21.15.57:0/1873808366 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1467400640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1468104040 0x7f146819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1467e00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1468105bf0 0x7f14681a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 -- 172.21.15.57:0/1873808366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1468104040 msgr2=0x7f146819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1468104040 0x7f146819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 -- 172.21.15.57:0/1873808366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1468105bf0 msgr2=0x7f14681a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1468105bf0 0x7f14681a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 -- 172.21.15.57:0/1873808366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14681a1e70 con 0x7f1468105230 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.295+0000 7f1466a00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f146819a790 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1454000c00 tx=0x7f14540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.296+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454043020 con 0x7f1468105230 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.296+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1454004aa0 con 0x7f1468105230 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.296+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454003d00 con 0x7f1468105230 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.296+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14681a2100 con 0x7f1468105230 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.296+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14681a2600 con 0x7f1468105230 2024-04-13T05:28:59.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.297+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146810b360 con 0x7f1468105230 2024-04-13T05:28:59.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.299+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1454046020 con 0x7f1468105230 2024-04-13T05:28:59.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.300+0000 7f145fe00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f144c0a3bb0 0x7f144c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.300+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f14540f2c30 con 0x7f1468105230 2024-04-13T05:28:59.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.301+0000 7f1467400640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f144c0a3bb0 0x7f144c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.301+0000 7f1467400640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f144c0a3bb0 0x7f144c0a6070 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1458005d00 tx=0x7f1458005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:59.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.304+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f145404d020 con 0x7f1468105230 2024-04-13T05:28:59.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:59 smithi189 ceph-mon[35531]: pgmap v797: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:59.486 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2709131347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:28:59.486 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:28:59 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1965873194' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:28:59.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:59 smithi083 ceph-mon[35133]: pgmap v797: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:28:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2709131347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:28:59.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:28:59 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1965873194' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:28:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.513+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1468062bf0 con 0x7f1468105230 2024-04-13T05:28:59.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.516+0000 7f145fe00640 1 -- 172.21.15.57:0/1873808366 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7f14540bf210 con 0x7f1468105230 2024-04-13T05:28:59.519 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:28:59.520 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:28:59.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f144c0a3bb0 msgr2=0x7f144c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f144c0a3bb0 0x7f144c0a6070 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1458005d00 tx=0x7f1458005c50 comp rx=0 tx=0).stop 2024-04-13T05:28:59.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 msgr2=0x7f146819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f146819a790 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1454000c00 tx=0x7f14540042f0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 shutdown_connections 2024-04-13T05:28:59.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f144c0a3bb0 0x7f144c0a6070 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1468105bf0 0x7f14681a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1468105230 0x7f146819a790 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 --2- 172.21.15.57:0/1873808366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1468104040 0x7f146819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.519+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 >> 172.21.15.57:0/1873808366 conn(0x7f14680ff5d0 msgr2=0x7f1468108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:59.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.520+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 shutdown_connections 2024-04-13T05:28:59.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.520+0000 7f146ca00640 1 -- 172.21.15.57:0/1873808366 wait complete. 2024-04-13T05:28:59.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-13T05:08:29.596487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-13T05:14:50.989082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-13T05:14:52.058878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-13T05:14:56.085226+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-13T05:28:59.544 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-13T05:28:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 -- 172.21.15.57:0/2760753575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 msgr2=0x7f770410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 --2- 172.21.15.57:0/2760753575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f770410c580 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009ca0 tx=0x7f76fc02c0f0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 -- 172.21.15.57:0/2760753575 shutdown_connections 2024-04-13T05:28:59.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 --2- 172.21.15.57:0/2760753575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f770410c580 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 --2- 172.21.15.57:0/2760753575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704105330 0x7f77041057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 --2- 172.21.15.57:0/2760753575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7704104140 0x7f7704104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 -- 172.21.15.57:0/2760753575 >> 172.21.15.57:0/2760753575 conn(0x7f77040ff6d0 msgr2=0x7f7704101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:28:59.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 -- 172.21.15.57:0/2760753575 shutdown_connections 2024-04-13T05:28:59.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.776+0000 7f770ae00640 1 -- 172.21.15.57:0/2760753575 wait complete. 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 Processor -- start 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 -- start start 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704104140 0x7f770419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7704105330 0x7f770419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f77041a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f770410fc50 con 0x7f7704105cf0 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f770410fad0 con 0x7f7704105330 2024-04-13T05:28:59.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.778+0000 7f770ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f770410fdd0 con 0x7f7704104140 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f77041a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f77041a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52050/0 (socket says 172.21.15.57:52050) 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 -- 172.21.15.57:0/2599458263 learned_addr learned my addr 172.21.15.57:0/2599458263 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f7709a00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704104140 0x7f770419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f7709000640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7704105330 0x7f770419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 -- 172.21.15.57:0/2599458263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704104140 msgr2=0x7f770419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704104140 0x7f770419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 -- 172.21.15.57:0/2599458263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7704105330 msgr2=0x7f770419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7704105330 0x7f770419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f770a400640 1 -- 172.21.15.57:0/2599458263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77041a1f10 con 0x7f7704105cf0 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.779+0000 7f7709a00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704104140 0x7f770419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.780+0000 7f770a400640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f77041a18a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f76fc032380 tx=0x7f76fc032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.780+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc041020 con 0x7f7704105cf0 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.780+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76fc003e20 con 0x7f7704105cf0 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.780+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc039040 con 0x7f7704105cf0 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.780+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77041a21a0 con 0x7f7704105cf0 2024-04-13T05:28:59.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.780+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77041a2640 con 0x7f7704105cf0 2024-04-13T05:28:59.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.782+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770410b4e0 con 0x7f7704105cf0 2024-04-13T05:28:59.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.784+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76fc044020 con 0x7f7704105cf0 2024-04-13T05:28:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.785+0000 7f7702000640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76ec0a3c30 0x7f76ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:28:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.785+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f76fc0f2a70 con 0x7f7704105cf0 2024-04-13T05:28:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.785+0000 7f7709a00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76ec0a3c30 0x7f76ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:28:59.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.786+0000 7f7709a00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76ec0a3c30 0x7f76ec0a60f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f76f80058c0 tx=0x7f76f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:28:59.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:28:59.789+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76fc04b020 con 0x7f7704105cf0 2024-04-13T05:29:00.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.000+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7704062bc0 con 0x7f7704105cf0 2024-04-13T05:29:00.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.001+0000 7f7702000640 1 -- 172.21.15.57:0/2599458263 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f76fc03b2f0 con 0x7f7704105cf0 2024-04-13T05:29:00.004 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.004+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76ec0a3c30 msgr2=0x7f76ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.004+0000 7f770ae00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76ec0a3c30 0x7f76ec0a60f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f76f80058c0 tx=0x7f76f8005850 comp rx=0 tx=0).stop 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.004+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 msgr2=0x7f77041a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.004+0000 7f770ae00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f77041a18a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f76fc032380 tx=0x7f76fc032820 comp rx=0 tx=0).stop 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 shutdown_connections 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76ec0a3c30 0x7f76ec0a60f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7704105330 0x7f770419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7704105cf0 0x7f77041a18a0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 --2- 172.21.15.57:0/2599458263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7704104140 0x7f770419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 >> 172.21.15.57:0/2599458263 conn(0x7f77040ff6d0 msgr2=0x7f77041086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 shutdown_connections 2024-04-13T05:29:00.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.005+0000 7f770ae00640 1 -- 172.21.15.57:0/2599458263 wait complete. 2024-04-13T05:29:00.026 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-13T05:29:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.252+0000 7f233dc00640 1 -- 172.21.15.57:0/4131796351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2338104100 msgr2=0x7f2338104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.255 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.252+0000 7f233dc00640 1 --2- 172.21.15.57:0/4131796351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2338104100 0x7f2338104520 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f2328009900 tx=0x7f232802bc30 comp rx=0 tx=0).stop 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 -- 172.21.15.57:0/4131796351 shutdown_connections 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 --2- 172.21.15.57:0/4131796351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2338105cb0 0x7f233810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 --2- 172.21.15.57:0/4131796351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23381052f0 0x7f2338105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 --2- 172.21.15.57:0/4131796351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2338104100 0x7f2338104520 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 -- 172.21.15.57:0/4131796351 >> 172.21.15.57:0/4131796351 conn(0x7f23380ff670 msgr2=0x7f2338101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 -- 172.21.15.57:0/4131796351 shutdown_connections 2024-04-13T05:29:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.253+0000 7f233dc00640 1 -- 172.21.15.57:0/4131796351 wait complete. 2024-04-13T05:29:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 Processor -- start 2024-04-13T05:29:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 -- start start 2024-04-13T05:29:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2338104100 0x7f233819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 0x7f233819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2338105cb0 0x7f23381a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f233810fc90 con 0x7f23381052f0 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f233810fb10 con 0x7f2338104100 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f233dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f233810fe10 con 0x7f2338105cb0 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f2337400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 0x7f233819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f2337400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 0x7f233819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52062/0 (socket says 172.21.15.57:52062) 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.254+0000 7f2337400640 1 -- 172.21.15.57:0/1567575142 learned_addr learned my addr 172.21.15.57:0/1567575142 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337e00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2338104100 0x7f233819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f233d200640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2338105cb0 0x7f23381a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337400640 1 -- 172.21.15.57:0/1567575142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2338105cb0 msgr2=0x7f23381a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337400640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2338105cb0 0x7f23381a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337400640 1 -- 172.21.15.57:0/1567575142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2338104100 msgr2=0x7f233819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337400640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2338104100 0x7f233819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337400640 1 -- 172.21.15.57:0/1567575142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23381a1f40 con 0x7f23381052f0 2024-04-13T05:29:00.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2337400640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 0x7f233819a860 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f232400eb90 tx=0x7f232400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.255+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324004d20 con 0x7f23381052f0 2024-04-13T05:29:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.256+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f232400ee90 con 0x7f23381052f0 2024-04-13T05:29:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.256+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23381a2230 con 0x7f23381052f0 2024-04-13T05:29:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.256+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324010670 con 0x7f23381052f0 2024-04-13T05:29:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.256+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23381a2730 con 0x7f23381052f0 2024-04-13T05:29:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.257+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233810b4a0 con 0x7f23381052f0 2024-04-13T05:29:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.260+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2324005020 con 0x7f23381052f0 2024-04-13T05:29:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.261+0000 7f2334c00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f231c0a3bb0 0x7f231c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.261+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f23240d0fb0 con 0x7f23381052f0 2024-04-13T05:29:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.262+0000 7f2337e00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f231c0a3bb0 0x7f231c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.263+0000 7f2337e00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f231c0a3bb0 0x7f231c0a6070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f232802c240 tx=0x7f2328005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:00.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.266+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f232409d590 con 0x7f23381052f0 2024-04-13T05:29:00.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1873808366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:29:00.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:00 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2599458263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:29:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.474+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2338062bc0 con 0x7f23381052f0 2024-04-13T05:29:00.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.475+0000 7f2334c00640 1 -- 172.21.15.57:0/1567575142 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f232409ce90 con 0x7f23381052f0 2024-04-13T05:29:00.478 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f231c0a3bb0 msgr2=0x7f231c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f231c0a3bb0 0x7f231c0a6070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f232802c240 tx=0x7f2328005fd0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 msgr2=0x7f233819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 0x7f233819a860 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f232400eb90 tx=0x7f232400c6a0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 shutdown_connections 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f231c0a3bb0 0x7f231c0a6070 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2338105cb0 0x7f23381a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23381052f0 0x7f233819a860 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 --2- 172.21.15.57:0/1567575142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2338104100 0x7f233819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 >> 172.21.15.57:0/1567575142 conn(0x7f23380ff670 msgr2=0x7f2338108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:00.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 shutdown_connections 2024-04-13T05:29:00.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.478+0000 7f233dc00640 1 -- 172.21.15.57:0/1567575142 wait complete. 2024-04-13T05:29:00.500 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-13T05:29:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1873808366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:29:00.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:00 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2599458263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:29:00.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1873808366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:29:00.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:00 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2599458263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-13T05:29:00.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 -- 172.21.15.57:0/3320087378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c105410 msgr2=0x7f716c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 --2- 172.21.15.57:0/3320087378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c105410 0x7f716c105890 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7154009900 tx=0x7f715402bc40 comp rx=0 tx=0).stop 2024-04-13T05:29:00.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 -- 172.21.15.57:0/3320087378 shutdown_connections 2024-04-13T05:29:00.721 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 --2- 172.21.15.57:0/3320087378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f716c105dd0 0x7f716c10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 --2- 172.21.15.57:0/3320087378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c105410 0x7f716c105890 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 --2- 172.21.15.57:0/3320087378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f716c104220 0x7f716c104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.718+0000 7f7171800640 1 -- 172.21.15.57:0/3320087378 >> 172.21.15.57:0/3320087378 conn(0x7f716c0ff770 msgr2=0x7f716c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:00.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.719+0000 7f7171800640 1 -- 172.21.15.57:0/3320087378 shutdown_connections 2024-04-13T05:29:00.722 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.719+0000 7f7171800640 1 -- 172.21.15.57:0/3320087378 wait complete. 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 Processor -- start 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 -- start start 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 0x7f716c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f716c105410 0x7f716c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f716c105dd0 0x7f716c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f716c10fd60 con 0x7f716c104220 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f716c10fbe0 con 0x7f716c105dd0 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.720+0000 7f7171800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f716c10fee0 con 0x7f716c105410 2024-04-13T05:29:00.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 0x7f716c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 0x7f716c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52076/0 (socket says 172.21.15.57:52076) 2024-04-13T05:29:00.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 -- 172.21.15.57:0/1364238902 learned_addr learned my addr 172.21.15.57:0/1364238902 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716a000640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f716c105410 0x7f716c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716b400640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f716c105dd0 0x7f716c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 -- 172.21.15.57:0/1364238902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f716c105410 msgr2=0x7f716c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f716c105410 0x7f716c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 -- 172.21.15.57:0/1364238902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f716c105dd0 msgr2=0x7f716c1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f716c105dd0 0x7f716c1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.721+0000 7f716aa00640 1 -- 172.21.15.57:0/1364238902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c1a2000 con 0x7f716c104220 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.722+0000 7f716aa00640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 0x7f716c19a3e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f716400ccb0 tx=0x7f716400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.722+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164003b70 con 0x7f716c104220 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.722+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f716400ae50 con 0x7f716c104220 2024-04-13T05:29:00.725 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.722+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716400f6f0 con 0x7f716c104220 2024-04-13T05:29:00.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.722+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c1a22f0 con 0x7f716c104220 2024-04-13T05:29:00.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.722+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c1a27f0 con 0x7f716c104220 2024-04-13T05:29:00.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.724+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716c10b540 con 0x7f716c104220 2024-04-13T05:29:00.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.726+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7164004000 con 0x7f716c104220 2024-04-13T05:29:00.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.728+0000 7f7163400640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71500a3c30 0x7f71500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:00.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.728+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f71640d0d60 con 0x7f716c104220 2024-04-13T05:29:00.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.728+0000 7f716a000640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71500a3c30 0x7f71500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:00.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.729+0000 7f716a000640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71500a3c30 0x7f71500a60f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f716c19b8f0 tx=0x7f715402c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:00.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.731+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7164014070 con 0x7f716c104220 2024-04-13T05:29:00.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.937+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f716c02d890 con 0x7f716c104220 2024-04-13T05:29:00.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.938+0000 7f7163400640 1 -- 172.21.15.57:0/1364238902 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f716409d3c0 con 0x7f716c104220 2024-04-13T05:29:00.940 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-04-13T05:29:00.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.940+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71500a3c30 msgr2=0x7f71500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.940+0000 7f7171800640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71500a3c30 0x7f71500a60f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f716c19b8f0 tx=0x7f715402c810 comp rx=0 tx=0).stop 2024-04-13T05:29:00.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.940+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 msgr2=0x7f716c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:00.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.940+0000 7f7171800640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 0x7f716c19a3e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f716400ccb0 tx=0x7f716400a620 comp rx=0 tx=0).stop 2024-04-13T05:29:00.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 shutdown_connections 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f71500a3c30 0x7f71500a60f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f716c105dd0 0x7f716c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f716c105410 0x7f716c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 --2- 172.21.15.57:0/1364238902 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f716c104220 0x7f716c19a3e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 >> 172.21.15.57:0/1364238902 conn(0x7f716c0ff770 msgr2=0x7f716c108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 shutdown_connections 2024-04-13T05:29:00.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:00.941+0000 7f7171800640 1 -- 172.21.15.57:0/1364238902 wait complete. 2024-04-13T05:29:00.961 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-13T05:29:01.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.188+0000 7f62a0a00640 1 -- 172.21.15.57:0/2491193308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 msgr2=0x7f629c10c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.188+0000 7f62a0a00640 1 --2- 172.21.15.57:0/2491193308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c10c540 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f6294009900 tx=0x7f629402bc30 comp rx=0 tx=0).stop 2024-04-13T05:29:01.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 -- 172.21.15.57:0/2491193308 shutdown_connections 2024-04-13T05:29:01.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 --2- 172.21.15.57:0/2491193308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c1052f0 0x7f629c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 --2- 172.21.15.57:0/2491193308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c10c540 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 --2- 172.21.15.57:0/2491193308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f629c104100 0x7f629c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 -- 172.21.15.57:0/2491193308 >> 172.21.15.57:0/2491193308 conn(0x7f629c0ff670 msgr2=0x7f629c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:01.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 -- 172.21.15.57:0/2491193308 shutdown_connections 2024-04-13T05:29:01.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.189+0000 7f62a0a00640 1 -- 172.21.15.57:0/2491193308 wait complete. 2024-04-13T05:29:01.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 Processor -- start 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 -- start start 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f629c104100 0x7f629c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c1052f0 0x7f629c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f629c10fc90 con 0x7f629c105cb0 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f629c10fb10 con 0x7f629c1052f0 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.190+0000 7f62a0a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f629c10fe10 con 0x7f629c104100 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52086/0 (socket says 172.21.15.57:52086) 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 -- 172.21.15.57:0/3881299270 learned_addr learned my addr 172.21.15.57:0/3881299270 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629a800640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f629c104100 0x7f629c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f6299e00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c1052f0 0x7f629c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 -- 172.21.15.57:0/3881299270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f629c104100 msgr2=0x7f629c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f629c104100 0x7f629c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 -- 172.21.15.57:0/3881299270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c1052f0 msgr2=0x7f629c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c1052f0 0x7f629c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.191+0000 7f629b200640 1 -- 172.21.15.57:0/3881299270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c1a1f30 con 0x7f629c105cb0 2024-04-13T05:29:01.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.192+0000 7f629b200640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c1a18c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6294036040 tx=0x7f6294004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.194+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294043020 con 0x7f629c105cb0 2024-04-13T05:29:01.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.194+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6294004490 con 0x7f629c105cb0 2024-04-13T05:29:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.194+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c1a21c0 con 0x7f629c105cb0 2024-04-13T05:29:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.194+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294003d10 con 0x7f629c105cb0 2024-04-13T05:29:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.194+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c1a2710 con 0x7f629c105cb0 2024-04-13T05:29:01.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.195+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c10b420 con 0x7f629c105cb0 2024-04-13T05:29:01.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.197+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6294046020 con 0x7f629c105cb0 2024-04-13T05:29:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.198+0000 7f6293400640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62800a3bb0 0x7f62800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.198+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f62940f2c80 con 0x7f629c105cb0 2024-04-13T05:29:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.199+0000 7f629a800640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62800a3bb0 0x7f62800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.199+0000 7f629a800640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62800a3bb0 0x7f62800a6070 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f629c105150 tx=0x7f628c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:01.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.203+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629404d020 con 0x7f629c105cb0 2024-04-13T05:29:01.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:01 smithi057 ceph-mon[32331]: pgmap v798: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:29:01.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1567575142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:29:01.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:01 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1364238902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:29:01.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.411+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f629c062bc0 con 0x7f629c105cb0 2024-04-13T05:29:01.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.412+0000 7f6293400640 1 -- 172.21.15.57:0/3881299270 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f62940bf260 con 0x7f629c105cb0 2024-04-13T05:29:01.415 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-04-13T05:29:01.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62800a3bb0 msgr2=0x7f62800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62800a3bb0 0x7f62800a6070 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f629c105150 tx=0x7f628c009870 comp rx=0 tx=0).stop 2024-04-13T05:29:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 msgr2=0x7f629c1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c1a18c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6294036040 tx=0x7f6294004270 comp rx=0 tx=0).stop 2024-04-13T05:29:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 shutdown_connections 2024-04-13T05:29:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f62800a3bb0 0x7f62800a6070 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c1052f0 0x7f629c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f629c105cb0 0x7f629c1a18c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 --2- 172.21.15.57:0/3881299270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f629c104100 0x7f629c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.415+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 >> 172.21.15.57:0/3881299270 conn(0x7f629c0ff670 msgr2=0x7f629c1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:01.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.416+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 shutdown_connections 2024-04-13T05:29:01.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.416+0000 7f62a0a00640 1 -- 172.21.15.57:0/3881299270 wait complete. 2024-04-13T05:29:01.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-13T05:29:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:01 smithi083 ceph-mon[35133]: pgmap v798: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:29:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1567575142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:29:01.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:01 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1364238902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:29:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:01 smithi189 ceph-mon[35531]: pgmap v798: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-13T05:29:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1567575142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-13T05:29:01.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:01 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1364238902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 -- 172.21.15.57:0/1062652236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94801052f0 msgr2=0x7f9480105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 --2- 172.21.15.57:0/1062652236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94801052f0 0x7f9480105770 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f946c009900 tx=0x7f946c02bc30 comp rx=0 tx=0).stop 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 -- 172.21.15.57:0/1062652236 shutdown_connections 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 --2- 172.21.15.57:0/1062652236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480105cb0 0x7f948010c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 --2- 172.21.15.57:0/1062652236 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94801052f0 0x7f9480105770 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 --2- 172.21.15.57:0/1062652236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9480104100 0x7f9480104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 -- 172.21.15.57:0/1062652236 >> 172.21.15.57:0/1062652236 conn(0x7f94800ff670 msgr2=0x7f9480101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 -- 172.21.15.57:0/1062652236 shutdown_connections 2024-04-13T05:29:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.667+0000 7f947f200640 1 -- 172.21.15.57:0/1062652236 wait complete. 2024-04-13T05:29:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.668+0000 7f947f200640 1 Processor -- start 2024-04-13T05:29:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 -- start start 2024-04-13T05:29:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480104100 0x7f948019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.671 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94801052f0 0x7f948019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 0x7f94801a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f948010fc90 con 0x7f9480105cb0 2024-04-13T05:29:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f948010fb10 con 0x7f9480104100 2024-04-13T05:29:01.672 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f948010fe10 con 0x7f94801052f0 2024-04-13T05:29:01.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 0x7f94801a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 0x7f94801a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52094/0 (socket says 172.21.15.57:52094) 2024-04-13T05:29:01.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.669+0000 7f947e800640 1 -- 172.21.15.57:0/1926800765 learned_addr learned my addr 172.21.15.57:0/1926800765 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:01.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947de00640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480104100 0x7f948019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947d400640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94801052f0 0x7f948019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947e800640 1 -- 172.21.15.57:0/1926800765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94801052f0 msgr2=0x7f948019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947e800640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94801052f0 0x7f948019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947e800640 1 -- 172.21.15.57:0/1926800765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480104100 msgr2=0x7f948019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947e800640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480104100 0x7f948019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947e800640 1 -- 172.21.15.57:0/1926800765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801a1f60 con 0x7f9480105cb0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947de00640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480104100 0x7f948019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.670+0000 7f947e800640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 0x7f94801a18f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f947800eaa0 tx=0x7f947800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.671+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947800cdb0 con 0x7f9480105cb0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.671+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801a2250 con 0x7f9480105cb0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.671+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9478005020 con 0x7f9480105cb0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.671+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94780116d0 con 0x7f9480105cb0 2024-04-13T05:29:01.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.671+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a2720 con 0x7f9480105cb0 2024-04-13T05:29:01.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.672+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948010b440 con 0x7f9480105cb0 2024-04-13T05:29:01.676 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.674+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f94780051d0 con 0x7f9480105cb0 2024-04-13T05:29:01.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.675+0000 7f9476a00640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94640a3bb0 0x7f94640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:01.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.675+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f94780d0ab0 con 0x7f9480105cb0 2024-04-13T05:29:01.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.676+0000 7f947de00640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94640a3bb0 0x7f94640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:01.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.676+0000 7f947de00640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94640a3bb0 0x7f94640a6070 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f94700058c0 tx=0x7f9470005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:01.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.680+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9478016030 con 0x7f9480105cb0 2024-04-13T05:29:01.904 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.901+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9480062bc0 con 0x7f9480105cb0 2024-04-13T05:29:01.905 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.903+0000 7f9476a00640 1 -- 172.21.15.57:0/1926800765 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+588 (secure 0 0 0) 0x7f947809d090 con 0x7f9480105cb0 2024-04-13T05:29:01.905 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2270359290 2024-04-14T05:05:11.407697+0000 2024-04-13T05:29:01.905 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1023620189 2024-04-14T05:04:34.865891+0000 2024-04-13T05:29:01.905 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:29:01.905 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/863867962 2024-04-14T05:05:11.407697+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1502699862 2024-04-14T05:04:34.865891+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/570139496 2024-04-14T05:04:34.865891+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1172825336 2024-04-14T05:04:34.865891+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/3670387557 2024-04-14T05:05:11.407697+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/980869677 2024-04-14T05:05:11.407697+0000 2024-04-13T05:29:01.906 INFO:teuthology.orchestra.run.smithi057.stderr:listed 10 entries 2024-04-13T05:29:01.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.905+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94640a3bb0 msgr2=0x7f94640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.905+0000 7f947f200640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94640a3bb0 0x7f94640a6070 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f94700058c0 tx=0x7f9470005850 comp rx=0 tx=0).stop 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.905+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 msgr2=0x7f94801a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.905+0000 7f947f200640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 0x7f94801a18f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f947800eaa0 tx=0x7f947800ef60 comp rx=0 tx=0).stop 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 shutdown_connections 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f94640a3bb0 0x7f94640a6070 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9480105cb0 0x7f94801a18f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94801052f0 0x7f948019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 --2- 172.21.15.57:0/1926800765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9480104100 0x7f948019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 >> 172.21.15.57:0/1926800765 conn(0x7f94800ff670 msgr2=0x7f9480108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 shutdown_connections 2024-04-13T05:29:01.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:01.906+0000 7f947f200640 1 -- 172.21.15.57:0/1926800765 wait complete. 2024-04-13T05:29:01.929 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2024-04-13T05:29:01.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:29:01.929 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-13T05:29:02.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-13T05:29:02.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[87665]: fuse finished with error 0 and tester_r 0 2024-04-13T05:29:02.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3881299270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:29:02.485 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:02 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1926800765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:29:02.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:02 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3881299270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:29:02.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:02 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1926800765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:29:02.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:02 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3881299270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-13T05:29:02.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:02 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1926800765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-13T05:29:03.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:03 smithi083 ceph-mon[35133]: pgmap v799: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:29:03.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:03 smithi057 ceph-mon[32331]: pgmap v799: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:29:03.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:03 smithi189 ceph-mon[35531]: pgmap v799: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-13T05:29:05.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:05 smithi057 ceph-mon[32331]: pgmap v800: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-13T05:29:05.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:05 smithi083 ceph-mon[35133]: pgmap v800: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-13T05:29:05.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:05 smithi189 ceph-mon[35531]: pgmap v800: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-13T05:29:07.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:07 smithi083 ceph-mon[35133]: pgmap v801: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-13T05:29:07.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:07 smithi057 ceph-mon[32331]: pgmap v801: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-13T05:29:07.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:07 smithi189 ceph-mon[35531]: pgmap v801: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-13T05:29:08.135 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T05:29:08.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:29:08.135 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-13T05:29:08.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:29:08.201 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-04-13T05:29:08.257 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-13T05:29:08.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:29:08.257 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-13T05:29:08.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-13T05:29:08.358 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2024-04-13T05:29:08.476 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-13T05:29:08.487 INFO:tasks.cephadm:Teardown begin 2024-04-13T05:29:08.487 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-13T05:29:08.488 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:29:08.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 -- 172.21.15.57:0/2379709518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109630 msgr2=0x7f1c08109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:08.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/2379709518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109630 0x7f1c08109ab0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ca0 tx=0x7f1bf002c100 comp rx=0 tx=0).stop 2024-04-13T05:29:08.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 -- 172.21.15.57:0/2379709518 shutdown_connections 2024-04-13T05:29:08.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/2379709518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c08109ff0 0x7f1c08110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/2379709518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109630 0x7f1c08109ab0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/2379709518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c08108440 0x7f1c08108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.684+0000 7f1c0ca00640 1 -- 172.21.15.57:0/2379709518 >> 172.21.15.57:0/2379709518 conn(0x7f1c08077400 msgr2=0x7f1c08079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:08.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.685+0000 7f1c0ca00640 1 -- 172.21.15.57:0/2379709518 shutdown_connections 2024-04-13T05:29:08.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.685+0000 7f1c0ca00640 1 -- 172.21.15.57:0/2379709518 wait complete. 2024-04-13T05:29:08.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 Processor -- start 2024-04-13T05:29:08.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 -- start start 2024-04-13T05:29:08.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c08108440 0x7f1c0819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:08.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c08109630 0x7f1c0819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:08.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 0x7f1c081a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:08.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113f80 con 0x7f1c08109ff0 2024-04-13T05:29:08.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08113e00 con 0x7f1c08109630 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c0ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08114100 con 0x7f1c08108440 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c07e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 0x7f1c081a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c07e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 0x7f1c081a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54780/0 (socket says 172.21.15.57:54780) 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.686+0000 7f1c07e00640 1 -- 172.21.15.57:0/1219269445 learned_addr learned my addr 172.21.15.57:0/1219269445 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c06a00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c08109630 0x7f1c0819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07400640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c08108440 0x7f1c0819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:08.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07e00640 1 -- 172.21.15.57:0/1219269445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c08108440 msgr2=0x7f1c0819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07e00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c08108440 0x7f1c0819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07e00640 1 -- 172.21.15.57:0/1219269445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c08109630 msgr2=0x7f1c0819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07e00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c08109630 0x7f1c0819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07e00640 1 -- 172.21.15.57:0/1219269445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c081a62b0 con 0x7f1c08109ff0 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1c07e00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 0x7f1c081a5c40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000d950 tx=0x7f1c0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00017030 con 0x7f1c08109ff0 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.687+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c00004020 con 0x7f1c08109ff0 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.688+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a65a0 con 0x7f1c08109ff0 2024-04-13T05:29:08.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.688+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00003b20 con 0x7f1c08109ff0 2024-04-13T05:29:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.688+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a6a70 con 0x7f1c08109ff0 2024-04-13T05:29:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.689+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0810f7e0 con 0x7f1c08109ff0 2024-04-13T05:29:08.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.691+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c0001a070 con 0x7f1c08109ff0 2024-04-13T05:29:08.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.693+0000 7f1bffe00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1bec0a3bb0 0x7f1bec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:08.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.693+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f1c000d00d0 con 0x7f1c08109ff0 2024-04-13T05:29:08.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.693+0000 7f1c07400640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1bec0a3bb0 0x7f1bec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:08.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.694+0000 7f1c07400640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1bec0a3bb0 0x7f1bec0a6070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8005fd0 tx=0x7f1bf8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:08.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.698+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c00021070 con 0x7f1c08109ff0 2024-04-13T05:29:08.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.949+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c08062bc0 con 0x7f1c08109ff0 2024-04-13T05:29:08.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.951+0000 7f1bffe00640 1 -- 172.21.15.57:0/1219269445 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11864 (secure 0 0 0) 0x7f1c0009c6b0 con 0x7f1c08109ff0 2024-04-13T05:29:08.954 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:29:08.954 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:53.740918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:29:08.955 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-04-13T05:29:08.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.954+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1bec0a3bb0 msgr2=0x7f1bec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:08.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.954+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1bec0a3bb0 0x7f1bec0a6070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8005fd0 tx=0x7f1bf8005850 comp rx=0 tx=0).stop 2024-04-13T05:29:08.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.954+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 msgr2=0x7f1c081a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.954+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 0x7f1c081a5c40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000d950 tx=0x7f1c0000de10 comp rx=0 tx=0).stop 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 shutdown_connections 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f1bec0a3bb0 0x7f1bec0a6070 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c08109630 0x7f1c0819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c08109ff0 0x7f1c081a5c40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 --2- 172.21.15.57:0/1219269445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c08108440 0x7f1c0819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 >> 172.21.15.57:0/1219269445 conn(0x7f1c08077400 msgr2=0x7f1c0810c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 shutdown_connections 2024-04-13T05:29:08.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:08.955+0000 7f1c0ca00640 1 -- 172.21.15.57:0/1219269445 wait complete. 2024-04-13T05:29:08.978 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-13T05:29:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 -- 172.21.15.57:0/2358675652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109630 msgr2=0x7f80f8109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 --2- 172.21.15.57:0/2358675652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109630 0x7f80f8109ab0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009900 tx=0x7f80e402bc30 comp rx=0 tx=0).stop 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 -- 172.21.15.57:0/2358675652 shutdown_connections 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 --2- 172.21.15.57:0/2358675652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f8109ff0 0x7f80f8110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 --2- 172.21.15.57:0/2358675652 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109630 0x7f80f8109ab0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 --2- 172.21.15.57:0/2358675652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80f8108440 0x7f80f8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 -- 172.21.15.57:0/2358675652 >> 172.21.15.57:0/2358675652 conn(0x7f80f8077400 msgr2=0x7f80f8079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 -- 172.21.15.57:0/2358675652 shutdown_connections 2024-04-13T05:29:09.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.199+0000 7f80fca00640 1 -- 172.21.15.57:0/2358675652 wait complete. 2024-04-13T05:29:09.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.200+0000 7f80fca00640 1 Processor -- start 2024-04-13T05:29:09.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.200+0000 7f80fca00640 1 -- start start 2024-04-13T05:29:09.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80fca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80f8108440 0x7f80f819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80fca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f8109630 0x7f80f819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80fca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 0x7f80f81a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80fca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8113f80 con 0x7f80f8109ff0 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80fca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8113e00 con 0x7f80f8108440 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80fca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8114100 con 0x7f80f8109630 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80f7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 0x7f80f81a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80f7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 0x7f80f81a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54786/0 (socket says 172.21.15.57:54786) 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80f7200640 1 -- 172.21.15.57:0/3112580271 learned_addr learned my addr 172.21.15.57:0/3112580271 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80f6800640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80f8108440 0x7f80f819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.201+0000 7f80f5e00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f8109630 0x7f80f819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80f7200640 1 -- 172.21.15.57:0/3112580271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f8109630 msgr2=0x7f80f819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80f7200640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f8109630 0x7f80f819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80f7200640 1 -- 172.21.15.57:0/3112580271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80f8108440 msgr2=0x7f80f819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80f7200640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80f8108440 0x7f80f819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80f7200640 1 -- 172.21.15.57:0/3112580271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f81a6260 con 0x7f80f8109ff0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80f7200640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 0x7f80f81a5bf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f80f000eaa0 tx=0x7f80f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f000cdb0 con 0x7f80f8109ff0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f81a6550 con 0x7f80f8109ff0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f0005020 con 0x7f80f8109ff0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.202+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f00116d0 con 0x7f80f8109ff0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.203+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81a6af0 con 0x7f80f8109ff0 2024-04-13T05:29:09.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.204+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f810f7e0 con 0x7f80f8109ff0 2024-04-13T05:29:09.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.206+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f80f00051d0 con 0x7f80f8109ff0 2024-04-13T05:29:09.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.207+0000 7f80ef400640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80dc0a3bb0 0x7f80dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.207+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f80f00d0a50 con 0x7f80f8109ff0 2024-04-13T05:29:09.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.207+0000 7f80f6800640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80dc0a3bb0 0x7f80dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.208+0000 7f80f6800640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80dc0a3bb0 0x7f80dc0a6070 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f80e8005fd0 tx=0x7f80e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:09.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.212+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80f0016030 con 0x7f80f8109ff0 2024-04-13T05:29:09.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.467+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80f8062bc0 con 0x7f80f8109ff0 2024-04-13T05:29:09.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.469+0000 7f80ef400640 1 -- 172.21.15.57:0/3112580271 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11864 (secure 0 0 0) 0x7f80f009d030 con 0x7f80f8109ff0 2024-04-13T05:29:09.472 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:29:09.472 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/378251169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":378251169},{"type":"v1","addr":"172.21.15.57:6835","nonce":378251169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3282486019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3282486019},{"type":"v1","addr":"172.21.15.57:6837","nonce":3282486019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/59311703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":59311703},{"type":"v1","addr":"172.21.15.57:6839","nonce":59311703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/2975868204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":2975868204},{"type":"v1","addr":"172.21.15.57:6841","nonce":2975868204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/550433387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":550433387},{"type":"v1","addr":"172.21.15.189:6833","nonce":550433387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3836752416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3836752416},{"type":"v1","addr":"172.21.15.189:6835","nonce":3836752416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-13T05:14:53.072497+0000","modified":"2024-04-13T05:25:53.740918+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34499,"mds_1":34457,"mds_2":34427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/563533422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":563533422},{"type":"v1","addr":"172.21.15.189:6837","nonce":563533422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/1915044018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":1915044018},{"type":"v1","addr":"172.21.15.189:6839","nonce":1915044018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34397":{"gid":34397,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6833/617814721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":617814721},{"type":"v1","addr":"172.21.15.83:6833","nonce":617814721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6835/3245344075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3245344075},{"type":"v1","addr":"172.21.15.83:6835","nonce":3245344075}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34457":{"gid":34457,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.83:6837/741118790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":741118790},{"type":"v1","addr":"172.21.15.83:6837","nonce":741118790}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34499":{"gid":34499,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6839/3095811120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":3095811120},{"type":"v1","addr":"172.21.15.83:6839","nonce":3095811120}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34499,"qdb_cluster":[34427,34457,34499]},"id":1}]} 2024-04-13T05:29:09.472 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 33 2024-04-13T05:29:09.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.471+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80dc0a3bb0 msgr2=0x7f80dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.471+0000 7f80fca00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80dc0a3bb0 0x7f80dc0a6070 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f80e8005fd0 tx=0x7f80e8005850 comp rx=0 tx=0).stop 2024-04-13T05:29:09.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.471+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 msgr2=0x7f80f81a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.471+0000 7f80fca00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 0x7f80f81a5bf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f80f000eaa0 tx=0x7f80f000ef60 comp rx=0 tx=0).stop 2024-04-13T05:29:09.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 shutdown_connections 2024-04-13T05:29:09.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f80dc0a3bb0 0x7f80dc0a6070 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80f8109ff0 0x7f80f81a5bf0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f8109630 0x7f80f819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 --2- 172.21.15.57:0/3112580271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80f8108440 0x7f80f819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 >> 172.21.15.57:0/3112580271 conn(0x7f80f8077400 msgr2=0x7f80f810c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:09.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 shutdown_connections 2024-04-13T05:29:09.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.472+0000 7f80fca00640 1 -- 172.21.15.57:0/3112580271 wait complete. 2024-04-13T05:29:09.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:09 smithi057 ceph-mon[32331]: pgmap v802: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-13T05:29:09.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:09 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/1219269445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:29:09.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-13T05:29:09.501 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:09 smithi083 ceph-mon[35133]: pgmap v802: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-13T05:29:09.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:09 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/1219269445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:29:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:09 smithi189 ceph-mon[35531]: pgmap v802: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-13T05:29:09.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:09 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/1219269445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:29:09.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 -- 172.21.15.57:0/1639889269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8109ff0 msgr2=0x7fdeb8110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 --2- 172.21.15.57:0/1639889269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8109ff0 0x7fdeb8110880 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009ca0 tx=0x7fdeb002c0f0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 -- 172.21.15.57:0/1639889269 shutdown_connections 2024-04-13T05:29:09.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 --2- 172.21.15.57:0/1639889269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8109ff0 0x7fdeb8110880 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 --2- 172.21.15.57:0/1639889269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 0x7fdeb8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 --2- 172.21.15.57:0/1639889269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb8108440 0x7fdeb8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 -- 172.21.15.57:0/1639889269 >> 172.21.15.57:0/1639889269 conn(0x7fdeb8077400 msgr2=0x7fdeb8079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.731+0000 7fdeb7200640 1 -- 172.21.15.57:0/1639889269 shutdown_connections 2024-04-13T05:29:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.732+0000 7fdeb7200640 1 -- 172.21.15.57:0/1639889269 wait complete. 2024-04-13T05:29:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.733+0000 7fdeb7200640 1 Processor -- start 2024-04-13T05:29:09.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.733+0000 7fdeb7200640 1 -- start start 2024-04-13T05:29:09.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.733+0000 7fdeb7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 0x7fdeb819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.733+0000 7fdeb7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 0x7fdeb819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb8109ff0 0x7fdeb81a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8113f80 con 0x7fdeb8108440 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8113e00 con 0x7fdeb8109ff0 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8114100 con 0x7fdeb8109630 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 0x7fdeb819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 0x7fdeb819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54806/0 (socket says 172.21.15.57:54806) 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 -- 172.21.15.57:0/2004226268 learned_addr learned my addr 172.21.15.57:0/2004226268 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5400640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 0x7fdeb819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 -- 172.21.15.57:0/2004226268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 msgr2=0x7fdeb819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 0x7fdeb819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 -- 172.21.15.57:0/2004226268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb8109ff0 msgr2=0x7fdeb81a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb8109ff0 0x7fdeb81a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5e00640 1 -- 172.21.15.57:0/2004226268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb81a6210 con 0x7fdeb8108440 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.734+0000 7fdeb5400640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 0x7fdeb819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.735+0000 7fdeb5e00640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 0x7fdeb819e5f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdea800d770 tx=0x7fdea800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.735+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8017070 con 0x7fdeb8108440 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.735+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdea8009c30 con 0x7fdeb8108440 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.735+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8003d00 con 0x7fdeb8108440 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.735+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb81a6500 con 0x7fdeb8108440 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.735+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb81a6aa0 con 0x7fdeb8108440 2024-04-13T05:29:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.736+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb810f830 con 0x7fdeb8108440 2024-04-13T05:29:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.738+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdea8003ea0 con 0x7fdeb8108440 2024-04-13T05:29:09.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.740+0000 7fdeaea00640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fde9c0a3c30 0x7fde9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.740+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7fdea80d0560 con 0x7fdeb8108440 2024-04-13T05:29:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.740+0000 7fdeb5400640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fde9c0a3c30 0x7fde9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:09.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.741+0000 7fdeb5400640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fde9c0a3c30 0x7fde9c0a60f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdeb819fb00 tx=0x7fdea4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.745+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdea809cb40 con 0x7fdeb8108440 2024-04-13T05:29:09.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.955+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdeb8062c60 con 0x7fdeb8108440 2024-04-13T05:29:09.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.960+0000 7fdeaea00640 1 -- 172.21.15.57:0/2004226268 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24695 (secure 0 0 0) 0x7fdea8009de0 con 0x7fdeb8108440 2024-04-13T05:29:09.963 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-13T05:29:09.963 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"0a178454-f953-11ee-bc8d-c7b262605968","created":"2024-04-13T05:04:01.249890+0000","modified":"2024-04-13T05:16:14.199430+0000","last_up_change":"2024-04-13T05:13:45.212945+0000","last_in_change":"2024-04-13T05:13:19.222393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-13T05:08:29.596487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-13T05:14:50.989082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-13T05:14:52.058878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-13T05:14:56.085226+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8f852b23-d1b5-4c86-8946-0a9213323ecd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6803","nonce":3403486452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6805","nonce":3403486452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6809","nonce":3403486452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3403486452},{"type":"v1","addr":"172.21.15.57:6807","nonce":3403486452}]},"public_addr":"172.21.15.57:6803/3403486452","cluster_addr":"172.21.15.57:6805/3403486452","heartbeat_back_addr":"172.21.15.57:6809/3403486452","heartbeat_front_addr":"172.21.15.57:6807/3403486452","state":["exists","up"]},{"osd":1,"uuid":"3eb9f311-c136-431f-a658-5084aa4c2fc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6801","nonce":3349778520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6803","nonce":3349778520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6807","nonce":3349778520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3349778520},{"type":"v1","addr":"172.21.15.83:6805","nonce":3349778520}]},"public_addr":"172.21.15.83:6801/3349778520","cluster_addr":"172.21.15.83:6803/3349778520","heartbeat_back_addr":"172.21.15.83:6807/3349778520","heartbeat_front_addr":"172.21.15.83:6805/3349778520","state":["exists","up"]},{"osd":2,"uuid":"83f5d37c-64d6-4a58-81c2-c0f1b7a492ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6801","nonce":2908933106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6803","nonce":2908933106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6807","nonce":2908933106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2908933106},{"type":"v1","addr":"172.21.15.189:6805","nonce":2908933106}]},"public_addr":"172.21.15.189:6801/2908933106","cluster_addr":"172.21.15.189:6803/2908933106","heartbeat_back_addr":"172.21.15.189:6807/2908933106","heartbeat_front_addr":"172.21.15.189:6805/2908933106","state":["exists","up"]},{"osd":3,"uuid":"1e4840c9-26d6-4827-bca1-2fe9391d51ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6811","nonce":744892058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6813","nonce":744892058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6817","nonce":744892058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":744892058},{"type":"v1","addr":"172.21.15.57:6815","nonce":744892058}]},"public_addr":"172.21.15.57:6811/744892058","cluster_addr":"172.21.15.57:6813/744892058","heartbeat_back_addr":"172.21.15.57:6817/744892058","heartbeat_front_addr":"172.21.15.57:6815/744892058","state":["exists","up"]},{"osd":4,"uuid":"03b6258a-179f-48c4-acce-0e2929065e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6809","nonce":4161937818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6811","nonce":4161937818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6815","nonce":4161937818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":4161937818},{"type":"v1","addr":"172.21.15.83:6813","nonce":4161937818}]},"public_addr":"172.21.15.83:6809/4161937818","cluster_addr":"172.21.15.83:6811/4161937818","heartbeat_back_addr":"172.21.15.83:6815/4161937818","heartbeat_front_addr":"172.21.15.83:6813/4161937818","state":["exists","up"]},{"osd":5,"uuid":"ad5b3787-2395-43e1-9595-98db29fc5e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6809","nonce":3029406488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6811","nonce":3029406488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6815","nonce":3029406488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3029406488},{"type":"v1","addr":"172.21.15.189:6813","nonce":3029406488}]},"public_addr":"172.21.15.189:6809/3029406488","cluster_addr":"172.21.15.189:6811/3029406488","heartbeat_back_addr":"172.21.15.189:6815/3029406488","heartbeat_front_addr":"172.21.15.189:6813/3029406488","state":["exists","up"]},{"osd":6,"uuid":"b7e957fe-81a0-4abe-828b-8c1cb7b14983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6819","nonce":830219588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6821","nonce":830219588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6825","nonce":830219588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":830219588},{"type":"v1","addr":"172.21.15.57:6823","nonce":830219588}]},"public_addr":"172.21.15.57:6819/830219588","cluster_addr":"172.21.15.57:6821/830219588","heartbeat_back_addr":"172.21.15.57:6825/830219588","heartbeat_front_addr":"172.21.15.57:6823/830219588","state":["exists","up"]},{"osd":7,"uuid":"14b5cd3c-1c3d-4896-949e-5f77372aac7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6817","nonce":977203651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6819","nonce":977203651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6823","nonce":977203651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":977203651},{"type":"v1","addr":"172.21.15.83:6821","nonce":977203651}]},"public_addr":"172.21.15.83:6817/977203651","cluster_addr":"172.21.15.83:6819/977203651","heartbeat_back_addr":"172.21.15.83:6823/977203651","heartbeat_front_addr":"172.21.15.83:6821/977203651","state":["exists","up"]},{"osd":8,"uuid":"041dc02b-fe99-4ac2-836c-33c8ac7ccc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6817","nonce":2146992789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6819","nonce":2146992789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6823","nonce":2146992789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2146992789},{"type":"v1","addr":"172.21.15.189:6821","nonce":2146992789}]},"public_addr":"172.21.15.189:6817/2146992789","cluster_addr":"172.21.15.189:6819/2146992789","heartbeat_back_addr":"172.21.15.189:6823/2146992789","heartbeat_front_addr":"172.21.15.189:6821/2146992789","state":["exists","up"]},{"osd":9,"uuid":"f43fea2e-9521-419d-85de-fc3068b68ffd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6827","nonce":772937506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6829","nonce":772937506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6833","nonce":772937506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":772937506},{"type":"v1","addr":"172.21.15.57:6831","nonce":772937506}]},"public_addr":"172.21.15.57:6827/772937506","cluster_addr":"172.21.15.57:6829/772937506","heartbeat_back_addr":"172.21.15.57:6833/772937506","heartbeat_front_addr":"172.21.15.57:6831/772937506","state":["exists","up"]},{"osd":10,"uuid":"bdef097c-4d96-492e-9d28-1eba9cc759fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6825","nonce":3322982382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6827","nonce":3322982382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6831","nonce":3322982382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3322982382},{"type":"v1","addr":"172.21.15.83:6829","nonce":3322982382}]},"public_addr":"172.21.15.83:6825/3322982382","cluster_addr":"172.21.15.83:6827/3322982382","heartbeat_back_addr":"172.21.15.83:6831/3322982382","heartbeat_front_addr":"172.21.15.83:6829/3322982382","state":["exists","up"]},{"osd":11,"uuid":"476f1e68-924c-49f6-87b8-bcb1267ff3e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6825","nonce":368428802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6827","nonce":368428802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6831","nonce":368428802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":368428802},{"type":"v1","addr":"172.21.15.189:6829","nonce":368428802}]},"public_addr":"172.21.15.189:6825/368428802","cluster_addr":"172.21.15.189:6827/368428802","heartbeat_back_addr":"172.21.15.189:6831/368428802","heartbeat_front_addr":"172.21.15.189:6829/368428802","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:19.391726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:07:52.492047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:08:25.949860+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:02.093194+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:09:35.686011+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:10.114753+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:10:45.903777+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:20.203987+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:11:55.282771+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:12:32.200668+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:06.866285+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-13T05:13:42.592851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8},{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2270359290":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1023620189":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/863867962":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/1502699862":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6801/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:6800/570139496":"2024-04-14T05:04:34.865891+0000","172.21.15.57:0/1172825336":"2024-04-14T05:04:34.865891+0000","172.21.15.57:6800/3670387557":"2024-04-14T05:05:11.407697+0000","172.21.15.57:0/980869677":"2024-04-14T05:05:11.407697+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-13T05:29:09.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.962+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fde9c0a3c30 msgr2=0x7fde9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.962+0000 7fdeb7200640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fde9c0a3c30 0x7fde9c0a60f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdeb819fb00 tx=0x7fdea4005e50 comp rx=0 tx=0).stop 2024-04-13T05:29:09.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.962+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 msgr2=0x7fdeb819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.962+0000 7fdeb7200640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 0x7fdeb819e5f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdea800d770 tx=0x7fdea800dc30 comp rx=0 tx=0).stop 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 shutdown_connections 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fde9c0a3c30 0x7fde9c0a60f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdeb8109ff0 0x7fdeb81a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdeb8109630 0x7fdeb819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 --2- 172.21.15.57:0/2004226268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdeb8108440 0x7fdeb819e5f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 >> 172.21.15.57:0/2004226268 conn(0x7fdeb8077400 msgr2=0x7fdeb810ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 shutdown_connections 2024-04-13T05:29:09.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:09.963+0000 7fdeb7200640 1 -- 172.21.15.57:0/2004226268 wait complete. 2024-04-13T05:29:09.986 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 -- 172.21.15.57:0/499176947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 msgr2=0x7f8228103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 --2- 172.21.15.57:0/499176947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228103610 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8214009930 tx=0x7f821402bc30 comp rx=0 tx=0).stop 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 -- 172.21.15.57:0/499176947 shutdown_connections 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 --2- 172.21.15.57:0/499176947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228103b50 0x7f822810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 --2- 172.21.15.57:0/499176947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228103610 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 --2- 172.21.15.57:0/499176947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82281001c0 0x7f82281005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 -- 172.21.15.57:0/499176947 >> 172.21.15.57:0/499176947 conn(0x7f82280fbaf0 msgr2=0x7f82280fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 -- 172.21.15.57:0/499176947 shutdown_connections 2024-04-13T05:29:10.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.202+0000 7f822ec00640 1 -- 172.21.15.57:0/499176947 wait complete. 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.203+0000 7f822ec00640 1 Processor -- start 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.203+0000 7f822ec00640 1 -- start start 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82281001c0 0x7f8228102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ec00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228103b50 0x7f8228101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ec00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f822810da40 con 0x7f8228103190 2024-04-13T05:29:10.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f822810d8c0 con 0x7f8228103b50 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f822810dbc0 con 0x7f82281001c0 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54822/0 (socket says 172.21.15.57:54822) 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 -- 172.21.15.57:0/149937583 learned_addr learned my addr 172.21.15.57:0/149937583 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822d800640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82281001c0 0x7f8228102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 -- 172.21.15.57:0/149937583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82281001c0 msgr2=0x7f8228102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82281001c0 0x7f8228102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 -- 172.21.15.57:0/149937583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228103b50 msgr2=0x7f8228101680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228103b50 0x7f8228101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.204+0000 7f822ce00640 1 -- 172.21.15.57:0/149937583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8228101f10 con 0x7f8228103190 2024-04-13T05:29:10.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.205+0000 7f822ce00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228101140 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8214009230 tx=0x7f82140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:10.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.205+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214043020 con 0x7f8228103190 2024-04-13T05:29:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.205+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82281a22e0 con 0x7f8228103190 2024-04-13T05:29:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.205+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82140044c0 con 0x7f8228103190 2024-04-13T05:29:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.205+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214003bd0 con 0x7f8228103190 2024-04-13T05:29:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.205+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a2830 con 0x7f8228103190 2024-04-13T05:29:10.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.206+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82281093b0 con 0x7f8228103190 2024-04-13T05:29:10.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.209+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8214046020 con 0x7f8228103190 2024-04-13T05:29:10.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.211+0000 7f8226000640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f820c0a3bb0 0x7f820c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.211+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9158+0+0 (secure 0 0 0) 0x7f82140f2bb0 con 0x7f8228103190 2024-04-13T05:29:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.212+0000 7f822d800640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f820c0a3bb0 0x7f820c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.212+0000 7f822d800640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f820c0a3bb0 0x7f820c0a6070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f822806a2c0 tx=0x7f8218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:10.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.216+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f821404d020 con 0x7f8228103190 2024-04-13T05:29:10.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.467+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8228062bc0 con 0x7f8228103190 2024-04-13T05:29:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.473+0000 7f8226000640 1 -- 172.21.15.57:0/149937583 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7f821403c350 con 0x7f8228103190 2024-04-13T05:29:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-13T05:29:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.476+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f820c0a3bb0 msgr2=0x7f820c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.476+0000 7f822ec00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f820c0a3bb0 0x7f820c0a6070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f822806a2c0 tx=0x7f8218005f90 comp rx=0 tx=0).stop 2024-04-13T05:29:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.476+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 msgr2=0x7f8228101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.476+0000 7f822ec00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228101140 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8214009230 tx=0x7f82140040c0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.477+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 shutdown_connections 2024-04-13T05:29:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.477+0000 7f822ec00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f820c0a3bb0 0x7f820c0a6070 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.477+0000 7f822ec00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228103b50 0x7f8228101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.477+0000 7f822ec00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8228103190 0x7f8228101140 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.477+0000 7f822ec00640 1 --2- 172.21.15.57:0/149937583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82281001c0 0x7f8228102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.477+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 >> 172.21.15.57:0/149937583 conn(0x7f82280fbaf0 msgr2=0x7f8228106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.478+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 shutdown_connections 2024-04-13T05:29:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.478+0000 7f822ec00640 1 -- 172.21.15.57:0/149937583 wait complete. 2024-04-13T05:29:10.500 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-13T05:29:10.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3112580271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:29:10.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/2004226268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:29:10.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:10 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3112580271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:29:10.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:10 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/2004226268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:29:10.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a[32327]: 2024-04-13T05:29:10.469+0000 7f4238600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T05:29:10.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3112580271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-13T05:29:10.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/2004226268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-13T05:29:10.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 -- 172.21.15.57:0/2884934487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 msgr2=0x7f768c110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 --2- 172.21.15.57:0/2884934487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c110940 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7684009900 tx=0x7f768402bc30 comp rx=0 tx=0).stop 2024-04-13T05:29:10.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 -- 172.21.15.57:0/2884934487 shutdown_connections 2024-04-13T05:29:10.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 --2- 172.21.15.57:0/2884934487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c110940 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 --2- 172.21.15.57:0/2884934487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f768c1096f0 0x7f768c109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 --2- 172.21.15.57:0/2884934487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f768c108500 0x7f768c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 -- 172.21.15.57:0/2884934487 >> 172.21.15.57:0/2884934487 conn(0x7f768c077490 msgr2=0x7f768c0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:10.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 -- 172.21.15.57:0/2884934487 shutdown_connections 2024-04-13T05:29:10.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.725+0000 7f7691800640 1 -- 172.21.15.57:0/2884934487 wait complete. 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.726+0000 7f7691800640 1 Processor -- start 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 -- start start 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f768c108500 0x7f768c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f768c1096f0 0x7f768c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f768c114090 con 0x7f768c10a0b0 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f768c113f10 con 0x7f768c108500 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7691800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f768c114210 con 0x7f768c1096f0 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7690e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7690e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c1a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54826/0 (socket says 172.21.15.57:54826) 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.727+0000 7f7690e00640 1 -- 172.21.15.57:0/4257866056 learned_addr learned my addr 172.21.15.57:0/4257866056 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f768be00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f768c108500 0x7f768c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f768b400640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f768c1096f0 0x7f768c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f7690e00640 1 -- 172.21.15.57:0/4257866056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f768c1096f0 msgr2=0x7f768c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f7690e00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f768c1096f0 0x7f768c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f7690e00640 1 -- 172.21.15.57:0/4257866056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f768c108500 msgr2=0x7f768c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:10.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f7690e00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f768c108500 0x7f768c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:10.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f7690e00640 1 -- 172.21.15.57:0/4257866056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c1a62a0 con 0x7f768c10a0b0 2024-04-13T05:29:10.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.728+0000 7f7690e00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c1a5c30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7684036040 tx=0x7f768402cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:10.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.729+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684043020 con 0x7f768c10a0b0 2024-04-13T05:29:10.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.729+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76840044c0 con 0x7f768c10a0b0 2024-04-13T05:29:10.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.729+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c1a6530 con 0x7f768c10a0b0 2024-04-13T05:29:10.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.729+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684003bf0 con 0x7f768c10a0b0 2024-04-13T05:29:10.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.729+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c1a6a80 con 0x7f768c10a0b0 2024-04-13T05:29:10.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.730+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c10f7c0 con 0x7f768c10a0b0 2024-04-13T05:29:10.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.732+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7684046020 con 0x7f768c10a0b0 2024-04-13T05:29:10.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.734+0000 7f7688c00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76700a3bb0 0x7f76700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:10.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.734+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9416+0+0 (secure 0 0 0) 0x7f76840f2c20 con 0x7f768c10a0b0 2024-04-13T05:29:10.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.734+0000 7f768be00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76700a3bb0 0x7f76700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:10.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.735+0000 7f768be00640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76700a3bb0 0x7f76700a6070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f768c109550 tx=0x7f767c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:10.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.738+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f768404d020 con 0x7f768c10a0b0 2024-04-13T05:29:10.833 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c[65915]: ignoring --setuser ceph since I am not root 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c[65915]: ignoring --setgroup ceph since I am not root 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-mds[65919]: -- 172.21.15.189:0/1933433872 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad8618da40 con 0x55ad86161000 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c[65915]: starting mds.c at 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i[66223]: ignoring --setuser ceph since I am not root 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i[66223]: ignoring --setgroup ceph since I am not root 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-mds[66227]: -- 172.21.15.189:0/1645589086 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6bd749a40 con 0x55c6bd71c800 2024-04-13T05:29:10.834 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:29:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i[66223]: starting mds.i at 2024-04-13T05:29:10.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:10.997+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f768c062bc0 con 0x7f768c10a0b0 2024-04-13T05:29:11.002 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e[65222]: ignoring --setuser ceph since I am not root 2024-04-13T05:29:11.002 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e[65222]: ignoring --setgroup ceph since I am not root 2024-04-13T05:29:11.002 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-mds[65226]: -- 172.21.15.83:0/4259107227 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ee1d7fa40 con 0x561ee1d52800 2024-04-13T05:29:11.002 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e[65222]: starting mds.e at 2024-04-13T05:29:11.002 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: ignoring --setuser ceph since I am not root 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: ignoring --setgroup ceph since I am not root 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-mds[65843]: -- 172.21.15.83:0/2928566669 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5615f7a4ba40 con 0x5615f7a1f000 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: starting mds.b at 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: ignoring --setuser ceph since I am not root 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: ignoring --setgroup ceph since I am not root 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-mds[65527]: -- 172.21.15.83:0/1214994554 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d22213a40 con 0x560d221e6c00 2024-04-13T05:29:11.003 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: starting mds.h at 2024-04-13T05:29:11.004 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k[64914]: ignoring --setuser ceph since I am not root 2024-04-13T05:29:11.004 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k[64914]: ignoring --setgroup ceph since I am not root 2024-04-13T05:29:11.004 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-mds[64920]: -- 172.21.15.83:0/2252498942 <== mon.2 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fedd443a40 con 0x55fedd417000 2024-04-13T05:29:11.004 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:29:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k[64914]: starting mds.k at 2024-04-13T05:29:11.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.477+0000 7f7688c00640 1 -- 172.21.15.57:0/4257866056 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f76840bf100 con 0x7f768c10a0b0 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76700a3bb0 msgr2=0x7f76700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76700a3bb0 0x7f76700a6070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f768c109550 tx=0x7f767c009500 comp rx=0 tx=0).stop 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 msgr2=0x7f768c1a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c1a5c30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7684036040 tx=0x7f768402cdd0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 shutdown_connections 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f76700a3bb0 0x7f76700a6070 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f768c10a0b0 0x7f768c1a5c30 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f768c1096f0 0x7f768c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 --2- 172.21.15.57:0/4257866056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f768c108500 0x7f768c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.481+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 >> 172.21.15.57:0/4257866056 conn(0x7f768c077490 msgr2=0x7f768c10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:11.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.482+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 shutdown_connections 2024-04-13T05:29:11.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.482+0000 7f7691800640 1 -- 172.21.15.57:0/4257866056 wait complete. 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: pgmap v803: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/149937583' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/149937583' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: osdmap e95: 12 total, 12 up, 12 in 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-13T05:29:11.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:11 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4257866056' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:29:11.508 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-13T05:29:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: pgmap v803: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-13T05:29:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/149937583' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-13T05:29:11.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-13T05:29:11.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T05:29:11.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/149937583' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-13T05:29:11.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: osdmap e95: 12 total, 12 up, 12 in 2024-04-13T05:29:11.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-13T05:29:11.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:11 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4257866056' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: pgmap v803: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/149937583' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/149937583' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: osdmap e95: 12 total, 12 up, 12 in 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-13T05:29:11.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:11 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4257866056' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 -- 172.21.15.57:0/544441280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c108620 msgr2=0x7f0b2c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 --2- 172.21.15.57:0/544441280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c108620 0x7f0b2c108a40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009ca0 tx=0x7f0b1c02c100 comp rx=0 tx=0).stop 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 -- 172.21.15.57:0/544441280 shutdown_connections 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 --2- 172.21.15.57:0/544441280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c109810 0x7f0b2c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 --2- 172.21.15.57:0/544441280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 --2- 172.21.15.57:0/544441280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c108620 0x7f0b2c108a40 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 -- 172.21.15.57:0/544441280 >> 172.21.15.57:0/544441280 conn(0x7f0b2c0775c0 msgr2=0x7f0b2c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 -- 172.21.15.57:0/544441280 shutdown_connections 2024-04-13T05:29:11.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.780+0000 7f0b32600640 1 -- 172.21.15.57:0/544441280 wait complete. 2024-04-13T05:29:11.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.781+0000 7f0b32600640 1 Processor -- start 2024-04-13T05:29:11.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 -- start start 2024-04-13T05:29:11.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c108620 0x7f0b2c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:11.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b2c109810 0x7f0b2c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c114150 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c113fd0 con 0x7f0b2c108620 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.782+0000 7f0b32600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1142d0 con 0x7f0b2c109810 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c1a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54848/0 (socket says 172.21.15.57:54848) 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 -- 172.21.15.57:0/600147932 learned_addr learned my addr 172.21.15.57:0/600147932 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2b400640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c108620 0x7f0b2c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2aa00640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b2c109810 0x7f0b2c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 -- 172.21.15.57:0/600147932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b2c109810 msgr2=0x7f0b2c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b2c109810 0x7f0b2c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 -- 172.21.15.57:0/600147932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c108620 msgr2=0x7f0b2c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c108620 0x7f0b2c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:11.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.783+0000 7f0b2be00640 1 -- 172.21.15.57:0/600147932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c1a64b0 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.784+0000 7f0b2be00640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c1a5e40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400d750 tx=0x7f0b2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:11.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.784+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24017070 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.784+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c1a67a0 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.784+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b240054d0 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.784+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24004c60 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.784+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1a6cf0 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.786+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2c10fa60 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.788+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0b24004e00 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.789+0000 7f0b23e00640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0b100a3bb0 0x7f0b100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:11.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.789+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9416+0+0 (secure 0 0 0) 0x7f0b240d01d0 con 0x7f0b2c10a1d0 2024-04-13T05:29:11.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.789+0000 7f0b2b400640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0b100a3bb0 0x7f0b100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:11.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.790+0000 7f0b2b400640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0b100a3bb0 0x7f0b100a6070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c02c710 tx=0x7f0b1c032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:11.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:11.794+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b2409c060 con 0x7f0b2c10a1d0 2024-04-13T05:29:12.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.001+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b2c062bc0 con 0x7f0b2c10a1d0 2024-04-13T05:29:12.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.482+0000 7f0b23e00640 1 -- 172.21.15.57:0/600147932 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f0b240a0360 con 0x7f0b2c10a1d0 2024-04-13T05:29:12.485 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2024-04-13T05:29:12.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.485+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0b100a3bb0 msgr2=0x7f0b100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:12.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.485+0000 7f0b32600640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0b100a3bb0 0x7f0b100a6070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c02c710 tx=0x7f0b1c032340 comp rx=0 tx=0).stop 2024-04-13T05:29:12.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.486+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 msgr2=0x7f0b2c1a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:12.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.486+0000 7f0b32600640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c1a5e40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400d750 tx=0x7f0b2400dc10 comp rx=0 tx=0).stop 2024-04-13T05:29:12.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.488+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 shutdown_connections 2024-04-13T05:29:12.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.488+0000 7f0b32600640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7f0b100a3bb0 0x7f0b100a6070 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.488+0000 7f0b32600640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0b2c10a1d0 0x7f0b2c1a5e40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.488+0000 7f0b32600640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b2c109810 0x7f0b2c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.488+0000 7f0b32600640 1 --2- 172.21.15.57:0/600147932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c108620 0x7f0b2c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.488+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 >> 172.21.15.57:0/600147932 conn(0x7f0b2c0775c0 msgr2=0x7f0b2c10cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:12.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.489+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 shutdown_connections 2024-04-13T05:29:12.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.489+0000 7f0b32600640 1 -- 172.21.15.57:0/600147932 wait complete. 2024-04-13T05:29:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-13T05:29:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-13T05:29:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: Cluster is now healthy 2024-04-13T05:29:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/4257866056' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-13T05:29:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6832/2018368733,v1:172.21.15.83:6833/2018368733] up:boot 2024-04-13T05:29:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.189:6836/2613221577,v1:172.21.15.189:6838/2613221577] up:boot 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.189:6837/2038360847,v1:172.21.15.189:6839/2038360847] up:boot 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6834/873103637,v1:172.21.15.83:6835/873103637] up:boot 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6836/3120864232,v1:172.21.15.83:6838/3120864232] up:boot 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: mds.? [v2:172.21.15.83:6837/499427958,v1:172.21.15.83:6839/499427958] up:boot 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: fsmap 12 up:standby 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-13T05:29:12.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:12 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/600147932' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:12.520 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-13T05:29:12.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-13T05:29:12.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-13T05:29:12.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: Cluster is now healthy 2024-04-13T05:29:12.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/4257866056' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-13T05:29:12.584 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6832/2018368733,v1:172.21.15.83:6833/2018368733] up:boot 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.189:6836/2613221577,v1:172.21.15.189:6838/2613221577] up:boot 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.189:6837/2038360847,v1:172.21.15.189:6839/2038360847] up:boot 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6834/873103637,v1:172.21.15.83:6835/873103637] up:boot 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6836/3120864232,v1:172.21.15.83:6838/3120864232] up:boot 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: mds.? [v2:172.21.15.83:6837/499427958,v1:172.21.15.83:6839/499427958] up:boot 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: fsmap 12 up:standby 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-13T05:29:12.585 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:12 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/600147932' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:12.753 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-13T05:29:12.753 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-13T05:29:12.753 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: Cluster is now healthy 2024-04-13T05:29:12.753 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/4257866056' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-13T05:29:12.753 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6832/2018368733,v1:172.21.15.83:6833/2018368733] up:boot 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.189:6836/2613221577,v1:172.21.15.189:6838/2613221577] up:boot 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.189:6837/2038360847,v1:172.21.15.189:6839/2038360847] up:boot 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6834/873103637,v1:172.21.15.83:6835/873103637] up:boot 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6836/3120864232,v1:172.21.15.83:6838/3120864232] up:boot 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: mds.? [v2:172.21.15.83:6837/499427958,v1:172.21.15.83:6839/499427958] up:boot 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: fsmap 12 up:standby 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='mgr.14152 172.21.15.57:0/620235197' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-13T05:29:12.754 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:12 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/600147932' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:12.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.754+0000 7fa536000640 1 -- 172.21.15.57:0/3222721515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 msgr2=0x7fa53010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:12.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.754+0000 7fa536000640 1 --2- 172.21.15.57:0/3222721515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 0x7fa53010e020 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa528009ca0 tx=0x7fa52802c100 comp rx=0 tx=0).stop 2024-04-13T05:29:12.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 -- 172.21.15.57:0/3222721515 shutdown_connections 2024-04-13T05:29:12.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 --2- 172.21.15.57:0/3222721515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa530073100 0x7fa530110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 --2- 172.21.15.57:0/3222721515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 0x7fa53010e020 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 --2- 172.21.15.57:0/3222721515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530109230 0x7fa53010b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 -- 172.21.15.57:0/3222721515 >> 172.21.15.57:0/3222721515 conn(0x7fa53006ee50 msgr2=0x7fa530071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:12.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 -- 172.21.15.57:0/3222721515 shutdown_connections 2024-04-13T05:29:12.758 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.755+0000 7fa536000640 1 -- 172.21.15.57:0/3222721515 wait complete. 2024-04-13T05:29:12.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.756+0000 7fa536000640 1 Processor -- start 2024-04-13T05:29:12.759 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.757+0000 7fa536000640 1 -- start start 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa536000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 0x7fa5301a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa536000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa530109230 0x7fa5301a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa536000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 0x7fa5301ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa536000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa530113e10 con 0x7fa53010bbf0 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa536000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa530113c90 con 0x7fa530073100 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa536000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa530113f90 con 0x7fa530109230 2024-04-13T05:29:12.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.758+0000 7fa52f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 0x7fa5301a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:12.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.759+0000 7fa52f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 0x7fa5301a70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.57:52424/0 (socket says 172.21.15.57:52424) 2024-04-13T05:29:12.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.759+0000 7fa52f400640 1 -- 172.21.15.57:0/3229390440 learned_addr learned my addr 172.21.15.57:0/3229390440 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:12.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.759+0000 7fa52fe00640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 0x7fa5301ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:12.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.760+0000 7fa52ea00640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa530109230 0x7fa5301a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:12.762 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.760+0000 7fa52f400640 1 -- 172.21.15.57:0/3229390440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa530109230 msgr2=0x7fa5301a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:12.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.760+0000 7fa52f400640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa530109230 0x7fa5301a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.760+0000 7fa52f400640 1 -- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 msgr2=0x7fa5301ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:12.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.760+0000 7fa52f400640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 0x7fa5301ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:12.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.760+0000 7fa52f400640 1 -- 172.21.15.57:0/3229390440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301aecd0 con 0x7fa530073100 2024-04-13T05:29:12.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.761+0000 7fa52f400640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 0x7fa5301a70b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00d950 tx=0x7fa51c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:12.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.761+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c017070 con 0x7fa530073100 2024-04-13T05:29:12.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.761+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa51c009c30 con 0x7fa530073100 2024-04-13T05:29:12.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.761+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301aefc0 con 0x7fa530073100 2024-04-13T05:29:12.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.761+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00f620 con 0x7fa530073100 2024-04-13T05:29:12.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.761+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301af420 con 0x7fa530073100 2024-04-13T05:29:12.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.763+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530071950 con 0x7fa530073100 2024-04-13T05:29:12.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.770+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa51c0109a0 con 0x7fa530073100 2024-04-13T05:29:12.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.772+0000 7fa527e00640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5140a3bb0 0x7fa5140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:12.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.772+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8862+0+0 (secure 0 0 0) 0x7fa51c0d0340 con 0x7fa530073100 2024-04-13T05:29:12.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.773+0000 7fa52ea00640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5140a3bb0 0x7fa5140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:12.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.774+0000 7fa52ea00640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5140a3bb0 0x7fa5140a6070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa52802c710 tx=0x7fa528000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:12.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.778+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa51c009de0 con 0x7fa530073100 2024-04-13T05:29:13.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:12.999+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa53002d5e0 con 0x7fa530073100 2024-04-13T05:29:13.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.507+0000 7fa527e00640 1 -- 172.21.15.57:0/3229390440 <== mon.2 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7fa51c09cab0 con 0x7fa530073100 2024-04-13T05:29:13.513 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2024-04-13T05:29:13.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.513+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5140a3bb0 msgr2=0x7fa5140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:13.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.513+0000 7fa536000640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5140a3bb0 0x7fa5140a6070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa52802c710 tx=0x7fa528000f40 comp rx=0 tx=0).stop 2024-04-13T05:29:13.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.513+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 msgr2=0x7fa5301a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:13.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.513+0000 7fa536000640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 0x7fa5301a70b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00d950 tx=0x7fa51c00de10 comp rx=0 tx=0).stop 2024-04-13T05:29:13.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 shutdown_connections 2024-04-13T05:29:13.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7fa5140a3bb0 0x7fa5140a6070 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa53010bbf0 0x7fa5301ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa530109230 0x7fa5301a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 --2- 172.21.15.57:0/3229390440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa530073100 0x7fa5301a70b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 >> 172.21.15.57:0/3229390440 conn(0x7fa53006ee50 msgr2=0x7fa53010bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:13.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.514+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 shutdown_connections 2024-04-13T05:29:13.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.515+0000 7fa536000640 1 -- 172.21.15.57:0/3229390440 wait complete. 2024-04-13T05:29:13.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:13 smithi057 ceph-mon[32331]: pgmap v805: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-13T05:29:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:13 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/600147932' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:13 smithi057 ceph-mon[32331]: osdmap e96: 12 total, 12 up, 12 in 2024-04-13T05:29:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:13 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3229390440' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:13.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:13 smithi057 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:13.542 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-13T05:29:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:13 smithi189 ceph-mon[35531]: pgmap v805: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-13T05:29:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:13 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/600147932' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:13 smithi189 ceph-mon[35531]: osdmap e96: 12 total, 12 up, 12 in 2024-04-13T05:29:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:13 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3229390440' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:13.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:13 smithi189 ceph-mon[35531]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:13 smithi083 ceph-mon[35133]: pgmap v805: 129 pgs: 129 active+clean; 296 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-13T05:29:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:13 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/600147932' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:13 smithi083 ceph-mon[35133]: osdmap e96: 12 total, 12 up, 12 in 2024-04-13T05:29:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:13 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3229390440' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:13.752 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:13 smithi083 ceph-mon[35133]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 -- 172.21.15.57:0/4262258196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff658104100 msgr2=0x7ff658104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 --2- 172.21.15.57:0/4262258196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff658104100 0x7ff658104520 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff648009ca0 tx=0x7ff64802c100 comp rx=0 tx=0).stop 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 -- 172.21.15.57:0/4262258196 shutdown_connections 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 --2- 172.21.15.57:0/4262258196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff658105cb0 0x7ff65810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 --2- 172.21.15.57:0/4262258196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6581052f0 0x7ff658105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 --2- 172.21.15.57:0/4262258196 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff658104100 0x7ff658104520 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 -- 172.21.15.57:0/4262258196 >> 172.21.15.57:0/4262258196 conn(0x7ff6580ff670 msgr2=0x7ff658101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:13.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 -- 172.21.15.57:0/4262258196 shutdown_connections 2024-04-13T05:29:13.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.763+0000 7ff65ca00640 1 -- 172.21.15.57:0/4262258196 wait complete. 2024-04-13T05:29:13.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.764+0000 7ff65ca00640 1 Processor -- start 2024-04-13T05:29:13.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.764+0000 7ff65ca00640 1 -- start start 2024-04-13T05:29:13.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff65ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff658104100 0x7ff65819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:13.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff65ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 0x7ff65819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff65ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff658105cb0 0x7ff6581a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff65ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810fc80 con 0x7ff6581052f0 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff65ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810fb00 con 0x7ff658105cb0 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff65ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810fe00 con 0x7ff658104100 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff656a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 0x7ff65819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff656a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 0x7ff65819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54882/0 (socket says 172.21.15.57:54882) 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff656a00640 1 -- 172.21.15.57:0/3005912795 learned_addr learned my addr 172.21.15.57:0/3005912795 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff657e00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff658105cb0 0x7ff6581a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.765+0000 7ff657400640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff658104100 0x7ff65819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff656a00640 1 -- 172.21.15.57:0/3005912795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff658104100 msgr2=0x7ff65819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff656a00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff658104100 0x7ff65819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff656a00640 1 -- 172.21.15.57:0/3005912795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff658105cb0 msgr2=0x7ff6581a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff656a00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff658105cb0 0x7ff6581a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff656a00640 1 -- 172.21.15.57:0/3005912795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6581a1fe0 con 0x7ff6581052f0 2024-04-13T05:29:13.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff656a00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 0x7ff65819a950 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff64000b4a0 tx=0x7ff64000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640014070 con 0x7ff6581052f0 2024-04-13T05:29:13.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6581a22d0 con 0x7ff6581052f0 2024-04-13T05:29:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff640010ac0 con 0x7ff6581052f0 2024-04-13T05:29:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640017c30 con 0x7ff6581052f0 2024-04-13T05:29:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.766+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6581a28f0 con 0x7ff6581052f0 2024-04-13T05:29:13.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.768+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65810b540 con 0x7ff6581052f0 2024-04-13T05:29:13.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.771+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff640017070 con 0x7ff6581052f0 2024-04-13T05:29:13.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.773+0000 7ff64fe00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff63c0a3bb0 0x7ff63c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-13T05:29:13.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.773+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8324+0+0 (secure 0 0 0) 0x7ff6400cfbc0 con 0x7ff6581052f0 2024-04-13T05:29:13.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.773+0000 7ff657400640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff63c0a3bb0 0x7ff63c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-13T05:29:13.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.774+0000 7ff657400640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff63c0a3bb0 0x7ff63c0a6070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff64802c710 tx=0x7ff648000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-13T05:29:13.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:13.777+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff64009c540 con 0x7ff6581052f0 2024-04-13T05:29:14.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.019+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff658062bc0 con 0x7ff6581052f0 2024-04-13T05:29:14.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.510+0000 7ff64fe00640 1 -- 172.21.15.57:0/3005912795 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7ff64009be40 con 0x7ff6581052f0 2024-04-13T05:29:14.513 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data_ec' removed 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff63c0a3bb0 msgr2=0x7ff63c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff63c0a3bb0 0x7ff63c0a6070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff64802c710 tx=0x7ff648000f40 comp rx=0 tx=0).stop 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 msgr2=0x7ff65819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 0x7ff65819a950 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff64000b4a0 tx=0x7ff64000b960 comp rx=0 tx=0).stop 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 shutdown_connections 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:6800/3239836246,v1:172.21.15.57:6801/3239836246] conn(0x7ff63c0a3bb0 0x7ff63c0a6070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff658105cb0 0x7ff6581a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:14.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6581052f0 0x7ff65819a950 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:14.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 --2- 172.21.15.57:0/3005912795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff658104100 0x7ff65819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-13T05:29:14.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 >> 172.21.15.57:0/3005912795 conn(0x7ff6580ff670 msgr2=0x7ff658108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-13T05:29:14.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 shutdown_connections 2024-04-13T05:29:14.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-13T05:29:14.514+0000 7ff65ca00640 1 -- 172.21.15.57:0/3005912795 wait complete. 2024-04-13T05:29:14.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:29:14.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:29:14.755 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:29:14.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:14 smithi057 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:14.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:14 smithi057 ceph-mon[32331]: osdmap e97: 12 total, 12 up, 12 in 2024-04-13T05:29:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:14 smithi057 ceph-mon[32331]: pgmap v808: 33 pgs: 33 active+clean; 577 KiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:29:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:14 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3005912795' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:14.818 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:14 smithi189 ceph-mon[35531]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:14.819 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:14 smithi189 ceph-mon[35531]: osdmap e97: 12 total, 12 up, 12 in 2024-04-13T05:29:14.819 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:14 smithi189 ceph-mon[35531]: pgmap v808: 33 pgs: 33 active+clean; 577 KiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:29:14.819 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:14 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3005912795' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:14.830 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-13T05:29:14.830 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-13T05:29:14.882 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-13T05:29:14.935 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-13T05:29:14.991 INFO:tasks.cephadm:Stopping all daemons... 2024-04-13T05:29:14.992 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-13T05:29:14.992 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a 2024-04-13T05:29:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:14 smithi083 ceph-mon[35133]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:14 smithi083 ceph-mon[35133]: osdmap e97: 12 total, 12 up, 12 in 2024-04-13T05:29:15.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:14 smithi083 ceph-mon[35133]: pgmap v808: 33 pgs: 33 active+clean; 577 KiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-13T05:29:15.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:14 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3005912795' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-13T05:29:15.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:15 smithi057 systemd[1]: Stopping Ceph mon.a for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:15.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:15 smithi057 ceph-mon[32331]: from='client.? 172.21.15.57:0/3005912795' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:15.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:15 smithi057 ceph-mon[32331]: osdmap e98: 12 total, 12 up, 12 in 2024-04-13T05:29:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a[32327]: 2024-04-13T05:29:15.618+0000 7f423f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a[32327]: 2024-04-13T05:29:15.618+0000 7f423f400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-13T05:29:15.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:15 smithi189 ceph-mon[35531]: from='client.? 172.21.15.57:0/3005912795' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:15.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:15 smithi189 ceph-mon[35531]: osdmap e98: 12 total, 12 up, 12 in 2024-04-13T05:29:16.001 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:15 smithi083 ceph-mon[35133]: from='client.? 172.21.15.57:0/3005912795' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-04-13T05:29:16.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:15 smithi083 ceph-mon[35133]: osdmap e98: 12 total, 12 up, 12 in 2024-04-13T05:29:16.157 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:15 smithi057 podman[169669]: 2024-04-13 05:29:15.815712515 +0000 UTC m=+0.344127949 container died 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-13T05:29:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:16 smithi057 podman[169669]: 2024-04-13 05:29:16.303342144 +0000 UTC m=+0.831757571 container cleanup 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-13T05:29:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:16 smithi057 bash[169669]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a 2024-04-13T05:29:17.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Apr 13 05:29:16 smithi057 podman[169683]: 2024-04-13 05:29:16.554862691 +0000 UTC m=+0.723286846 container remove 2e6ff8f4edab7f5d42978d49b47cc579cbb7e4bcbfcb7a071d9de18b8a4f0888 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-a, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-13T05:29:17.125 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.a.service' 2024-04-13T05:29:17.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:17.274 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-13T05:29:17.274 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-13T05:29:17.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.b 2024-04-13T05:29:17.751 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:17 smithi083 systemd[1]: Stopping Ceph mon.b for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:18.206 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b[35129]: 2024-04-13T05:29:17.850+0000 7fa538000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:18.206 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b[35129]: 2024-04-13T05:29:17.850+0000 7fa538000700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-13T05:29:18.206 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:17 smithi083 podman[70774]: 2024-04-13 05:29:17.882809475 +0000 UTC m=+0.169838806 container died 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-13T05:29:18.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:18 smithi083 podman[70774]: 2024-04-13 05:29:18.337091831 +0000 UTC m=+0.624121199 container cleanup 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:29:18.502 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:18 smithi083 bash[70774]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b 2024-04-13T05:29:19.002 INFO:journalctl@ceph.mon.b.smithi083.stdout:Apr 13 05:29:18 smithi083 podman[70788]: 2024-04-13 05:29:18.625017484 +0000 UTC m=+0.730550438 container remove 6ac9349280a8ca8e34782759334cadc7cfbbaa5f1fb1ba0e2a4a1903c50fa884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:29:19.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.b.service' 2024-04-13T05:29:19.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:19.252 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-13T05:29:19.252 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-13T05:29:19.252 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.c 2024-04-13T05:29:19.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:19 smithi189 systemd[1]: Stopping Ceph mon.c for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:20.322 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c[35505]: 2024-04-13T05:29:19.912+0000 7f7dd4c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:20.322 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c[35505]: 2024-04-13T05:29:19.912+0000 7f7dd4c00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-13T05:29:20.322 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:20 smithi189 podman[71492]: 2024-04-13 05:29:20.025914241 +0000 UTC m=+0.319145988 container died c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:29:20.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:20 smithi189 podman[71492]: 2024-04-13 05:29:20.480471298 +0000 UTC m=+0.773703078 container cleanup c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-13T05:29:20.583 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:20 smithi189 bash[71492]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c 2024-04-13T05:29:21.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 13 05:29:20 smithi189 podman[71506]: 2024-04-13 05:29:20.698829431 +0000 UTC m=+0.656790511 container remove c9c076e507de31c2f729bb11d7b7f6bfdc02c4d25ccd46bf9d38e09bd964f2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:29:21.245 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mon.c.service' 2024-04-13T05:29:21.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:21.399 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-13T05:29:21.399 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-13T05:29:21.399 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x 2024-04-13T05:29:21.783 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:21 smithi057 systemd[1]: Stopping Ceph mgr.x for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:22.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:29:22.005+0000 7fb201a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:22.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x[32558]: 2024-04-13T05:29:22.005+0000 7fb201a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-13T05:29:22.284 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:22 smithi057 podman[169790]: 2024-04-13 05:29:22.101615159 +0000 UTC m=+0.261850946 container died 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-13T05:29:22.662 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:22 smithi057 podman[169790]: 2024-04-13 05:29:22.649926333 +0000 UTC m=+0.810162120 container cleanup 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-13T05:29:22.662 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:22 smithi057 bash[169790]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x 2024-04-13T05:29:23.283 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Apr 13 05:29:22 smithi057 podman[169804]: 2024-04-13 05:29:22.885186047 +0000 UTC m=+0.768727785 container remove 86589998f3f4730590b9b3552bc5ab21390f48eda234fd95f49347b9606bcd7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:218a691417ef57c5d22a8d5cf3c5f41f7b542838, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:29:23.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.x.service' 2024-04-13T05:29:23.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:23.529 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-13T05:29:23.529 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-13T05:29:23.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.y 2024-04-13T05:29:24.001 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:29:23 smithi083 systemd[1]: Stopping Ceph mgr.y for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:24.503 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:29:24 smithi083 podman[70906]: 2024-04-13 05:29:24.147419427 +0000 UTC m=+0.216344188 container died 7e61f2ee308f84c8be8477f6348ef4128211db6f4e1e191965b21436c8a7d25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True) 2024-04-13T05:29:24.972 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:29:24 smithi083 podman[70906]: 2024-04-13 05:29:24.718359997 +0000 UTC m=+0.787284763 container cleanup 7e61f2ee308f84c8be8477f6348ef4128211db6f4e1e191965b21436c8a7d25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-13T05:29:24.972 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:29:24 smithi083 bash[70906]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y 2024-04-13T05:29:25.251 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:29:24 smithi083 podman[70920]: 2024-04-13 05:29:24.969905393 +0000 UTC m=+0.810551409 container remove 7e61f2ee308f84c8be8477f6348ef4128211db6f4e1e191965b21436c8a7d25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:29:25.251 INFO:journalctl@ceph.mgr.y.smithi083.stdout:Apr 13 05:29:24 smithi083 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-13T05:29:25.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.y.service' 2024-04-13T05:29:25.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:25.533 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-13T05:29:25.534 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-13T05:29:25.534 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.z 2024-04-13T05:29:26.083 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:29:25 smithi189 systemd[1]: Stopping Ceph mgr.z for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:26.545 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:29:26 smithi189 podman[71623]: 2024-04-13 05:29:26.217225545 +0000 UTC m=+0.237574394 container died 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:29:26.833 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:29:26 smithi189 podman[71623]: 2024-04-13 05:29:26.710442913 +0000 UTC m=+0.730791732 container cleanup 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , ceph=True) 2024-04-13T05:29:26.833 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:29:26 smithi189 bash[71623]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z 2024-04-13T05:29:27.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:29:26 smithi189 podman[71637]: 2024-04-13 05:29:26.953775334 +0000 UTC m=+0.722983550 container remove 093273afa422946889d73aa5b445683b54ad39af21112dee84f034a4b91554a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-13T05:29:27.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 13 05:29:26 smithi189 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-13T05:29:27.477 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mgr.z.service' 2024-04-13T05:29:27.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:27.600 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-13T05:29:27.600 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-13T05:29:27.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.0 2024-04-13T05:29:28.032 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:27 smithi057 systemd[1]: Stopping Ceph osd.0 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:28.533 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0[47287]: 2024-04-13T05:29:28.254+0000 7efd28800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:28.533 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0[47287]: 2024-04-13T05:29:28.254+0000 7efd28800700 -1 osd.0 98 *** Got signal Terminated *** 2024-04-13T05:29:28.533 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0[47287]: 2024-04-13T05:29:28.254+0000 7efd28800700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:29:34.137 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:33 smithi057 podman[169921]: 2024-04-13 05:29:33.78455527 +0000 UTC m=+5.687562339 container died 6823cb58d3a25bc9cb7918c25664978ea66165071a4743c11b47fe6480aa0022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240409) 2024-04-13T05:29:34.475 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:34 smithi057 podman[169921]: 2024-04-13 05:29:34.263348565 +0000 UTC m=+6.166355640 container cleanup 6823cb58d3a25bc9cb7918c25664978ea66165071a4743c11b47fe6480aa0022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux ) 2024-04-13T05:29:34.475 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:34 smithi057 bash[169921]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0 2024-04-13T05:29:34.783 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:34 smithi057 podman[169942]: 2024-04-13 05:29:34.472802229 +0000 UTC m=+0.668868738 container remove 6823cb58d3a25bc9cb7918c25664978ea66165071a4743c11b47fe6480aa0022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-13T05:29:35.534 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:35 smithi057 podman[170031]: 2024-04-13 05:29:35.177292571 +0000 UTC m=+0.036400923 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:29:35.534 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:35 smithi057 podman[170031]: 2024-04-13 05:29:35.29061146 +0000 UTC m=+0.149719854 container create 51341364f9301c48436bce9bcf1f0437f8da8e60e558345ce73cf4ea512ea9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:29:36.033 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:35 smithi057 podman[170031]: 2024-04-13 05:29:35.619711662 +0000 UTC m=+0.478820023 container init 51341364f9301c48436bce9bcf1f0437f8da8e60e558345ce73cf4ea512ea9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-13T05:29:36.033 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:35 smithi057 podman[170031]: 2024-04-13 05:29:35.632946086 +0000 UTC m=+0.492054483 container start 51341364f9301c48436bce9bcf1f0437f8da8e60e558345ce73cf4ea512ea9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-13T05:29:36.033 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:35 smithi057 podman[170031]: 2024-04-13 05:29:35.735104862 +0000 UTC m=+0.594213285 container attach 51341364f9301c48436bce9bcf1f0437f8da8e60e558345ce73cf4ea512ea9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:29:36.470 INFO:journalctl@ceph.osd.0.smithi057.stdout:Apr 13 05:29:36 smithi057 podman[170031]: 2024-04-13 05:29:36.097113786 +0000 UTC m=+0.956222189 container died 51341364f9301c48436bce9bcf1f0437f8da8e60e558345ce73cf4ea512ea9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:29:36.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.0.service' 2024-04-13T05:29:37.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:37.028 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-13T05:29:37.028 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-13T05:29:37.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.1 2024-04-13T05:29:37.501 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:37 smithi083 systemd[1]: Stopping Ceph osd.1 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:38.002 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1[39681]: 2024-04-13T05:29:37.631+0000 7fa245200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:38.002 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1[39681]: 2024-04-13T05:29:37.631+0000 7fa245200700 -1 osd.1 98 *** Got signal Terminated *** 2024-04-13T05:29:38.002 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1[39681]: 2024-04-13T05:29:37.631+0000 7fa245200700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:29:43.482 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:43 smithi083 podman[71028]: 2024-04-13 05:29:43.188364268 +0000 UTC m=+5.759789070 container died 802ae1273d88fd4361034e853ebc3ce3e671eb78817d92eeb1555fb10aedd912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-13T05:29:43.752 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:43 smithi083 podman[71028]: 2024-04-13 05:29:43.606878322 +0000 UTC m=+6.178303162 container cleanup 802ae1273d88fd4361034e853ebc3ce3e671eb78817d92eeb1555fb10aedd912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-13T05:29:43.752 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:43 smithi083 bash[71028]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1 2024-04-13T05:29:44.251 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:43 smithi083 podman[71047]: 2024-04-13 05:29:43.941791407 +0000 UTC m=+0.742978877 container remove 802ae1273d88fd4361034e853ebc3ce3e671eb78817d92eeb1555fb10aedd912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:29:45.116 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:44 smithi083 podman[71139]: 2024-04-13 05:29:44.709993701 +0000 UTC m=+0.032390678 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:29:45.117 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:44 smithi083 podman[71139]: 2024-04-13 05:29:44.842774558 +0000 UTC m=+0.165171568 container create b114436d5539a7a4a75b5167d9aaa9ddaf3e902f82bb4291b1e65a1f69e25397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-13T05:29:45.504 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:45 smithi083 podman[71139]: 2024-04-13 05:29:45.113441062 +0000 UTC m=+0.435838079 container init b114436d5539a7a4a75b5167d9aaa9ddaf3e902f82bb4291b1e65a1f69e25397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:29:45.504 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:45 smithi083 podman[71139]: 2024-04-13 05:29:45.124676703 +0000 UTC m=+0.447073707 container start b114436d5539a7a4a75b5167d9aaa9ddaf3e902f82bb4291b1e65a1f69e25397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:29:45.504 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:45 smithi083 podman[71139]: 2024-04-13 05:29:45.250275115 +0000 UTC m=+0.572672165 container attach b114436d5539a7a4a75b5167d9aaa9ddaf3e902f82bb4291b1e65a1f69e25397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-13T05:29:45.947 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:45 smithi083 podman[71139]: 2024-04-13 05:29:45.576323588 +0000 UTC m=+0.898720632 container died b114436d5539a7a4a75b5167d9aaa9ddaf3e902f82bb4291b1e65a1f69e25397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-13T05:29:46.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.1.service' 2024-04-13T05:29:46.525 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:46 smithi083 podman[71165]: 2024-04-13 05:29:46.408202404 +0000 UTC m=+0.818356222 container remove b114436d5539a7a4a75b5167d9aaa9ddaf3e902f82bb4291b1e65a1f69e25397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-13T05:29:46.525 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:46 smithi083 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-13T05:29:46.525 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:46 smithi083 systemd[1]: Stopped Ceph osd.1 for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:29:46.525 INFO:journalctl@ceph.osd.1.smithi083.stdout:Apr 13 05:29:46 smithi083 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.1.service: Consumed 1min 17.038s CPU time. 2024-04-13T05:29:46.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:46.550 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-13T05:29:46.550 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-13T05:29:46.550 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.2 2024-04-13T05:29:47.083 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:46 smithi189 systemd[1]: Stopping Ceph osd.2 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:47.583 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2[40679]: 2024-04-13T05:29:47.128+0000 7f93eba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:47.583 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2[40679]: 2024-04-13T05:29:47.128+0000 7f93eba00700 -1 osd.2 98 *** Got signal Terminated *** 2024-04-13T05:29:47.583 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2[40679]: 2024-04-13T05:29:47.128+0000 7f93eba00700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:29:53.083 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:52 smithi189 podman[71748]: 2024-04-13 05:29:52.784881039 +0000 UTC m=+5.806981226 container died 574b77fe01baadf993dfd1123c83bb37db73ad591e112d68b883577cfcc2f4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:29:53.475 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:53 smithi189 podman[71748]: 2024-04-13 05:29:53.246919336 +0000 UTC m=+6.269019525 container cleanup 574b77fe01baadf993dfd1123c83bb37db73ad591e112d68b883577cfcc2f4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:29:53.475 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:53 smithi189 bash[71748]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2 2024-04-13T05:29:53.833 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:53 smithi189 podman[71768]: 2024-04-13 05:29:53.472982258 +0000 UTC m=+0.674379794 container remove 574b77fe01baadf993dfd1123c83bb37db73ad591e112d68b883577cfcc2f4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-13T05:29:54.583 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:54 smithi189 podman[71858]: 2024-04-13 05:29:54.211503912 +0000 UTC m=+0.034084924 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:29:54.584 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:54 smithi189 podman[71858]: 2024-04-13 05:29:54.3326957 +0000 UTC m=+0.155276752 container create ce80f4ea64d9cfd232161a8807a7dfe40b11d041713d434bd5563a02b56d53fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:29:55.083 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:54 smithi189 podman[71858]: 2024-04-13 05:29:54.653112505 +0000 UTC m=+0.475693564 container init ce80f4ea64d9cfd232161a8807a7dfe40b11d041713d434bd5563a02b56d53fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True) 2024-04-13T05:29:55.084 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:54 smithi189 podman[71858]: 2024-04-13 05:29:54.664007929 +0000 UTC m=+0.486589011 container start ce80f4ea64d9cfd232161a8807a7dfe40b11d041713d434bd5563a02b56d53fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:29:55.084 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:54 smithi189 podman[71858]: 2024-04-13 05:29:54.760237241 +0000 UTC m=+0.582818347 container attach ce80f4ea64d9cfd232161a8807a7dfe40b11d041713d434bd5563a02b56d53fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:29:55.478 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 13 05:29:55 smithi189 podman[71858]: 2024-04-13 05:29:55.112670942 +0000 UTC m=+0.935252014 container died ce80f4ea64d9cfd232161a8807a7dfe40b11d041713d434bd5563a02b56d53fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=) 2024-04-13T05:29:55.777 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:29:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:29:55.614+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:29:55.944 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.2.service' 2024-04-13T05:29:56.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:29:56.036 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-13T05:29:56.036 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-13T05:29:56.036 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.3 2024-04-13T05:29:56.130 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:55.839+0000 7f8b97400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:30.699706+0000 front 2024-04-13T05:29:30.699845+0000 (oldest deadline 2024-04-13T05:29:55.400008+0000) 2024-04-13T05:29:56.533 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:56 smithi057 systemd[1]: Stopping Ceph osd.3 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:29:56.796 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:56.662+0000 7f8b9bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:29:56.796 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:56.662+0000 7f8b9bc00700 -1 osd.3 98 *** Got signal Terminated *** 2024-04-13T05:29:56.796 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:56.662+0000 7f8b9bc00700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:29:56.796 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:56.792+0000 7f8b97400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:30.699706+0000 front 2024-04-13T05:29:30.699845+0000 (oldest deadline 2024-04-13T05:29:55.400008+0000) 2024-04-13T05:29:57.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:29:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:29:56.644+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:29:57.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:29:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:29:57.048+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:29:57.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:29:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:29:57.392+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:29:58.033 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:57.754+0000 7f8b97400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:30.699706+0000 front 2024-04-13T05:29:30.699845+0000 (oldest deadline 2024-04-13T05:29:55.400008+0000) 2024-04-13T05:29:58.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:29:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:29:57.641+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:29:58.532 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:29:58 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:29:58.082+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:29:58.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:29:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:29:58.411+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:29:59.033 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:58 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:58.750+0000 7f8b97400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:30.699706+0000 front 2024-04-13T05:29:30.699845+0000 (oldest deadline 2024-04-13T05:29:55.400008+0000) 2024-04-13T05:29:59.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:29:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:29:58.671+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:29:59.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:29:59 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:29:59.050+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:29:59.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:29:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:29:59.437+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:00.033 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:29:59 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:29:59.756+0000 7f8b97400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:30.699706+0000 front 2024-04-13T05:29:30.699845+0000 (oldest deadline 2024-04-13T05:29:55.400008+0000) 2024-04-13T05:30:00.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:29:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:29:59.628+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:00.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:00 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:00.035+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:00.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:00.442+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:00.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:00.597+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:01.033 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:00 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3[54274]: 2024-04-13T05:30:00.711+0000 7f8b97400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:30.699706+0000 front 2024-04-13T05:29:30.699845+0000 (oldest deadline 2024-04-13T05:29:55.400008+0000) 2024-04-13T05:30:01.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:01 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:01.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:01.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:01.416+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:01.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:01.559+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:02.533 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:02 smithi057 podman[170138]: 2024-04-13 05:30:02.315153931 +0000 UTC m=+5.817368949 container died 1e7e9c8ffd6d0eee2fb463a37aa1d8b5ebcd327611eb2146c6ce45a537cb045c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:02.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:02 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:02.118+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:02.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:02.420+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:02.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:02.562+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:03.033 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:02 smithi057 podman[170138]: 2024-04-13 05:30:02.750509696 +0000 UTC m=+6.252724708 container cleanup 1e7e9c8ffd6d0eee2fb463a37aa1d8b5ebcd327611eb2146c6ce45a537cb045c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:30:03.033 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:02 smithi057 bash[170138]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3 2024-04-13T05:30:03.533 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:03 smithi057 podman[170168]: 2024-04-13 05:30:03.085638452 +0000 UTC m=+0.748275484 container remove 1e7e9c8ffd6d0eee2fb463a37aa1d8b5ebcd327611eb2146c6ce45a537cb045c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True) 2024-04-13T05:30:03.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:03 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:03.095+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:03.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:03.373+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:03.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:03.550+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:04.229 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:03 smithi057 podman[170255]: 2024-04-13 05:30:03.880090587 +0000 UTC m=+0.034128121 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:30:04.230 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:04 smithi057 podman[170255]: 2024-04-13 05:30:04.003205325 +0000 UTC m=+0.157242894 container create e1d5391cc634cc9f91d1d8040a9763a3a9c7b6b16db32dbc96189a666f61fdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-13T05:30:04.230 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:04 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:04.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:04.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:04.419+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:04.783 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:04 smithi057 podman[170255]: 2024-04-13 05:30:04.343282177 +0000 UTC m=+0.497319752 container init e1d5391cc634cc9f91d1d8040a9763a3a9c7b6b16db32dbc96189a666f61fdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-13T05:30:04.783 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:04 smithi057 podman[170255]: 2024-04-13 05:30:04.355097591 +0000 UTC m=+0.509135164 container start e1d5391cc634cc9f91d1d8040a9763a3a9c7b6b16db32dbc96189a666f61fdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:30:04.783 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:04 smithi057 podman[170255]: 2024-04-13 05:30:04.447787655 +0000 UTC m=+0.601825278 container attach e1d5391cc634cc9f91d1d8040a9763a3a9c7b6b16db32dbc96189a666f61fdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:30:04.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:04.515+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:04.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:04.661+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:05.174 INFO:journalctl@ceph.osd.3.smithi057.stdout:Apr 13 05:30:04 smithi057 podman[170255]: 2024-04-13 05:30:04.825273668 +0000 UTC m=+0.979311234 container died e1d5391cc634cc9f91d1d8040a9763a3a9c7b6b16db32dbc96189a666f61fdc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:30:05.174 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:05 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:05.069+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:05.616 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.3.service' 2024-04-13T05:30:05.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:05.464+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:05.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:30:05.754 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-13T05:30:05.754 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-13T05:30:05.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.4 2024-04-13T05:30:05.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:05.479+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:05.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:05.655+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:06.252 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:05 smithi083 systemd[1]: Stopping Ceph osd.4 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:30:06.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:06 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:06.076+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:06.693 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:06.461+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:06.693 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:06.461+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:06.693 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:06.325+0000 7ff3f4c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:30:06.693 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:06.325+0000 7ff3f4c00700 -1 osd.4 98 *** Got signal Terminated *** 2024-04-13T05:30:06.693 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:06.325+0000 7ff3f4c00700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:30:06.702 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:06.431+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:07.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:06.690+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:07.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:06 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:06.717+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:07.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:06.700+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:07.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:07.062+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:07.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:07.062+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:07.707 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:07 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:07.379+0000 7ff3f0400700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.027675+0000 front 2024-04-13T05:29:41.027742+0000 (oldest deadline 2024-04-13T05:30:06.927932+0000) 2024-04-13T05:30:07.707 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:07 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:07.437+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:07.707 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:07 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:07.437+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:07.806 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:07.423+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:07.806 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:07.670+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:08.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:07 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:07.704+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:08.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:07 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:07.732+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:08.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:07.804+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:08.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:08 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:08.106+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:08.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:08 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:08.106+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:08.751 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:08 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:08.352+0000 7ff3f0400700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.027675+0000 front 2024-04-13T05:29:41.027742+0000 (oldest deadline 2024-04-13T05:30:06.927932+0000) 2024-04-13T05:30:08.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:08 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:08.425+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:08.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:08 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:08.425+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:08.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:08.654+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:08.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:08.421+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:09.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:08 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:08.760+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:09.251 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:08 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:08.754+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:09.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:08.846+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:09.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:09 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:09.109+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:09.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:09 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:09.109+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:09.735 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:09 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:09.336+0000 7ff3f0400700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.027675+0000 front 2024-04-13T05:29:41.027742+0000 (oldest deadline 2024-04-13T05:30:06.927932+0000) 2024-04-13T05:30:09.736 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:09 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:09.461+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:09.736 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:09 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:09.461+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:09.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:09.682+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:09.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:09.462+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:10.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:09 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:09.733+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:10.096 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:09 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:09.792+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:10.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:09.847+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:10.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:10.094+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:10.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:10.094+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:10.715 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:10.412+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:10.751 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4[46633]: 2024-04-13T05:30:10.330+0000 7ff3f0400700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.027675+0000 front 2024-04-13T05:29:41.027742+0000 (oldest deadline 2024-04-13T05:30:06.927932+0000) 2024-04-13T05:30:10.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:10.414+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:10.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:10.414+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:11.032 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:10 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:10.770+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:11.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:10.838+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:11.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:10.713+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:11.251 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:10 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:10.774+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:11.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:11.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:11.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:11.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:11.721 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:11.398+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:11.741 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:11 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:11.430+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:11.741 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:11 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:11.430+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:12.002 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:11 smithi083 podman[71258]: 2024-04-13 05:30:11.823832332 +0000 UTC m=+5.663124078 container died 8e2b7b150434548ff1bb965fb810549f76e736d26e56e33ad86666bd3ecc26d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, ceph=True) 2024-04-13T05:30:12.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:11 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:11.738+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:12.032 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:11 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:11.751+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:12.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:11.836+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:12.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:11.718+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:12.442 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:12 smithi083 podman[71258]: 2024-04-13 05:30:12.253071176 +0000 UTC m=+6.092362952 container cleanup 8e2b7b150434548ff1bb965fb810549f76e736d26e56e33ad86666bd3ecc26d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:30:12.443 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:12 smithi083 bash[71258]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4 2024-04-13T05:30:12.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:12 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:12.101+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:12.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:12 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:12.101+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:12.735 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:12.379+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:12.752 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:12 smithi083 podman[71280]: 2024-04-13 05:30:12.546011869 +0000 UTC m=+0.709379232 container remove 8e2b7b150434548ff1bb965fb810549f76e736d26e56e33ad86666bd3ecc26d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux ) 2024-04-13T05:30:12.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:12 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:12.440+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:12.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:12 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:12.440+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:13.054 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:12 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:12.780+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:13.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:12.858+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:13.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:12.733+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:13.251 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:12 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:12.760+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:13.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:13.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:13.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:13.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:13.567 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:13 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:13.414+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:13.567 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:13 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:13.414+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:13.568 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:13 smithi083 podman[71371]: 2024-04-13 05:30:13.245353581 +0000 UTC m=+0.036423129 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:30:13.568 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:13 smithi083 podman[71371]: 2024-04-13 05:30:13.397095064 +0000 UTC m=+0.188164639 container create 3a4dc2dc8d688ebdc6e77f6b2e0dcbba1d889fb75452c7a3f3c9faa63fef6c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-13T05:30:13.706 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:13.393+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:13.843 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:13 smithi083 podman[71371]: 2024-04-13 05:30:13.701438546 +0000 UTC m=+0.492508127 container init 3a4dc2dc8d688ebdc6e77f6b2e0dcbba1d889fb75452c7a3f3c9faa63fef6c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409) 2024-04-13T05:30:13.844 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:13 smithi083 podman[71371]: 2024-04-13 05:30:13.712268601 +0000 UTC m=+0.503338166 container start 3a4dc2dc8d688ebdc6e77f6b2e0dcbba1d889fb75452c7a3f3c9faa63fef6c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-13T05:30:13.844 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:13 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:13.775+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:14.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:13 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:13.733+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:14.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:13.844+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:14.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:13.702+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:14.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:13.702+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:14.154 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:13 smithi083 podman[71371]: 2024-04-13 05:30:13.841824738 +0000 UTC m=+0.632894360 container attach 3a4dc2dc8d688ebdc6e77f6b2e0dcbba1d889fb75452c7a3f3c9faa63fef6c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-13T05:30:14.448 INFO:journalctl@ceph.osd.4.smithi083.stdout:Apr 13 05:30:14 smithi083 podman[71371]: 2024-04-13 05:30:14.152719078 +0000 UTC m=+0.943788703 container died 3a4dc2dc8d688ebdc6e77f6b2e0dcbba1d889fb75452c7a3f3c9faa63fef6c72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:14.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:14 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:14.054+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:14.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:14 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:14.054+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:14.706 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:14.347+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:14.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:14 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:14.445+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:14.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:14 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:14.445+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:14.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.4.service' 2024-04-13T05:30:14.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:30:14.954 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-13T05:30:14.954 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-13T05:30:14.954 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.5 2024-04-13T05:30:15.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:14 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:14.744+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:15.042 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:14.824+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:15.042 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:14.703+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:15.042 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:14.703+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:15.251 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:14 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:14.777+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:15.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:15 smithi189 systemd[1]: Stopping Ceph osd.5 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:30:15.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:15.093+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:15.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:15.093+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:15.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:15.093+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:15.592 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:15.337+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:15.710 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:15 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:15.428+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:15.710 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:15 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:15.428+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:15.710 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:15 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:15.428+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:16.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:15 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:15.784+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:16.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:15 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:15.766+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:16.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:15.697+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:16.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:15.697+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:16.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:15.589+0000 7f9a9cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:30:16.084 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:15.589+0000 7f9a9cc00700 -1 osd.5 98 *** Got signal Terminated *** 2024-04-13T05:30:16.084 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:15.589+0000 7f9a9cc00700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:30:16.084 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:15.825+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:16.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:16.114+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:16.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:16.114+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:16.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:16.114+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:16.686 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:16.336+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:16.686 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:16.336+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:16.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:16 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:16.417+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:16.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:16 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:16.417+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:16.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:16 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:16.417+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:17.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:16.741+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:17.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:16 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:16.741+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:17.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:16.682+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:17.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:16.682+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:17.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:16.808+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:17.084 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:16.808+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.509650+0000 front 2024-04-13T05:29:51.509723+0000 (oldest deadline 2024-04-13T05:30:16.209818+0000) 2024-04-13T05:30:17.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:16 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:16.792+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:17.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:16 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:16.792+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:17.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:17.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:17.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:17.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:17.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:17.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:17.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:17 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:17.322+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:17.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:17 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:17.322+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:17.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:17.456+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:17.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:17.456+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:17.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:17.456+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:18.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:17.775+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:18.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:17 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:17.775+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:18.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:17 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:17.648+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:18.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:17 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:17.648+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:18.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:17 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:17.856+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:18.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:17 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:17.856+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.509650+0000 front 2024-04-13T05:29:51.509723+0000 (oldest deadline 2024-04-13T05:30:16.209818+0000) 2024-04-13T05:30:18.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:17.757+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:18.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:17 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:17.757+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:18.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:18 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:18.093+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:18.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:18 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:18.093+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:18.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:18 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:18.093+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:18.687 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:18 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:18.335+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:18.688 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:18 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:18.335+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:18.722 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:18 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:18.455+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:18.723 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:18 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:18.455+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:18.723 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:18 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:18.455+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:19.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:18 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:18.720+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:19.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:18 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:18.720+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:19.032 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:18 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:18.754+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:19.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:18 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:18.754+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:19.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:18 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:18.872+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:19.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:18 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:18.872+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.509650+0000 front 2024-04-13T05:29:51.509723+0000 (oldest deadline 2024-04-13T05:30:16.209818+0000) 2024-04-13T05:30:19.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:18 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:18.684+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:19.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:18 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:18.684+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:19.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:19.132+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:19.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:19.132+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:19.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:19.132+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:19.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:19.323+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:19.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:19.323+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:19.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:19.457+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:19.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:19.457+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:19.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:19.457+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:19.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:19.698+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:19.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:19 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:19.698+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:19.902 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:19.642+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:19.902 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:19.642+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:20.156 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:19.785+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:20.156 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:19 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:19.785+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:20.284 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:19.899+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.606498+0000 front 2024-04-13T05:29:41.606322+0000 (oldest deadline 2024-04-13T05:30:07.506442+0000) 2024-04-13T05:30:20.284 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:19 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5[47312]: 2024-04-13T05:30:19.899+0000 7f9a93400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.509650+0000 front 2024-04-13T05:29:51.509723+0000 (oldest deadline 2024-04-13T05:30:16.209818+0000) 2024-04-13T05:30:20.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:20.153+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:20.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:20.153+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:20.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:20.153+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:20.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:20 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:20.281+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:20.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:20 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:20.281+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:20.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:20 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:20.422+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:20.751 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:20 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:20.422+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:20.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:20 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:20.422+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:20.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:20 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:20.649+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:20.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:20 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:20.649+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:21.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:20.765+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:21.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:20 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:20.765+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:21.078 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:20 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:20.641+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:21.078 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:20 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:20.641+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:21.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:21 smithi189 podman[71976]: 2024-04-13 05:30:21.075865174 +0000 UTC m=+5.672740333 container died 04ba64ebf40eaabe079dc9caaca0b77f52e100327004ebdaa441829652a75dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-13T05:30:21.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:21 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:21.272+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:21.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:21 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:21.272+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:21.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:21.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:21.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:21.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:21.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:21.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:21.697 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:21.431+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:21.697 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:21.431+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:21.697 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:21.431+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:21.804 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:21 smithi189 podman[71976]: 2024-04-13 05:30:21.57456392 +0000 UTC m=+6.171439115 container cleanup 04ba64ebf40eaabe079dc9caaca0b77f52e100327004ebdaa441829652a75dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-13T05:30:21.804 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:21 smithi189 bash[71976]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5 2024-04-13T05:30:21.804 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:21 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:21.644+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:21.804 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:21 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:21.644+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:22.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:21.694+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:22.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:21 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:21.694+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:22.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:21.736+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:22.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:21.736+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:22.083 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:21 smithi189 podman[71996]: 2024-04-13 05:30:21.800694443 +0000 UTC m=+0.710631364 container remove 04ba64ebf40eaabe079dc9caaca0b77f52e100327004ebdaa441829652a75dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-13T05:30:22.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:22.198+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:22.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:22.198+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:22.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:22.198+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:22.583 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:22 smithi189 podman[72090]: 2024-04-13 05:30:22.432630086 +0000 UTC m=+0.040858605 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:30:22.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:22 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:22.320+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:22.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:22 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:22.320+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:22.751 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:22 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:22.707+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:22.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:22 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:22.707+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:22.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:22 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:22.475+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:22.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:22 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:22.476+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:22.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:22 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:22.476+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:22.915 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:22 smithi189 podman[72090]: 2024-04-13 05:30:22.598242748 +0000 UTC m=+0.206471275 container create a00ce5cb7d2ed0f7c19e70f2c953df6aed1e912d164a5ee6933d1afea2ef185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:22.916 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:22 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:22.662+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:22.916 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:22 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:22.662+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:23.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:22.771+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:23.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:22.771+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:23.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:22 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:22.771+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:23.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:22 smithi189 podman[72090]: 2024-04-13 05:30:22.913772695 +0000 UTC m=+0.522001228 container init a00ce5cb7d2ed0f7c19e70f2c953df6aed1e912d164a5ee6933d1afea2ef185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:30:23.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:22 smithi189 podman[72090]: 2024-04-13 05:30:22.925320306 +0000 UTC m=+0.533548863 container start a00ce5cb7d2ed0f7c19e70f2c953df6aed1e912d164a5ee6933d1afea2ef185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-13T05:30:23.333 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:23 smithi189 podman[72090]: 2024-04-13 05:30:23.012858713 +0000 UTC m=+0.621087247 container attach a00ce5cb7d2ed0f7c19e70f2c953df6aed1e912d164a5ee6933d1afea2ef185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-13T05:30:23.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:23.187+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:23.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:23.187+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:23.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:23.187+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:23.669 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 13 05:30:23 smithi189 podman[72090]: 2024-04-13 05:30:23.374286915 +0000 UTC m=+0.982515437 container died a00ce5cb7d2ed0f7c19e70f2c953df6aed1e912d164a5ee6933d1afea2ef185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-04-13T05:30:23.669 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:23 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:23.343+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:23.669 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:23 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:23.343+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:23.669 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:23 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:23.343+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:23.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:23 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:23.451+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:23.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:23 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:23.451+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:23.724 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:23 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:23.451+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:24.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:23 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:23.720+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:24.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:23 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:23.720+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:24.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:23 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:23.666+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:24.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:23 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:23.666+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:24.168 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.5.service' 2024-04-13T05:30:24.237 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:23.810+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:24.237 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:23.810+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:24.237 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:23 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:23.810+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:24.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:30:24.295 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-13T05:30:24.295 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-13T05:30:24.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.6 2024-04-13T05:30:24.533 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 systemd[1]: Stopping Ceph osd.6 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:30:24.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:24.234+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:24.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:24.234+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:24.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:24.234+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:24.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:24.234+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:24.671 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:24 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:24.348+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:24.671 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:24 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:24.348+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:24.671 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:24 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:24.348+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:24.733 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:24.459+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:24.733 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:24.459+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:24.733 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:24.459+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:24.733 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:24.459+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:25.001 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:24.729+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:25.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:24.729+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:25.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:24 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:24.729+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:25.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:24.772+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:25.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:24.772+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:25.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:24.772+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:25.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:24.903+0000 7f8fb8c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:30:25.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:24.903+0000 7f8fb8c00700 -1 osd.6 98 *** Got signal Terminated *** 2024-04-13T05:30:25.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:24.903+0000 7f8fb8c00700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:30:25.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:24 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:24.668+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:25.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:24 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:24.668+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:25.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:25.203+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:25.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:25.203+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:25.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:25.203+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:25.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:25.203+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:25.685 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:25 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:25.334+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:25.685 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:25 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:25.334+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:25.685 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:25 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:25.334+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:25.681+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:25.681+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:25.681+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:25.468+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:25.468+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:25.468+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:25.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:25 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:25.468+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:26.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:25.774+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:26.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:25.774+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:26.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:25 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:25.774+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:26.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:25 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:25.682+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:26.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:25 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:25.682+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:26.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:25 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:25.682+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:26.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:26.179+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:26.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:26.179+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:26.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:26.179+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:26.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:26.179+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:26.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:26.288+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:26.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:26.288+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:26.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:26.288+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:26.706+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:26.706+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:26.706+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:26.466+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:26.466+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:26.466+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:26.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:26 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:26.466+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:27.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:26.755+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:27.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:26.755+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:27.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:26 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:26.755+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:27.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:26.715+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:27.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:26.715+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:27.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:26 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:26.715+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:27.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:27.149+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:27.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:27.149+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:27.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:27.149+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:27.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:27.149+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:27.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:27.243+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:27.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:27.243+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:27.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:27.243+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:27.700+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:27.700+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:27.700+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:27.470+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:27.470+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:27.470+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:27.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:27 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:27.470+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:28.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:27.732+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:28.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:27.732+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:28.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:27 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:27.732+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:28.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:27.706+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:28.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:27.706+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:28.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:27 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:27.706+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:28.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:28.108+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:28.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:28.108+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:28.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:28.108+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:28.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:28.108+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:28.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:28 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:28.281+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:28.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:28 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:28.281+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:28.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:28 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:28.281+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:28.750+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:28.750+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:28.750+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:28.503+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:28.503+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:28.503+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:29.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:28 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:28.503+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:29.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:28.753+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:29.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:28.753+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:29.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:28.753+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:29.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:28 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:28.671+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:29.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:28 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:28.671+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:29.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:28 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:28.671+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:29.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:29.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:29.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:29.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:29.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:29.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:29.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:29.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:29.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:29 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:29.254+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:29.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:29 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:29.254+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:29.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:29 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:29.254+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:29.791 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:29.527+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:29.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:29.527+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:29.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:29.527+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:29.792 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:29.527+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:30.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:29.713+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.143320+0000 front 2024-04-13T05:29:40.143380+0000 (oldest deadline 2024-04-13T05:30:06.544186+0000) 2024-04-13T05:30:30.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:29.713+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.646511+0000 front 2024-04-13T05:29:50.646552+0000 (oldest deadline 2024-04-13T05:30:15.946713+0000) 2024-04-13T05:30:30.033 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:29 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6[61332]: 2024-04-13T05:30:29.713+0000 7f8fb4400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.648736+0000 front 2024-04-13T05:30:00.648878+0000 (oldest deadline 2024-04-13T05:30:22.348916+0000) 2024-04-13T05:30:30.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:29 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:29.647+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:30.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:29 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:29.647+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:30.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:29 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:29.647+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:30.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:29.788+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:30.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:29.788+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:30.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:29 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:29.788+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:30.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:30.054+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:30.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:30.054+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:30.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:30.054+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:30.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:30 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:30.054+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:30.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:30 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:30.265+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:30.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:30 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:30.265+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:30.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:30 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:30.265+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:30.854 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:30 smithi057 podman[170359]: 2024-04-13 05:30:30.546398311 +0000 UTC m=+5.800509952 container died c6b1c2225113a91c24a21021748abfc9f2addd686789747108f464f1a618ebac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:30.757+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:30.757+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:30.757+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:30.570+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:30.570+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:30.570+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:31.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:30 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:30.570+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:31.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:30 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:30.613+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:31.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:30 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:30.613+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:31.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:30 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:30.613+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:31.242 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.008+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:31.243 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.008+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:31.243 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.008+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:31.243 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.008+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:31.243 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:30 smithi057 podman[170359]: 2024-04-13 05:30:30.963690526 +0000 UTC m=+6.217802156 container cleanup c6b1c2225113a91c24a21021748abfc9f2addd686789747108f464f1a618ebac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:30:31.243 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:30 smithi057 bash[170359]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6 2024-04-13T05:30:31.532 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:31 smithi057 podman[170378]: 2024-04-13 05:30:31.240031709 +0000 UTC m=+0.678853284 container remove c6b1c2225113a91c24a21021748abfc9f2addd686789747108f464f1a618ebac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-13T05:30:31.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:31 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:31.223+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:31.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:31 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:31.223+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:31.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:31 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:31.223+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:31.744+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:31.744+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:31.744+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:31.558+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:31.558+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:31.558+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:32.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:31 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:31.558+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:32.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:31 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:31.644+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:32.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:31 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:31.644+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:32.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:31 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:31.644+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:32.283 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:32 smithi057 podman[170462]: 2024-04-13 05:30:32.107434228 +0000 UTC m=+0.119328980 container create 61af92adf60c602ab309d46a0066e3acc8bbae269c0aa20fc8b9baa1f070d7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-13T05:30:32.283 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:32 smithi057 podman[170462]: 2024-04-13 05:30:32.020275771 +0000 UTC m=+0.032170513 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:30:32.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.965+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:32.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.965+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:32.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.965+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:32.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:31 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:31.965+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:32.460 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:32.222+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:32.460 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:32.222+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:32.460 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:32.222+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:32.771 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:32.626+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:32.772 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:32.626+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:32.772 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:32.626+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:32.772 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:32 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:32.626+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:32.782 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:32 smithi057 podman[170462]: 2024-04-13 05:30:32.403420081 +0000 UTC m=+0.415314856 container init 61af92adf60c602ab309d46a0066e3acc8bbae269c0aa20fc8b9baa1f070d7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-13T05:30:32.783 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:32 smithi057 podman[170462]: 2024-04-13 05:30:32.415078431 +0000 UTC m=+0.426973202 container start 61af92adf60c602ab309d46a0066e3acc8bbae269c0aa20fc8b9baa1f070d7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:32.783 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:32 smithi057 podman[170462]: 2024-04-13 05:30:32.501901251 +0000 UTC m=+0.513796062 container attach 61af92adf60c602ab309d46a0066e3acc8bbae269c0aa20fc8b9baa1f070d7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:33.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:32.728+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:33.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:32.728+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:33.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:32.728+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:33.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:32.579+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:33.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:32.579+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:33.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:32.579+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:33.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:32 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:32.580+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:33.283 INFO:journalctl@ceph.osd.6.smithi057.stdout:Apr 13 05:30:32 smithi057 podman[170462]: 2024-04-13 05:30:32.887754484 +0000 UTC m=+0.899649253 container died 61af92adf60c602ab309d46a0066e3acc8bbae269c0aa20fc8b9baa1f070d7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True) 2024-04-13T05:30:33.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:32.977+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:33.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:32.977+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:33.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:32.977+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:33.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:32.977+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:33.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:32.977+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:33.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:33.184+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:33.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:33.184+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:33.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:33.184+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:33.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:33.184+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:33.750 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.6.service' 2024-04-13T05:30:33.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:30:33.880 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-13T05:30:33.880 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-13T05:30:33.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.7 2024-04-13T05:30:33.967 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:33.557+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:33.557+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:33.557+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:33.557+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:33.755+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:33.755+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:33.755+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:33.968 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:33 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:33.755+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.987004+0000 front 2024-04-13T05:30:09.986844+0000 (oldest deadline 2024-04-13T05:30:33.486781+0000) 2024-04-13T05:30:34.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:33.603+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:34.084 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:33.603+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:34.084 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:33.603+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:34.084 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:33 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:33.603+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:34.251 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 systemd[1]: Stopping Ceph osd.7 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:30:34.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:33.984+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:34.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:33.984+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:34.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:33.984+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:34.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:33.984+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:34.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:33 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:33.984+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:34.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:34.191+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:34.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:34.191+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:34.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:34.191+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:34.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:34.191+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:34.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.438+0000 7fa23f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:30:34.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.438+0000 7fa23f600700 -1 osd.7 98 *** Got signal Terminated *** 2024-04-13T05:30:34.752 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.438+0000 7fa23f600700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:30:34.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:34.551+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:34.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:34.551+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:34.752 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:34.551+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:34.753 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:34.551+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:35.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:34.591+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:35.083 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:34.591+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:35.084 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:34.591+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:35.084 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:34 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:34.591+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:35.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.761+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:35.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.761+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:35.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.761+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:35.253 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:34 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:34.761+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.987004+0000 front 2024-04-13T05:30:09.986844+0000 (oldest deadline 2024-04-13T05:30:33.486781+0000) 2024-04-13T05:30:35.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:34.980+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:35.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:34.980+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:35.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:34.980+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:35.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:34.980+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:35.282 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:34 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:34.980+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:35.570 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:35.179+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:35.570 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:35.179+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:35.570 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:35.179+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:35.570 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:35.179+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:35.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:35.567+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:35.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:35.567+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:35.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:35.567+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:35.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:35 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:35.567+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:35.715+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:35.715+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:35.715+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:35.715+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.987004+0000 front 2024-04-13T05:30:09.986844+0000 (oldest deadline 2024-04-13T05:30:33.486781+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:35.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:35.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:36.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:35.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:36.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:35.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:36.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:35 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:35.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:36.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:35 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:35.951+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:36.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:35 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:35.951+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:36.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:35 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:35.951+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:36.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:35 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:35.951+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:36.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:35 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:35.951+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:36.522 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:36.196+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:36.522 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:36.196+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:36.522 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:36.196+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:36.522 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:36.196+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:36.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:36.519+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:36.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:36.519+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:36.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:36.519+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:36.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:36 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:36.519+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:37.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:36.740+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:37.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:36.740+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:37.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:36.740+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:37.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:36.740+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.987004+0000 front 2024-04-13T05:30:09.986844+0000 (oldest deadline 2024-04-13T05:30:33.486781+0000) 2024-04-13T05:30:37.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:36.559+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:37.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:36.559+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:37.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:36.559+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:37.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:36.559+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:37.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:36.559+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:37.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:36 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:36.990+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:37.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:36 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:36.990+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:37.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:36 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:36.990+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:37.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:36 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:36.990+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:37.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:36 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:36.990+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:37.500 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:37.204+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:37.500 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:37.204+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:37.500 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:37.204+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:37.500 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:37.204+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:37.789 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:37.512+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:37.789 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:37.512+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:37.789 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:37.512+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:37.789 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:37.512+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:37.789 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:37.512+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:37.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:37.496+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:37.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:37.496+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:37.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:37.497+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:37.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:37 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:37.497+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:38.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:37.785+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:38.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:37.785+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:38.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:37.785+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:38.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:37 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:37.785+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.987004+0000 front 2024-04-13T05:30:09.986844+0000 (oldest deadline 2024-04-13T05:30:33.486781+0000) 2024-04-13T05:30:38.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:37 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:37.949+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:38.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:37 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:37.949+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:38.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:37 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:37.949+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:38.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:37 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:37.949+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:38.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:37 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:37.949+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:38.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:38.253+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:38.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:38.253+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:38.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:38.253+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:38.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:38.253+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:38.583 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:38.466+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:38.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:38.466+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:38.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:38.466+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:38.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:38 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:38.466+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:38.738+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.879696+0000 front 2024-04-13T05:29:41.879866+0000 (oldest deadline 2024-04-13T05:30:06.579803+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:38.738+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.881872+0000 front 2024-04-13T05:29:51.881703+0000 (oldest deadline 2024-04-13T05:30:15.981962+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:38.738+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.483867+0000 front 2024-04-13T05:29:59.484058+0000 (oldest deadline 2024-04-13T05:30:24.183724+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7[53265]: 2024-04-13T05:30:38.738+0000 7fa23ae00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.987004+0000 front 2024-04-13T05:30:09.986844+0000 (oldest deadline 2024-04-13T05:30:33.486781+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:38.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:38.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:39.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:38.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:39.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:38.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:39.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:38 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:38.561+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:39.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:38 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:38.957+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:39.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:38 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:38.957+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:39.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:38 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:38.957+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:39.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:38 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:38.957+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:39.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:38 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:38.957+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:39.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:39.230+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:39.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:39.230+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:39.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:39.230+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:39.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:39.230+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:39.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:39.448+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:39.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:39.448+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:39.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:39.448+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:39.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:39 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:39.448+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:39.984 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:39 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:39.571+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:39.984 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:39 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:39.571+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:39.984 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:39 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:39.571+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:39.984 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:39 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:39.571+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:39.984 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:39 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:39.571+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:40.252 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:39 smithi083 podman[71488]: 2024-04-13 05:30:39.982017574 +0000 UTC m=+5.698179548 container died 1a5b1b0587bbfd6f372a0c6ebb0caa8fd85377605a602fb2ced84b2fb29530ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-13T05:30:40.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:39 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:39.975+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:40.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:39 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:39.975+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:40.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:39 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:39.975+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:40.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:39 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:39.975+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:40.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:39 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:39.975+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:40.482 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:40.218+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:40.482 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:40.218+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:40.482 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:40.218+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:40.482 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:40.218+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:40.593 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:40 smithi083 podman[71488]: 2024-04-13 05:30:40.372969934 +0000 UTC m=+6.089131917 container cleanup 1a5b1b0587bbfd6f372a0c6ebb0caa8fd85377605a602fb2ced84b2fb29530ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-13T05:30:40.593 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:40 smithi083 bash[71488]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7 2024-04-13T05:30:40.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:40.479+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:40.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:40.479+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:40.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:40.479+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:40.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:40 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:40.479+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:41.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:40 smithi083 podman[71506]: 2024-04-13 05:30:40.590717833 +0000 UTC m=+0.592529327 container remove 1a5b1b0587bbfd6f372a0c6ebb0caa8fd85377605a602fb2ced84b2fb29530ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:30:41.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:40.594+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:41.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:40.594+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:41.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:40.594+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:41.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:40.594+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:41.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:40.594+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:41.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:40 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:40.987+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:41.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:40 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:40.987+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:41.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:40 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:40.987+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:41.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:40 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:40.987+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:41.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:40 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:40.987+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:41.489 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:41.214+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:41.489 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:41.214+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:41.489 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:41.214+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:41.489 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:41.214+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:41.664 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:41 smithi083 podman[71592]: 2024-04-13 05:30:41.39989144 +0000 UTC m=+0.111020369 container create 56fbe20b6522e32437b3cb743d0d0356f04c8dd1463e12a7164e82b994c23b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-13T05:30:41.664 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:41 smithi083 podman[71592]: 2024-04-13 05:30:41.320061083 +0000 UTC m=+0.031189976 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:30:41.665 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:41 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:41.617+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:41.665 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:41 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:41.617+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:41.665 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:41 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:41.617+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:41.665 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:41 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:41.617+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:41.665 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:41 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:41.617+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:41.665 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:41 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:41.617+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:41.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:41.486+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:41.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:41.486+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:41.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:41.486+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:41.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:41 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:41.486+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:42.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:41 smithi083 podman[71592]: 2024-04-13 05:30:41.661997338 +0000 UTC m=+0.373126277 container init 56fbe20b6522e32437b3cb743d0d0356f04c8dd1463e12a7164e82b994c23b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-13T05:30:42.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:41 smithi083 podman[71592]: 2024-04-13 05:30:41.675672628 +0000 UTC m=+0.386801552 container start 56fbe20b6522e32437b3cb743d0d0356f04c8dd1463e12a7164e82b994c23b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-13T05:30:42.002 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:41 smithi083 podman[71592]: 2024-04-13 05:30:41.769097052 +0000 UTC m=+0.480225997 container attach 56fbe20b6522e32437b3cb743d0d0356f04c8dd1463e12a7164e82b994c23b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-13T05:30:42.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:41.996+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:42.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:41.996+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:42.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:41.996+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:42.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:41.996+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:42.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:41 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:41.996+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:42.400 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:42 smithi083 podman[71592]: 2024-04-13 05:30:42.128040745 +0000 UTC m=+0.839169654 container died 56fbe20b6522e32437b3cb743d0d0356f04c8dd1463e12a7164e82b994c23b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-13T05:30:42.532 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:42.192+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:42.533 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:42.192+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:42.533 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:42.192+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:42.533 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:42.192+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:42.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:42.529+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:42.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:42.529+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:42.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:42.529+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:42.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:42.529+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:42.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:42 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:42.529+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:18.102529+0000 front 2024-04-13T05:30:18.102233+0000 (oldest deadline 2024-04-13T05:30:42.103177+0000) 2024-04-13T05:30:42.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.7.service' 2024-04-13T05:30:42.945 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:42 smithi083 podman[71618]: 2024-04-13 05:30:42.814545243 +0000 UTC m=+0.670621363 container remove 56fbe20b6522e32437b3cb743d0d0356f04c8dd1463e12a7164e82b994c23b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True) 2024-04-13T05:30:42.945 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:42 smithi083 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-13T05:30:42.945 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:42 smithi083 systemd[1]: Stopped Ceph osd.7 for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:30:42.945 INFO:journalctl@ceph.osd.7.smithi083.stdout:Apr 13 05:30:42 smithi083 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.7.service: Consumed 1min 39.998s CPU time. 2024-04-13T05:30:42.945 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:42 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:42.664+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:42.945 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:42 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:42.664+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:42.946 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:42 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:42.664+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:42.946 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:42 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:42.664+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:42.946 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:42 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:42.664+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:42.946 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:42 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:42.664+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:42.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:30:42.980 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-13T05:30:42.980 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-13T05:30:42.980 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.8 2024-04-13T05:30:43.334 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 systemd[1]: Stopping Ceph osd.8 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:30:43.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:43.194+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:43.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:43.194+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:43.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:43.194+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:43.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:43.194+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:43.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:43 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:43.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:43.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:43 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:43.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:43.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:43 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:43.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:43.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:43 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:43.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:43.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:43 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:43.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:43.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:43 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:43.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:18.102529+0000 front 2024-04-13T05:30:18.102233+0000 (oldest deadline 2024-04-13T05:30:42.103177+0000) 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.559+0000 7fe521800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.559+0000 7fe521800700 -1 osd.8 98 *** Got signal Terminated *** 2024-04-13T05:30:43.766 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:43 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:43.559+0000 7fe521800700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:30:44.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:43 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:43.667+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:44.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:43 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:43.667+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:44.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:43 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:43.667+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:44.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:43 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:43.667+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:44.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:43 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:43.667+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:44.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:43 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:43.667+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:44.514 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:44.190+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:44.514 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:44.190+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:44.515 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:44.190+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:44.515 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:44.190+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:44.515 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:44.190+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:44.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:44 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:44.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:44.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:44 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:44.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:44.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:44 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:44.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:44.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:44 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:44.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:44.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:44 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:44.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:44.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:44 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:44.074+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:44.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:44.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:44.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:44.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:44.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:44.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:44.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:44.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:44.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:44 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:44.512+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:18.102529+0000 front 2024-04-13T05:30:18.102233+0000 (oldest deadline 2024-04-13T05:30:42.103177+0000) 2024-04-13T05:30:45.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:44.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:45.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:44.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:45.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:44.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:45.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:44.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:45.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:44.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:45.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:44.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:45.516 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:45.149+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:45.517 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:45.149+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:45.517 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:45.149+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:45.517 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:45.149+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:45.517 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:45.149+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:45.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:45 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:45.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:45.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:45 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:45.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:45.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:45 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:45.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:45.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:45 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:45.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:45.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:45 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:45.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:45.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:45 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:45.122+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:45.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:45.513+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:45.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:45.513+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:45.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:45.513+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:45.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:45.514+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:45.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:45 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:45.514+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:18.102529+0000 front 2024-04-13T05:30:18.102233+0000 (oldest deadline 2024-04-13T05:30:42.103177+0000) 2024-04-13T05:30:46.001 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:45 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:45.726+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:46.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:45 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:45.726+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:46.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:45 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:45.726+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:46.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:45 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:45.726+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:46.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:45 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:45.726+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:46.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:45 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:45.726+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:46.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:46 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:46.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:46.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:46 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:46.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:46.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:46 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:46.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:46.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:46 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:46.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:46.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:46 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:46.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:46.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:46 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:46.155+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:46.528 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:46.136+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:46.528 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:46.136+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:46.528 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:46.136+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:46.528 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:46.136+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:46.528 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:46.136+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:46.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:46.525+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:46.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:46.525+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:46.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:46.525+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:46.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:46.525+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:46.834 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:46 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:46.525+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:18.102529+0000 front 2024-04-13T05:30:18.102233+0000 (oldest deadline 2024-04-13T05:30:42.103177+0000) 2024-04-13T05:30:47.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:46.702+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:47.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:46.702+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:47.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:46.702+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:47.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:46.702+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:47.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:46.702+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:47.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:46 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:46.702+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:47.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:47 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:47.125+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:47.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:47 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:47.125+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:47.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:47 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:47.125+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:47.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:47 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:47.125+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:47.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:47 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:47.125+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:47.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:47 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:47.125+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:47.564 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:47.086+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:47.564 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:47.086+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:47.564 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:47.086+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:47.564 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:47.086+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:47.564 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:47.086+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:47.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:47.562+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:38.892667+0000 front 2024-04-13T05:29:38.892624+0000 (oldest deadline 2024-04-13T05:30:04.192896+0000) 2024-04-13T05:30:47.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:47.562+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:48.894777+0000 front 2024-04-13T05:29:48.895269+0000 (oldest deadline 2024-04-13T05:30:12.994941+0000) 2024-04-13T05:30:47.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:47.562+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:59.398043+0000 front 2024-04-13T05:29:59.398110+0000 (oldest deadline 2024-04-13T05:30:24.698243+0000) 2024-04-13T05:30:47.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:47.562+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.000464+0000 front 2024-04-13T05:30:10.000117+0000 (oldest deadline 2024-04-13T05:30:32.300014+0000) 2024-04-13T05:30:47.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:47 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8[53889]: 2024-04-13T05:30:47.562+0000 7fe51d000700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:18.102529+0000 front 2024-04-13T05:30:18.102233+0000 (oldest deadline 2024-04-13T05:30:42.103177+0000) 2024-04-13T05:30:48.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:47 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:47.673+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:48.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:47 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:47.673+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:48.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:47 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:47.673+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:48.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:47 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:47.673+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:48.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:47 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:47.673+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:48.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:47 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:47.673+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:48.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:48.070+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:48.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:48.070+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:48.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:48.070+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:48.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:48.070+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:48.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:48 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:48.070+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:48.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:48 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:48.127+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:48.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:48 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:48.127+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:48.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:48 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:48.127+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:48.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:48 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:48.127+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:48.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:48 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:48.127+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:48.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:48 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:48.127+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:49.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:48.678+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:49.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:48.678+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:49.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:48.678+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:49.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:48.678+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:49.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:48.678+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:49.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:48.678+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:49.333 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:49 smithi189 podman[72205]: 2024-04-13 05:30:49.117782406 +0000 UTC m=+5.699892031 container died 82332f64c71edbfe25862acb283fdad0d5c6a105caf56647bf2bbc18c05bcf1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:49.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:49.073+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:49.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:49.073+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:49.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:49.073+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:49.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:49.073+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:49.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:49 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:49.073+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:49.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:49 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:49.089+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:49.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:49 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:49.089+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:49.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:49 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:49.089+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:49.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:49 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:49.089+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:49.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:49 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:49.089+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:49.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:49 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:49.089+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:49.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:49 smithi189 podman[72205]: 2024-04-13 05:30:49.593471789 +0000 UTC m=+6.175581808 container cleanup 82332f64c71edbfe25862acb283fdad0d5c6a105caf56647bf2bbc18c05bcf1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:30:49.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:49 smithi189 bash[72205]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8 2024-04-13T05:30:50.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:49 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:49.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:50.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:49 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:49.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:50.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:49 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:49.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:50.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:49 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:49.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:50.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:49 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:49.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:50.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:49 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:49.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:50.333 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:49 smithi189 podman[72224]: 2024-04-13 05:30:49.936694629 +0000 UTC m=+0.801193048 container remove 82332f64c71edbfe25862acb283fdad0d5c6a105caf56647bf2bbc18c05bcf1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-13T05:30:50.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:50.056+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:50.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:50.056+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:50.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:50.056+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:50.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:50.056+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:50.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:50 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:50.056+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:50.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:50 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:50.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:50.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:50 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:50.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:50.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:50 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:50.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:50.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:50 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:50.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:50.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:50 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:50.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:50.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:50 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:50.039+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:50.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:50 smithi189 podman[72315]: 2024-04-13 05:30:50.452533753 +0000 UTC m=+0.035477797 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:30:50.833 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:50 smithi189 podman[72315]: 2024-04-13 05:30:50.604114929 +0000 UTC m=+0.187059009 container create cd3850cdcc288fe0192111a0f321a1491a76eb65946c56e76f9b98e47d8cf657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-13T05:30:51.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:50 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:50.643+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:51.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:50 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:50.643+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:51.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:50 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:50.643+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:51.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:50 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:50.643+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:51.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:50 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:50.643+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:51.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:50 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:50.643+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:51.333 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:50 smithi189 podman[72315]: 2024-04-13 05:30:50.924896602 +0000 UTC m=+0.507840725 container init cd3850cdcc288fe0192111a0f321a1491a76eb65946c56e76f9b98e47d8cf657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-13T05:30:51.333 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:50 smithi189 podman[72315]: 2024-04-13 05:30:50.936047736 +0000 UTC m=+0.518991846 container start cd3850cdcc288fe0192111a0f321a1491a76eb65946c56e76f9b98e47d8cf657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-13T05:30:51.333 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:51 smithi189 podman[72315]: 2024-04-13 05:30:51.040284632 +0000 UTC m=+0.623228759 container attach cd3850cdcc288fe0192111a0f321a1491a76eb65946c56e76f9b98e47d8cf657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-13T05:30:51.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:51.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:51.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:51.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:51.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:51.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:51 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:51.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:51.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:51 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:51.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:51.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:51 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:51.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:51.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:51 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:51.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:51.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:51 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:51.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:51.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:51 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:51.083+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:51.741 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:51 smithi189 podman[72315]: 2024-04-13 05:30:51.390435893 +0000 UTC m=+0.973379970 container died cd3850cdcc288fe0192111a0f321a1491a76eb65946c56e76f9b98e47d8cf657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-13T05:30:52.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:51 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:51.662+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:52.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:51 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:51.662+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:52.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:51 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:51.662+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:52.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:51 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:51.662+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:52.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:51 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:51.662+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:52.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:51 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:51.662+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:52.224 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.8.service' 2024-04-13T05:30:52.307 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:52 smithi189 podman[72340]: 2024-04-13 05:30:52.167943866 +0000 UTC m=+0.756114462 container remove cd3850cdcc288fe0192111a0f321a1491a76eb65946c56e76f9b98e47d8cf657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-13T05:30:52.307 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:52 smithi189 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-13T05:30:52.307 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:52 smithi189 systemd[1]: Stopped Ceph osd.8 for 0a178454-f953-11ee-bc8d-c7b262605968. 2024-04-13T05:30:52.307 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 13 05:30:52 smithi189 systemd[1]: ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.8.service: Consumed 1min 50.689s CPU time. 2024-04-13T05:30:52.307 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.997+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:52.307 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.997+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:52.308 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.997+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:52.308 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.997+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:52.308 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.997+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:52.308 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:51 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:51.997+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:52.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:30:52.350 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-13T05:30:52.350 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-13T05:30:52.350 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.9 2024-04-13T05:30:52.446 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:52.446 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:52.446 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:52.446 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:52.446 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:52.446 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.052+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:52.782 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 systemd[1]: Stopping Ceph osd.9 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:30:53.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:53.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:53.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:53.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:53.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:53.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:53.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:52 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:52.711+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:53.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.957+0000 7f73b4400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.957+0000 7f73b4400700 -1 osd.9 98 *** Got signal Terminated *** 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:52 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:52.957+0000 7f73b4400700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:53 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:53.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:53 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:53.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:53 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:53.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:53 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:53.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:53 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:53.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:53.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:53 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:53.072+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:53.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:53.045+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:53.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:53.045+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:53.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:53.045+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:53.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:53.045+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:53.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:53.045+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:53.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:53 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:53.045+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:54.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:53 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:53.755+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:54.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:54.048+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:54.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:54.048+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:54.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:54.048+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:54.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:54.048+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:54.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:54.048+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:54.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:54 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:54.048+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:54.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:54 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:54.092+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:54.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:54 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:54.092+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:54.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:54 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:54.092+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:54.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:54 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:54.092+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:54.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:54 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:54.092+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:54.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:54 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:54.092+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:55.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:54 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:54.749+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:55.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:55.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:55.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:55.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:55.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:55.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:55.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:55.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:55.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:55.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:55.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:55 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:55.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:55.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:55.068+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:55.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:55.068+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:55.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:55.068+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:55.534 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:55.068+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:55.534 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:55.068+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:55.534 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:55 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:55.068+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:56.030 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:56.030 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:56.030 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:56.030 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:56.030 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:56.030 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:56.031 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:55 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:55.759+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:56.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:56.024+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:56.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:56.024+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:56.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:56.024+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:56.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:56.024+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:56.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:56.024+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:56.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:56 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:56.024+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:56.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:56.039+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:56.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:56.039+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:56.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:56.039+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:56.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:56.039+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:56.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:56.039+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:56.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:56.039+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:57.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:57.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:57.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:57.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:56 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:56.725+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:57.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:57.037+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:31.505643+0000 front 2024-04-13T05:29:31.505194+0000 (oldest deadline 2024-04-13T05:29:56.205488+0000) 2024-04-13T05:30:57.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:57.037+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:40.809939+0000 front 2024-04-13T05:29:40.809767+0000 (oldest deadline 2024-04-13T05:30:06.710023+0000) 2024-04-13T05:30:57.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:57.037+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:50.713059+0000 front 2024-04-13T05:29:50.713898+0000 (oldest deadline 2024-04-13T05:30:14.212969+0000) 2024-04-13T05:30:57.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:57.037+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:01.216542+0000 front 2024-04-13T05:30:01.216924+0000 (oldest deadline 2024-04-13T05:30:24.116520+0000) 2024-04-13T05:30:57.284 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:57.037+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:08.218872+0000 front 2024-04-13T05:30:08.218474+0000 (oldest deadline 2024-04-13T05:30:32.918102+0000) 2024-04-13T05:30:57.285 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:57 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9[68367]: 2024-04-13T05:30:57.037+0000 7f73afc00700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:16.321046+0000 front 2024-04-13T05:30:16.320996+0000 (oldest deadline 2024-04-13T05:30:42.721759+0000) 2024-04-13T05:30:57.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:57.030+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:57.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:57.030+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:57.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:57.030+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:57.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:57.030+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:57.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:57.030+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:57.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:57 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:57.030+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:58.252 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:57 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:57.758+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:58.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:58.068+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:58.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:58.068+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:58.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:58.068+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:58.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:58.068+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:58.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:58.068+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:58.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:58 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:58.068+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:58.866 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:58 smithi057 podman[170578]: 2024-04-13 05:30:58.548684525 +0000 UTC m=+5.728766826 container died 7e1bc1d72448caabc67438dbbb258be1f9f1c3e2684e4a220fe6750db4a49d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:30:59.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:58 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:58.715+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:30:59.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:58 smithi057 podman[170578]: 2024-04-13 05:30:58.966412415 +0000 UTC m=+6.146494689 container cleanup 7e1bc1d72448caabc67438dbbb258be1f9f1c3e2684e4a220fe6750db4a49d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:30:59.283 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:58 smithi057 bash[170578]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9 2024-04-13T05:30:59.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:59.091+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:30:59.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:59.091+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:30:59.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:59.091+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:30:59.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:59.091+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:30:59.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:59.091+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:30:59.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:30:59 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:30:59.091+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:30:59.783 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:30:59 smithi057 podman[170608]: 2024-04-13 05:30:59.284424878 +0000 UTC m=+0.719211762 container remove 7e1bc1d72448caabc67438dbbb258be1f9f1c3e2684e4a220fe6750db4a49d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:00.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:30:59 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:30:59.695+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:00.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:31:00 smithi057 podman[170694]: 2024-04-13 05:31:00.226756917 +0000 UTC m=+0.127053431 container create 3ab0023b32c37ff28998f087e45c2797037ccb1cbc58b7427c80a0a42a508e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-13T05:31:00.533 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:31:00 smithi057 podman[170694]: 2024-04-13 05:31:00.134138015 +0000 UTC m=+0.034434481 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:31:00.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:00.096+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:00.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:00.096+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:00.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:00.096+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:00.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:00.096+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:00.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:00.096+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:00.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:00.096+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:01.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:00 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:00.683+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:01.032 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:31:00 smithi057 podman[170694]: 2024-04-13 05:31:00.55132681 +0000 UTC m=+0.451623307 container init 3ab0023b32c37ff28998f087e45c2797037ccb1cbc58b7427c80a0a42a508e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-13T05:31:01.033 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:31:00 smithi057 podman[170694]: 2024-04-13 05:31:00.562854528 +0000 UTC m=+0.463151044 container start 3ab0023b32c37ff28998f087e45c2797037ccb1cbc58b7427c80a0a42a508e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-13T05:31:01.033 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:31:00 smithi057 podman[170694]: 2024-04-13 05:31:00.645917185 +0000 UTC m=+0.546213681 container attach 3ab0023b32c37ff28998f087e45c2797037ccb1cbc58b7427c80a0a42a508e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:01.382 INFO:journalctl@ceph.osd.9.smithi057.stdout:Apr 13 05:31:01 smithi057 podman[170694]: 2024-04-13 05:31:01.033880125 +0000 UTC m=+0.934176675 container died 3ab0023b32c37ff28998f087e45c2797037ccb1cbc58b7427c80a0a42a508e83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-13T05:31:01.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:01.081+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:01.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:01.081+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:01.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:01.081+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:01.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:01.081+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:01.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:01.081+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:01.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:01 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:01.081+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:01.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.9.service' 2024-04-13T05:31:01.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:01.918 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-13T05:31:01.918 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-13T05:31:01.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.10 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:02.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:01 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:01.703+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:02.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:02.078+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:02.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:02.078+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:02.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:02.078+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:02.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:02.078+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:02.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:02.078+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:02.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:02 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:02.078+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:02.502 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 systemd[1]: Stopping Ceph osd.10 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:03.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.524+0000 7f830ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:03.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.524+0000 7f830ca00700 -1 osd.10 98 *** Got signal Terminated *** 2024-04-13T05:31:03.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.524+0000 7f830ca00700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:31:03.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:03.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:03.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:03.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:03.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:03.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:03.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:02 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:02.676+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:03.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:03.128+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:03.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:03.128+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:03.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:03.128+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:03.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:03.128+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:03.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:03.128+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:03.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:03 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:03.128+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:04.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:04.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:04.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:04.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:04.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:04.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:04.003 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:03 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:03.644+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:04.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:04.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:04.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:04.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:04.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:04.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:04.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:04.155+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:05.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:04 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:04.692+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:05.583 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:05.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:05.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:05.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:05.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:05.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:05.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:05 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:05.106+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:06.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:05 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:05.642+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:06.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:06 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:06.072+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.383099+0000 front 2024-04-13T05:29:32.383335+0000 (oldest deadline 2024-04-13T05:29:56.483317+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6804 osd.1 since back 2024-04-13T05:29:41.185772+0000 front 2024-04-13T05:29:41.185694+0000 (oldest deadline 2024-04-13T05:30:06.387044+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.089038+0000 front 2024-04-13T05:29:51.088940+0000 (oldest deadline 2024-04-13T05:30:14.589186+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:30:00.893788+0000 front 2024-04-13T05:30:00.894119+0000 (oldest deadline 2024-04-13T05:30:24.294756+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:10.697395+0000 front 2024-04-13T05:30:10.697440+0000 (oldest deadline 2024-04-13T05:30:34.797446+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:17.099426+0000 front 2024-04-13T05:30:17.099770+0000 (oldest deadline 2024-04-13T05:30:41.199324+0000) 2024-04-13T05:31:07.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:06 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10[60166]: 2024-04-13T05:31:06.654+0000 7f8303200700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:28.202137+0000 front 2024-04-13T05:30:28.202332+0000 (oldest deadline 2024-04-13T05:30:52.302211+0000) 2024-04-13T05:31:07.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:07.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:07.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:07.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:07.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:07.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:07.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:07 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:07.066+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:08.283 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:08 smithi083 podman[71710]: 2024-04-13 05:31:08.011256391 +0000 UTC m=+5.661419143 container died 2ed01318733ba4c34812288f668e946224fcf17f9f2467943432515bb9251193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=) 2024-04-13T05:31:08.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:08.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:08.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:08.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:08.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:08.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:08.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:08 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:08.036+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:08.537 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:08 smithi083 podman[71710]: 2024-04-13 05:31:08.391979674 +0000 UTC m=+6.042142462 container cleanup 2ed01318733ba4c34812288f668e946224fcf17f9f2467943432515bb9251193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-13T05:31:08.537 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:08 smithi083 bash[71710]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10 2024-04-13T05:31:09.001 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:08 smithi083 podman[71729]: 2024-04-13 05:31:08.534705766 +0000 UTC m=+0.516802807 container remove 2ed01318733ba4c34812288f668e946224fcf17f9f2467943432515bb9251193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, io.buildah.version=1.33.5) 2024-04-13T05:31:09.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:09.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:09.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:09.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:09.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:09.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:09.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:09 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:09.017+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:09.605 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:09 smithi083 podman[71821]: 2024-04-13 05:31:09.242983916 +0000 UTC m=+0.032368913 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:31:09.605 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:09 smithi083 podman[71821]: 2024-04-13 05:31:09.368966246 +0000 UTC m=+0.158351253 container create 05e60b8ae4cd9ea272615885c19b4400290c0c99de27919ed2b0b1bd0fe1c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-13T05:31:09.606 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:09 smithi083 podman[71821]: 2024-04-13 05:31:09.60197633 +0000 UTC m=+0.391361361 container init 05e60b8ae4cd9ea272615885c19b4400290c0c99de27919ed2b0b1bd0fe1c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, ceph=True) 2024-04-13T05:31:10.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:09 smithi083 podman[71821]: 2024-04-13 05:31:09.61370815 +0000 UTC m=+0.403093141 container start 05e60b8ae4cd9ea272615885c19b4400290c0c99de27919ed2b0b1bd0fe1c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:31:10.002 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:09 smithi083 podman[71821]: 2024-04-13 05:31:09.730749523 +0000 UTC m=+0.520134528 container attach 05e60b8ae4cd9ea272615885c19b4400290c0c99de27919ed2b0b1bd0fe1c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True) 2024-04-13T05:31:10.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:10.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:10.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:10.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:10.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:10.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:10.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:10 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:10.006+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:10.466 INFO:journalctl@ceph.osd.10.smithi083.stdout:Apr 13 05:31:10 smithi083 podman[71821]: 2024-04-13 05:31:10.072286182 +0000 UTC m=+0.861671218 container died 05e60b8ae4cd9ea272615885c19b4400290c0c99de27919ed2b0b1bd0fe1c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-13T05:31:10.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.10.service' 2024-04-13T05:31:10.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:10.960 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-13T05:31:10.960 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-13T05:31:10.960 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.11 2024-04-13T05:31:11.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.052+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-13T05:30:44.863440+0000 front 2024-04-13T05:30:44.863356+0000 (oldest deadline 2024-04-13T05:31:10.163172+0000) 2024-04-13T05:31:11.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 systemd[1]: Stopping Ceph osd.11 for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:12.082 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.659+0000 7fd0d1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:12.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.659+0000 7fd0d1c00700 -1 osd.11 98 *** Got signal Terminated *** 2024-04-13T05:31:12.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:11 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:11.659+0000 7fd0d1c00700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:12.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:12 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:12.079+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-13T05:30:44.863440+0000 front 2024-04-13T05:30:44.863356+0000 (oldest deadline 2024-04-13T05:31:10.163172+0000) 2024-04-13T05:31:13.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:13.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:13.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:13.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:13.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:13.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:13.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:13.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:13 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:13.040+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-13T05:30:44.863440+0000 front 2024-04-13T05:30:44.863356+0000 (oldest deadline 2024-04-13T05:31:10.163172+0000) 2024-04-13T05:31:14.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.034+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:14.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.034+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:14.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.034+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:14.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:14.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:14.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:14.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:14.335 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:14 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:14.035+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-13T05:30:44.863440+0000 front 2024-04-13T05:30:44.863356+0000 (oldest deadline 2024-04-13T05:31:10.163172+0000) 2024-04-13T05:31:15.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:15.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:15 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:15.029+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-13T05:30:44.863440+0000 front 2024-04-13T05:30:44.863356+0000 (oldest deadline 2024-04-13T05:31:10.163172+0000) 2024-04-13T05:31:16.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-04-13T05:29:32.037960+0000 front 2024-04-13T05:29:32.038498+0000 (oldest deadline 2024-04-13T05:29:54.938084+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-13T05:29:51.944283+0000 front 2024-04-13T05:29:51.944329+0000 (oldest deadline 2024-04-13T05:30:16.044394+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-04-13T05:29:58.946729+0000 front 2024-04-13T05:29:58.946492+0000 (oldest deadline 2024-04-13T05:30:22.446577+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6812 osd.4 since back 2024-04-13T05:30:09.351887+0000 front 2024-04-13T05:30:09.352150+0000 (oldest deadline 2024-04-13T05:30:32.252049+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-13T05:30:19.156695+0000 front 2024-04-13T05:30:19.156379+0000 (oldest deadline 2024-04-13T05:30:43.856523+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-04-13T05:30:26.759640+0000 front 2024-04-13T05:30:26.759589+0000 (oldest deadline 2024-04-13T05:30:51.459477+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.83:6820 osd.7 since back 2024-04-13T05:30:34.961694+0000 front 2024-04-13T05:30:34.961600+0000 (oldest deadline 2024-04-13T05:31:03.761973+0000) 2024-04-13T05:31:16.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:16 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11[60576]: 2024-04-13T05:31:16.059+0000 7fd0c8400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-13T05:30:44.863440+0000 front 2024-04-13T05:30:44.863356+0000 (oldest deadline 2024-04-13T05:31:10.163172+0000) 2024-04-13T05:31:17.584 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:17 smithi189 podman[72431]: 2024-04-13 05:31:17.323533343 +0000 UTC m=+5.917644717 container died bab63ebb3039141b46f75bf7e72a46d3f05687c8805ee745c52e8d7799910118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-13T05:31:17.924 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:17 smithi189 podman[72431]: 2024-04-13 05:31:17.720260117 +0000 UTC m=+6.314371501 container cleanup bab63ebb3039141b46f75bf7e72a46d3f05687c8805ee745c52e8d7799910118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:17.924 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:17 smithi189 bash[72431]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11 2024-04-13T05:31:18.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:17 smithi189 podman[72452]: 2024-04-13 05:31:17.921195436 +0000 UTC m=+0.588360213 container remove bab63ebb3039141b46f75bf7e72a46d3f05687c8805ee745c52e8d7799910118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-13T05:31:19.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:18 smithi189 podman[72540]: 2024-04-13 05:31:18.684258608 +0000 UTC m=+0.033354911 image pull a4e4811815e4b21a2c2eb3d6af4424883cb2ff8e6a7dd48947e0a0b5759e4f05 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa 2024-04-13T05:31:19.083 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:18 smithi189 podman[72540]: 2024-04-13 05:31:18.789057046 +0000 UTC m=+0.138153391 container create 2c77871bfbd0458e7a63edb6c4d07cdd82d4c169903fdd92f23336b90118b091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, io.buildah.version=1.33.5) 2024-04-13T05:31:19.567 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:19 smithi189 podman[72540]: 2024-04-13 05:31:19.092906557 +0000 UTC m=+0.442002889 container init 2c77871bfbd0458e7a63edb6c4d07cdd82d4c169903fdd92f23336b90118b091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:19.567 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:19 smithi189 podman[72540]: 2024-04-13 05:31:19.103828587 +0000 UTC m=+0.452924898 container start 2c77871bfbd0458e7a63edb6c4d07cdd82d4c169903fdd92f23336b90118b091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-13T05:31:19.567 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:19 smithi189 podman[72540]: 2024-04-13 05:31:19.191315496 +0000 UTC m=+0.540411878 container attach 2c77871bfbd0458e7a63edb6c4d07cdd82d4c169903fdd92f23336b90118b091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:19.833 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 13 05:31:19 smithi189 podman[72540]: 2024-04-13 05:31:19.565102035 +0000 UTC m=+0.914198400 container died 2c77871bfbd0458e7a63edb6c4d07cdd82d4c169903fdd92f23336b90118b091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-13T05:31:20.282 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@osd.11.service' 2024-04-13T05:31:20.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:20.381 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-13T05:31:20.381 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-13T05:31:20.381 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.a 2024-04-13T05:31:20.783 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:20 smithi057 systemd[1]: Stopping Ceph mds.a for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:21.283 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a[75316]: 2024-04-13T05:31:21.015+0000 7f79b3e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:21.283 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:21 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a[75316]: 2024-04-13T05:31:21.015+0000 7f79b3e00700 -1 mds.a *** got signal Terminated *** 2024-04-13T05:31:23.252 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:22 smithi057 podman[170797]: 2024-04-13 05:31:22.954509462 +0000 UTC m=+2.073241559 container died 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-13T05:31:23.533 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:23 smithi057 podman[170797]: 2024-04-13 05:31:23.366877102 +0000 UTC m=+2.485609192 container cleanup 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-13T05:31:23.533 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:23 smithi057 bash[170797]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a 2024-04-13T05:31:24.032 INFO:journalctl@ceph.mds.a.smithi057.stdout:Apr 13 05:31:23 smithi057 podman[170812]: 2024-04-13 05:31:23.676435867 +0000 UTC m=+0.709125339 container remove 3c7383db5f10c854650208c3d4d4d5d12ff49aafd8184a600c72a8e66549736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-a, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-13T05:31:24.240 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.a.service' 2024-04-13T05:31:24.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:24.371 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-13T05:31:24.371 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-13T05:31:24.371 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.d 2024-04-13T05:31:24.782 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:24 smithi057 systemd[1]: Stopping Ceph mds.d for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:25.283 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d[74238]: 2024-04-13T05:31:24.976+0000 7f44f4000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:25.283 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:24 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d[74238]: 2024-04-13T05:31:24.976+0000 7f44f4000700 -1 mds.d *** got signal Terminated *** 2024-04-13T05:31:26.533 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:26 smithi057 podman[170917]: 2024-04-13 05:31:26.395028346 +0000 UTC m=+1.550842347 container died 5339d0a98846c613ac8c8f775b430a916b82a21be517e238dc3bb0be0c514cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-13T05:31:27.033 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:26 smithi057 podman[170917]: 2024-04-13 05:31:26.820942481 +0000 UTC m=+1.976756502 container cleanup 5339d0a98846c613ac8c8f775b430a916b82a21be517e238dc3bb0be0c514cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-13T05:31:27.033 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:26 smithi057 bash[170917]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d 2024-04-13T05:31:27.529 INFO:journalctl@ceph.mds.d.smithi057.stdout:Apr 13 05:31:27 smithi057 podman[170940]: 2024-04-13 05:31:27.06397403 +0000 UTC m=+0.653563015 container remove 5339d0a98846c613ac8c8f775b430a916b82a21be517e238dc3bb0be0c514cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-13T05:31:27.574 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.d.service' 2024-04-13T05:31:27.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:27.701 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-13T05:31:27.701 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-13T05:31:27.701 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.g 2024-04-13T05:31:28.283 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:27 smithi057 systemd[1]: Stopping Ceph mds.g for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:28.783 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-g[74947]: 2024-04-13T05:31:28.316+0000 7f6bcea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:28.783 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:28 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-g[74947]: 2024-04-13T05:31:28.316+0000 7f6bcea00700 -1 mds.g *** got signal Terminated *** 2024-04-13T05:31:31.215 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:30 smithi057 podman[171048]: 2024-04-13 05:31:30.888567811 +0000 UTC m=+2.692463524 container died b39c9f192169d1bde8db565a05113a085cd56e604a8ed7a2c7ec69344c96218d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-13T05:31:31.529 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:31 smithi057 podman[171048]: 2024-04-13 05:31:31.325019313 +0000 UTC m=+3.128915019 container cleanup b39c9f192169d1bde8db565a05113a085cd56e604a8ed7a2c7ec69344c96218d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409) 2024-04-13T05:31:31.529 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:31 smithi057 bash[171048]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-g 2024-04-13T05:31:31.783 INFO:journalctl@ceph.mds.g.smithi057.stdout:Apr 13 05:31:31 smithi057 podman[171062]: 2024-04-13 05:31:31.526098183 +0000 UTC m=+0.613332438 container remove b39c9f192169d1bde8db565a05113a085cd56e604a8ed7a2c7ec69344c96218d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-13T05:31:32.018 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.g.service' 2024-04-13T05:31:32.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:32.115 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-13T05:31:32.115 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-13T05:31:32.115 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.j 2024-04-13T05:31:32.533 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:32 smithi057 systemd[1]: Stopping Ceph mds.j for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:33.033 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j[73851]: 2024-04-13T05:31:32.726+0000 7efc4de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:33.033 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:32 smithi057 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j[73851]: 2024-04-13T05:31:32.726+0000 7efc4de00700 -1 mds.j *** got signal Terminated *** 2024-04-13T05:31:34.783 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:34 smithi057 podman[171168]: 2024-04-13 05:31:34.400566583 +0000 UTC m=+1.790400454 container died a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-13T05:31:35.190 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:34 smithi057 podman[171168]: 2024-04-13 05:31:34.91257566 +0000 UTC m=+2.302409524 container cleanup a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:31:35.191 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:34 smithi057 bash[171168]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j 2024-04-13T05:31:35.533 INFO:journalctl@ceph.mds.j.smithi057.stdout:Apr 13 05:31:35 smithi057 podman[171183]: 2024-04-13 05:31:35.188447932 +0000 UTC m=+0.768229010 container remove a7ae329c7c7466a4dc67b88ee0127e430ca5e1f6cc8f3a083027e3b9f25fa5f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:35.689 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.j.service' 2024-04-13T05:31:35.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:35.816 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-13T05:31:35.816 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-13T05:31:35.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.b 2024-04-13T05:31:36.251 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:35 smithi083 systemd[1]: Stopping Ceph mds.b for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:36.751 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: 2024-04-13T05:31:36.392+0000 7f98d7200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:36.752 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:36 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b[65839]: 2024-04-13T05:31:36.392+0000 7f98d7200700 -1 mds.b *** got signal Terminated *** 2024-04-13T05:31:38.926 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:38 smithi083 podman[71926]: 2024-04-13 05:31:38.664250147 +0000 UTC m=+2.412327508 container died 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-13T05:31:39.214 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:39 smithi083 podman[71926]: 2024-04-13 05:31:39.035948058 +0000 UTC m=+2.784025446 container cleanup 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:39.214 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:39 smithi083 bash[71926]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b 2024-04-13T05:31:39.501 INFO:journalctl@ceph.mds.b.smithi083.stdout:Apr 13 05:31:39 smithi083 podman[71952]: 2024-04-13 05:31:39.211890491 +0000 UTC m=+0.532132240 container remove 77b13aa5330fb2a25c77359a8d0a531109814cb2f0af3214ef390f06cdd2fc7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240412.042930-218a691, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-13T05:31:39.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.b.service' 2024-04-13T05:31:39.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:39.825 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-13T05:31:39.825 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-13T05:31:39.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.e 2024-04-13T05:31:40.251 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:39 smithi083 systemd[1]: Stopping Ceph mds.e for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:40.752 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e[65222]: 2024-04-13T05:31:40.390+0000 7f58df000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:40.752 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:40 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e[65222]: 2024-04-13T05:31:40.390+0000 7f58df000700 -1 mds.e *** got signal Terminated *** 2024-04-13T05:31:42.923 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:42 smithi083 podman[72060]: 2024-04-13 05:31:42.664118424 +0000 UTC m=+2.407997121 container died b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-13T05:31:43.251 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:43 smithi083 podman[72060]: 2024-04-13 05:31:43.131236321 +0000 UTC m=+2.875115012 container cleanup b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:31:43.252 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:43 smithi083 bash[72060]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e 2024-04-13T05:31:43.751 INFO:journalctl@ceph.mds.e.smithi083.stdout:Apr 13 05:31:43 smithi083 podman[72074]: 2024-04-13 05:31:43.332424173 +0000 UTC m=+0.648606456 container remove b1c3859e1ae6d96179437051cf6817de588d2f657fa08da53cda1c0358e2e1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-13T05:31:43.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.e.service' 2024-04-13T05:31:43.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:43.915 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-13T05:31:43.915 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-13T05:31:43.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.h 2024-04-13T05:31:44.501 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:44 smithi083 systemd[1]: Stopping Ceph mds.h for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:45.002 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:31:44.508+0000 7fb2d8600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:45.002 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:44 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h[65523]: 2024-04-13T05:31:44.508+0000 7fb2d8600700 -1 mds.h *** got signal Terminated *** 2024-04-13T05:31:46.919 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:46 smithi083 podman[72180]: 2024-04-13 05:31:46.621003968 +0000 UTC m=+2.268254544 container died 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:47.252 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:47 smithi083 podman[72180]: 2024-04-13 05:31:47.06028018 +0000 UTC m=+2.707530754 container cleanup 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-13T05:31:47.252 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:47 smithi083 bash[72180]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h 2024-04-13T05:31:47.751 INFO:journalctl@ceph.mds.h.smithi083.stdout:Apr 13 05:31:47 smithi083 podman[72194]: 2024-04-13 05:31:47.271318185 +0000 UTC m=+0.634854931 container remove 62c3fc99265c046b637dc382443847ff924aadd7bb9635b0dfc7cf8658a4029b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-13T05:31:47.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.h.service' 2024-04-13T05:31:47.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:47.911 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-13T05:31:47.911 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-13T05:31:47.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.k 2024-04-13T05:31:48.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:48 smithi083 systemd[1]: Stopping Ceph mds.k for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:48.964 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k[64914]: 2024-04-13T05:31:48.581+0000 7f3266200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:48.964 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:48 smithi083 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k[64914]: 2024-04-13T05:31:48.581+0000 7f3266200700 -1 mds.k *** got signal Terminated *** 2024-04-13T05:31:50.922 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:50 smithi083 podman[72301]: 2024-04-13 05:31:50.641895271 +0000 UTC m=+2.227769230 container died 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:31:51.227 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:51 smithi083 podman[72301]: 2024-04-13 05:31:51.040329749 +0000 UTC m=+2.626203668 container cleanup 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-13T05:31:51.227 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:51 smithi083 bash[72301]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k 2024-04-13T05:31:51.501 INFO:journalctl@ceph.mds.k.smithi083.stdout:Apr 13 05:31:51 smithi083 podman[72314]: 2024-04-13 05:31:51.224748069 +0000 UTC m=+0.567234517 container remove 313a3904677d29db8dfb532d8bfbf509c84e9692f8ee1c776f1d8488d57ea5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD) 2024-04-13T05:31:51.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.k.service' 2024-04-13T05:31:51.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:51.882 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-13T05:31:51.882 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-13T05:31:51.882 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.c 2024-04-13T05:31:52.306 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:52 smithi189 systemd[1]: Stopping Ceph mds.c for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:52.583 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:52 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c[65915]: 2024-04-13T05:31:52.517+0000 7fbcf7800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:52.583 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:52 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c[65915]: 2024-04-13T05:31:52.517+0000 7fbcf7800700 -1 mds.c *** got signal Terminated *** 2024-04-13T05:31:54.907 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:54 smithi189 podman[72659]: 2024-04-13 05:31:54.631264817 +0000 UTC m=+2.334271698 container died adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-13T05:31:55.264 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:55 smithi189 podman[72659]: 2024-04-13 05:31:55.06901796 +0000 UTC m=+2.772024901 container cleanup adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-13T05:31:55.264 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:55 smithi189 bash[72659]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c 2024-04-13T05:31:55.583 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 13 05:31:55 smithi189 podman[72684]: 2024-04-13 05:31:55.261630325 +0000 UTC m=+0.611525990 container remove adfde2f770c85101848394b0a1cddda10d01535cf2d2dce80bb602f22a740310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240412.042930-218a691) 2024-04-13T05:31:55.728 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.c.service' 2024-04-13T05:31:55.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:55.854 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-13T05:31:55.854 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-13T05:31:55.855 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.f 2024-04-13T05:31:56.333 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:55 smithi189 systemd[1]: Stopping Ceph mds.f for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:31:56.833 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f[65600]: 2024-04-13T05:31:56.452+0000 7f15e4400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:31:56.833 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:56 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f[65600]: 2024-04-13T05:31:56.452+0000 7f15e4400700 -1 mds.f *** got signal Terminated *** 2024-04-13T05:31:58.833 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:58 smithi189 podman[72791]: 2024-04-13 05:31:58.542926486 +0000 UTC m=+2.212366576 container died 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:31:59.184 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:58 smithi189 podman[72791]: 2024-04-13 05:31:58.989331621 +0000 UTC m=+2.658771738 container cleanup 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-13T05:31:59.184 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:58 smithi189 bash[72791]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f 2024-04-13T05:31:59.583 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 13 05:31:59 smithi189 podman[72805]: 2024-04-13 05:31:59.18200616 +0000 UTC m=+0.622182931 container remove 3c258f412145c1ef4c72323f2aaa9f6a3a5fa5a9cf878ac1a29f75122819f1e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-13T05:31:59.627 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.f.service' 2024-04-13T05:31:59.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:31:59.741 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-13T05:31:59.741 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-13T05:31:59.741 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.i 2024-04-13T05:32:00.333 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:31:59 smithi189 systemd[1]: Stopping Ceph mds.i for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:32:00.833 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:32:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i[66223]: 2024-04-13T05:32:00.340+0000 7f019cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:32:00.833 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:32:00 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i[66223]: 2024-04-13T05:32:00.340+0000 7f019cc00700 -1 mds.i *** got signal Terminated *** 2024-04-13T05:32:02.932 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:32:02 smithi189 podman[72913]: 2024-04-13 05:32:02.632479384 +0000 UTC m=+2.409007329 container died 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-13T05:32:03.333 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:32:03 smithi189 podman[72913]: 2024-04-13 05:32:03.051240228 +0000 UTC m=+2.827768216 container cleanup 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-13T05:32:03.333 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:32:03 smithi189 bash[72913]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i 2024-04-13T05:32:03.833 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 13 05:32:03 smithi189 podman[72927]: 2024-04-13 05:32:03.343756833 +0000 UTC m=+0.701037548 container remove 30c89f050b750470e430c3b404164dbff4ed3defcceccef7725b6ad7428e84cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-13T05:32:03.899 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.i.service' 2024-04-13T05:32:04.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:32:04.013 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-13T05:32:04.013 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-13T05:32:04.013 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.l 2024-04-13T05:32:04.583 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:04 smithi189 systemd[1]: Stopping Ceph mds.l for 0a178454-f953-11ee-bc8d-c7b262605968... 2024-04-13T05:32:05.014 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l[65301]: 2024-04-13T05:32:04.601+0000 7f393f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-13T05:32:05.014 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:04 smithi189 ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l[65301]: 2024-04-13T05:32:04.601+0000 7f393f800700 -1 mds.l *** got signal Terminated *** 2024-04-13T05:32:06.662 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:06 smithi189 podman[73032]: 2024-04-13 05:32:06.343192911 +0000 UTC m=+1.880517648 container died 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-13T05:32:07.016 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:06 smithi189 podman[73032]: 2024-04-13 05:32:06.779709976 +0000 UTC m=+2.317034733 container cleanup 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240412.042930-218a691, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True) 2024-04-13T05:32:07.016 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:06 smithi189 bash[73032]: ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l 2024-04-13T05:32:07.333 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 13 05:32:07 smithi189 podman[73045]: 2024-04-13 05:32:07.013822394 +0000 UTC m=+0.660321874 container remove 1775acccbbd041a490fe290954e826095211959db1327dd57fe53469bfae5f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b2bdb0d9d0471afa314bb0b5ecd318ce61e4dda086bd028e6a35664d56dc32aa, name=ceph-0a178454-f953-11ee-bc8d-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240412.042930-218a691, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-13T05:32:07.534 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a178454-f953-11ee-bc8d-c7b262605968@mds.l.service' 2024-04-13T05:32:07.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-13T05:32:07.650 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-13T05:32:07.650 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --force --keep-logs 2024-04-13T05:32:07.993 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:32:13.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --force --keep-logs 2024-04-13T05:32:13.442 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:32:18.304 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --force --keep-logs 2024-04-13T05:32:18.632 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:32:23.481 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:32:23.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:32:23.693 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-13T05:32:23.791 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-13T05:32:23.793 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi057/crash 2024-04-13T05:32:23.794 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash -- . 2024-04-13T05:32:23.887 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash: Cannot open: No such file or directory 2024-04-13T05:32:23.888 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2024-04-13T05:32:23.891 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi083/crash 2024-04-13T05:32:23.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash -- . 2024-04-13T05:32:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash: Cannot open: No such file or directory 2024-04-13T05:32:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-04-13T05:32:24.026 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi189/crash 2024-04-13T05:32:24.027 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash -- . 2024-04-13T05:32:24.134 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/0a178454-f953-11ee-bc8d-c7b262605968/crash: Cannot open: No such file or directory 2024-04-13T05:32:24.134 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-04-13T05:32:24.139 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-13T05:32:24.139 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-13T05:32:24.256 INFO:teuthology.orchestra.run.smithi057.stdout:2024-04-13T05:24:46.284907+0000 mds.h (mds.1) 26 : cluster [WRN] Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:32:24.257 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-13T05:32:24.258 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[SEC\]' /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-13T05:32:24.352 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]' /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-13T05:32:24.450 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[WRN\]' /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-13T05:32:24.538 INFO:teuthology.orchestra.run.smithi057.stdout:2024-04-13T05:24:46.284907+0000 mds.h (mds.1) 26 : cluster [WRN] Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details 2024-04-13T05:32:24.539 INFO:tasks.cephadm:Compressing logs... 2024-04-13T05:32:24.539 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-13T05:32:24.543 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-13T05:32:24.551 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-13T05:32:24.607 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-13T05:32:24.607 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-13T05:32:24.608 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-13T05:32:24.609 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-13T05:32:24.610 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.6.log 2024-04-13T05:32:24.611 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.a.log 2024-04-13T05:32:24.612 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log 2024-04-13T05:32:24.613 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.a.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-13T05:32:24.614 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.g.log 2024-04-13T05:32:24.615 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.j.log 2024-04-13T05:32:24.615 INFO:teuthology.orchestra.run.smithi057.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-13T05:32:24.616 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.3.log 2024-04-13T05:32:24.618 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.x.log 2024-04-13T05:32:24.619 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.3.log: 90.0% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log.gz 2024-04-13T05:32:24.622 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.x.log: gzip 94.6% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.g.log.gz -5 2024-04-13T05:32:24.622 INFO:teuthology.orchestra.run.smithi057.stderr: --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.0.log 2024-04-13T05:32:24.623 INFO:teuthology.orchestra.run.smithi057.stderr: 94.6% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.j.log.gz 2024-04-13T05:32:24.623 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log 2024-04-13T05:32:24.625 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log 2024-04-13T05:32:24.626 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log.gz 2024-04-13T05:32:24.628 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-13T05:32:24.629 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log 2024-04-13T05:32:24.629 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-13T05:32:24.631 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.9.log 2024-04-13T05:32:24.631 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.l.log 2024-04-13T05:32:24.632 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log 2024-04-13T05:32:24.633 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.d.log 2024-04-13T05:32:24.633 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.l.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-13T05:32:24.633 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.i.log 2024-04-13T05:32:24.635 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.11.log 2024-04-13T05:32:24.636 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.z.log 2024-04-13T05:32:24.637 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.9.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.d.log: 87.1% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log.gz 2024-04-13T05:32:24.637 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.a.log 2024-04-13T05:32:24.637 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.8.log 2024-04-13T05:32:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-13T05:32:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-04-13T05:32:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.10.log 2024-04-13T05:32:24.640 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.z.log: 90.2% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log.gz 2024-04-13T05:32:24.640 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.8.log: 94.5% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.l.log.gz 2024-04-13T05:32:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log 2024-04-13T05:32:24.641 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.5.log 2024-04-13T05:32:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.10.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-13T05:32:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.b.log 2024-04-13T05:32:24.642 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.f.log 2024-04-13T05:32:24.643 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.4.log 2024-04-13T05:32:24.645 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.5.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log 2024-04-13T05:32:24.645 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.1.log 2024-04-13T05:32:24.646 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.k.log 2024-04-13T05:32:24.647 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- 93.0% /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log 2024-04-13T05:32:24.647 INFO:teuthology.orchestra.run.smithi189.stderr: -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.z.log.gz 2024-04-13T05:32:24.648 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log.gz 2024-04-13T05:32:24.648 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.h.log 2024-04-13T05:32:24.648 INFO:teuthology.orchestra.run.smithi083.stderr: 90.2% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.audit.log.gz 2024-04-13T05:32:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.d.log.gz 90.8% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log.gz 2024-04-13T05:32:24.649 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-04-13T05:32:24.650 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.2.log 2024-04-13T05:32:24.651 INFO:teuthology.orchestra.run.smithi057.stderr: 94.7% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.a.log.gz 2024-04-13T05:32:24.651 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.k.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.7.log 2024-04-13T05:32:24.653 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.c.log 2024-04-13T05:32:24.653 INFO:teuthology.orchestra.run.smithi189.stderr: 94.6% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.f.log.gz 2024-04-13T05:32:24.654 INFO:teuthology.orchestra.run.smithi189.stderr: 87.2%/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.c.log: -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log.gz 2024-04-13T05:32:24.654 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log 2024-04-13T05:32:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.e.log 2024-04-13T05:32:24.655 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.y.log 2024-04-13T05:32:24.657 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.c.log 2024-04-13T05:32:24.660 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.y.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log 2024-04-13T05:32:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.cephadm.log.gz 2024-04-13T05:32:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log 2024-04-13T05:32:24.668 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log 2024-04-13T05:32:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log: /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.b.log 2024-04-13T05:32:24.677 INFO:teuthology.orchestra.run.smithi083.stderr: 87.1% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph.log.gz 2024-04-13T05:32:24.679 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.b.log: 92.8% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.y.log.gz 2024-04-13T05:32:24.690 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log.gz 2024-04-13T05:32:24.695 INFO:teuthology.orchestra.run.smithi083.stderr: 90.8% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-volume.log.gz 2024-04-13T05:32:24.969 INFO:teuthology.orchestra.run.smithi057.stderr: 90.3% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mgr.x.log.gz 2024-04-13T05:32:25.212 INFO:teuthology.orchestra.run.smithi189.stderr: 91.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.c.log.gz 2024-04-13T05:32:25.445 INFO:teuthology.orchestra.run.smithi083.stderr: 91.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.b.log.gz 2024-04-13T05:32:26.112 INFO:teuthology.orchestra.run.smithi057.stderr: 90.4% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mon.a.log.gz 2024-04-13T05:32:27.256 INFO:teuthology.orchestra.run.smithi189.stderr: 93.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.c.log.gz 2024-04-13T05:32:27.926 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.k.log.gz 2024-04-13T05:32:31.571 INFO:teuthology.orchestra.run.smithi189.stderr: 94.1% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.i.log.gz 2024-04-13T05:32:32.311 INFO:teuthology.orchestra.run.smithi189.stderr: 91.8% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.11.log.gz 2024-04-13T05:32:33.667 INFO:teuthology.orchestra.run.smithi189.stderr: 91.2% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.8.log.gz 2024-04-13T05:32:33.904 INFO:teuthology.orchestra.run.smithi083.stderr: 91.7% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.10.log.gz 2024-04-13T05:32:33.965 INFO:teuthology.orchestra.run.smithi057.stderr: 91.4% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.3.log.gz 2024-04-13T05:32:34.741 INFO:teuthology.orchestra.run.smithi083.stderr: 91.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.1.log.gz 2024-04-13T05:32:34.924 INFO:teuthology.orchestra.run.smithi189.stderr: 91.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.2.log.gz 2024-04-13T05:32:35.260 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.7.log.gz 2024-04-13T05:32:35.925 INFO:teuthology.orchestra.run.smithi057.stderr: 91.6% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.9.log.gz 2024-04-13T05:32:36.208 INFO:teuthology.orchestra.run.smithi057.stderr: 91.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.0.log.gz 2024-04-13T05:32:37.202 INFO:teuthology.orchestra.run.smithi189.stderr: 91.6% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.5.log.gz 2024-04-13T05:32:37.206 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-04-13T05:32:37.206 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m12.610s 2024-04-13T05:32:37.206 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m46.179s 2024-04-13T05:32:37.206 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m3.503s 2024-04-13T05:32:37.304 INFO:teuthology.orchestra.run.smithi057.stderr: 91.5% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.6.log.gz 2024-04-13T05:32:38.593 INFO:teuthology.orchestra.run.smithi083.stderr: 91.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-osd.4.log.gz 2024-04-13T05:33:07.634 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.b.log.gz 2024-04-13T05:33:08.787 INFO:teuthology.orchestra.run.smithi083.stderr: 93.9% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.e.log.gz 2024-04-13T05:33:52.341 INFO:teuthology.orchestra.run.smithi083.stderr: 93.5% -- replaced with /var/log/ceph/0a178454-f953-11ee-bc8d-c7b262605968/ceph-mds.h.log.gz 2024-04-13T05:33:52.345 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-13T05:33:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:real 1m27.749s 2024-04-13T05:33:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:user 3m25.642s 2024-04-13T05:33:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m15.652s 2024-04-13T05:34:17.956 INFO:teuthology.orchestra.run.smithi057.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-13T05:34:17.960 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-04-13T05:34:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:real 1m53.386s 2024-04-13T05:34:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:user 2m28.289s 2024-04-13T05:34:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m10.484s 2024-04-13T05:34:17.962 INFO:tasks.cephadm:Archiving logs... 2024-04-13T05:34:17.962 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi057/log 2024-04-13T05:34:17.963 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-13T05:34:40.848 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi083/log 2024-04-13T05:34:40.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-13T05:35:08.642 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi189/log 2024-04-13T05:35:08.643 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-13T05:35:15.287 INFO:tasks.cephadm:Removing cluster... 2024-04-13T05:35:15.287 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --force 2024-04-13T05:35:15.640 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:35:16.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --force 2024-04-13T05:35:16.660 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:35:17.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a178454-f953-11ee-bc8d-c7b262605968 --force 2024-04-13T05:35:18.168 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 0a178454-f953-11ee-bc8d-c7b262605968 2024-04-13T05:35:18.935 INFO:tasks.cephadm:Removing cephadm ... 2024-04-13T05:35:18.935 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-13T05:35:18.999 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-13T05:35:19.057 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-13T05:35:19.115 INFO:tasks.cephadm:Teardown complete 2024-04-13T05:35:19.115 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-13T05:35:19.133 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-13T05:35:19.133 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-13T05:35:19.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-13T05:35:19.149 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-13T05:35:19.318 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-13T05:35:19.328 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-13T05:35:19.329 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T05:35:19.337 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T05:35:19.344 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-13T05:35:19.388 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-04-13T05:35:19.391 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-13T05:35:19.398 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T05:35:19.398 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-04-13T05:35:19.398 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +505us[ +533us] +/- 39ms 2024-04-13T05:35:19.398 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1451us[-1451us] +/- 65ms 2024-04-13T05:35:19.398 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +805us[ +832us] +/- 71ms 2024-04-13T05:35:19.399 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T05:35:19.400 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-04-13T05:35:19.401 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T05:35:19.401 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-13T05:35:19.401 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 7 377 115 +592us[ +711us] +/- 38ms 2024-04-13T05:35:19.401 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 114 -1426us[-1426us] +/- 64ms 2024-04-13T05:35:19.401 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +851us[ +851us] +/- 71ms 2024-04-13T05:35:19.401 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T05:35:19.410 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-13T05:35:19.410 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-04-13T05:35:19.410 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +522us[ +520us] +/- 39ms 2024-04-13T05:35:19.411 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -1469us[-1469us] +/- 65ms 2024-04-13T05:35:19.411 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +807us[ +805us] +/- 71ms 2024-04-13T05:35:19.411 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-13T05:35:19.412 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-13T05:35:19.423 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-13T05:35:19.423 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-13T05:35:19.433 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-13T05:35:19.677 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-13T05:35:19.895 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-13T05:35:20.124 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T05:35:20.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T05:35:20.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T05:35:20.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T05:35:20.336 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-13T05:35:20.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T05:35:20.439 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-13T05:35:20.451 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-13T05:35:20.461 INFO:teuthology.task.internal:Duration was 2534.626918 seconds 2024-04-13T05:35:20.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-13T05:35:20.472 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-13T05:35:20.473 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-13T05:35:20.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-13T05:35:20.488 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-13T05:35:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T05:35:20.655 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T05:35:20.656 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-13T05:35:21.129 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-13T05:35:21.129 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2024-04-13T05:35:21.129 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-13T05:35:21.199 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-13T05:35:21.200 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-13T05:35:21.274 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-04-13T05:35:21.275 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-13T05:35:21.351 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-13T05:35:21.351 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-13T05:35:21.359 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-13T05:35:21.367 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-13T05:35:21.463 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-13T05:35:21.463 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-13T05:35:21.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-13T05:35:21.474 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-13T05:35:21.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-13T05:35:22.005 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-13T05:35:22.005 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-13T05:35:22.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-13T05:35:22.208 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-13T05:35:22.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-13T05:35:22.322 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-13T05:35:22.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-13T05:35:22.338 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-13T05:35:22.438 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-13T05:35:22.440 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2024-04-13T05:35:22.446 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-04-13T05:35:22.511 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-13T05:35:22.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T05:35:22.568 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-13T05:35:22.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T05:35:22.625 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-13T05:35:22.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-13T05:35:22.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-13T05:35:22.709 INFO:teuthology.task.internal:Transferring archived files... 2024-04-13T05:35:22.711 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi057 2024-04-13T05:35:22.711 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-13T05:35:22.853 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi083 2024-04-13T05:35:22.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-13T05:35:22.964 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333/remote/smithi189 2024-04-13T05:35:22.964 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-13T05:35:23.088 INFO:teuthology.task.internal:Removing archive directory... 2024-04-13T05:35:23.088 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-13T05:35:23.097 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-13T05:35:23.107 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-13T05:35:23.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-13T05:35:23.184 INFO:teuthology.task.internal:Not uploading archives. 2024-04-13T05:35:23.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-13T05:35:23.213 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-13T05:35:23.213 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-13T05:35:23.221 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-13T05:35:23.228 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-13T05:35:23.270 INFO:teuthology.orchestra.run.smithi057.stdout: 524694 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 13 05:35 /home/ubuntu/cephtest 2024-04-13T05:35:23.278 INFO:teuthology.orchestra.run.smithi083.stdout: 395174 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 13 05:35 /home/ubuntu/cephtest 2024-04-13T05:35:23.286 INFO:teuthology.orchestra.run.smithi189.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 13 05:35 /home/ubuntu/cephtest 2024-04-13T05:35:23.289 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-13T05:35:23.302 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-13T05:35:23.328 INFO:teuthology.nuke:Checking targets against current locks 2024-04-13T05:35:23.354 DEBUG:teuthology.nuke:shortname: smithi057 2024-04-13T05:35:23.354 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T05:35:23.379 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 04:37:44.318644', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDL696pxUtIvSGsIC5IxNAufcSOOrHmaypFhy12QJcliW3277EYfme8HPSz4bhfjHSWFpZvvsGViIy4z7lNnLJs='} 2024-04-13T05:35:23.383 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-13T05:35:23.383 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T05:35:23.406 DEBUG:teuthology.nuke:shortname: smithi189 2024-04-13T05:35:23.406 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-13T05:35:23.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 04:37:44.316080', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHIz3tzAlZi3o6XF3NbEaoHuU43YGUfsQ23hf5io0UwzEQQzcIUp+cgc0KV9Qy2JpI/rXST96idmIhDJXVrDXsY='} 2024-04-13T05:35:23.435 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_17:34:28-fs-wip-vshankar-testing-20240412.042930-testing-default-smithi/7654333', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-13 04:37:44.317472', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ8rd9bLFYlXuasK4uvpAMOtj6Mb1e9zAaiS8Ec5MQ8iMZrrPVl4kEHlM0/Zm4qyDwYipskKnBus6+r+G07mIE='} 2024-04-13T05:35:23.441 INFO:teuthology.orchestra.console.smithi057:Power off 2024-04-13T05:35:23.442 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T05:35:23.466 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2024-04-13T05:35:23.467 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:23.486 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-13T05:35:23.486 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T05:35:23.498 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2024-04-13T05:35:23.507 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-13T05:35:23.507 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:23.528 INFO:teuthology.orchestra.console.smithi189:Power off 2024-04-13T05:35:23.528 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T05:35:23.538 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-13T05:35:23.551 DEBUG:teuthology.orchestra.console.smithi189:power off output: Chassis Power Control: Down/Off 2024-04-13T05:35:23.551 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:23.576 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-13T05:35:27.499 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:27.539 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:27.577 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:27.610 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2024-04-13T05:35:27.649 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-13T05:35:27.687 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-13T05:35:31.614 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:31.650 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:31.688 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:31.724 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2024-04-13T05:35:31.759 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-13T05:35:31.799 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-13T05:35:35.728 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:35.761 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:35.799 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T05:35:35.838 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2024-04-13T05:35:35.871 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-13T05:35:35.910 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is off 2024-04-13T05:35:35.939 INFO:teuthology.orchestra.console.smithi057:Power off completed 2024-04-13T05:35:35.972 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-13T05:35:36.010 INFO:teuthology.orchestra.console.smithi189:Power off completed 2024-04-13T05:35:36.116 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} duration: 2534.6269178390503 failure_reason: '"2024-04-13T05:24:46.284907+0000 mds.h (mds.1) 26 : cluster [WRN] Scrub error on inode 0x20000001a4d (/volumes/qa/sv_0/695f39a4-c9d5-4816-845e-3fe961904a07/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-44) see mds.h log and `damage ls` output for details" in cluster log' flavor: default mon.a-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 mon.b-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 mon.c-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 owner: scheduled_vshankar@teuthology success: false 2024-04-13T05:35:36.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T05:35:36.195 INFO:teuthology.run:FAIL